earGram Actors: An Interactive Audiovisual System Based on Social Behavior

Keywords: Audiovisual system, Distributed agent system, Emergence, Complexity, Concatenative sound synthesis, Interactive musical improvisation


In multi-agent systems, local interactions among system components following relatively simple rules often result in complex overall systemic behavior. Complex behavioral and morphological patterns have been used to generate and organize audiovisual systems with artistic purposes. In this work, we propose to use the Actor model of social interactions to drive a concatenative synthesis engine called earGram in real time. The Actor model was originally developed to explore the emergence of complex visual patterns. On the other hand, earGram was originally developed to facilitate the creative exploration of concatenative sound synthesis. The integrated audiovisual system allows a human performer to interact with the system dynamics while receiving visual and auditory feedback. The interaction happens indirectly by disturbing the rules governing the social relationships amongst the actors, which results in a wide range of dynamic spatiotemporal patterns. A performer thus improvises within the behavioural scope of the system while evaluating the apparent connections between parameter values and actual complexity of the system output.


Bak, P. (1996) How Nature Works: The Science of Self-Organized Criticality. New York: Springer.

Bernardes, G. (2014) Composing Music by Selection: Content-Based Algorithmic-Assisted Audio Composition. PhD dissertation, University of Porto.

Bernardes, G., Guedes, C. and Pennycook, B. (2013). Eargram: An application for interactive exploration of concatenative sound synthesis in Pure Data. In M. Aramaki, M. Barthet, R. Kronland-Martinet and S. Ystad, (eds), From Sounds to Music and Emotions, LNCS, 7900, 110-129. Berlin-Heidelberg: Springer.

Beyls, P. A Molecular Collision Model of Musical Interaction. (2005). In C. Soddu (ed.), Proceedings of the Generative Arts Conference, Milan, Italy.

Beyls, P. (2010). Structural Coupling in a Musical Agency. In E. Miranda (ed.), Artificial Life and Music, Evanston, WI: A-R Editions.

Beyls, P. (2012). Autonomy, Influence and Emergence in an Audiovisual Ecosystem. In C. Soddu (ed.), Proceedings of the Generative Arts Conference, Rome, Italy, 2012.

Blackwell, T., Bentley, P. (2002). Improvised music with swarms. In Proceedings of the 2002 Congress on Evolutionary Computation, 2, 1462-1467.

Borgo, D. (2005). Sync or Swarm: Improvising Music in a Complex Age. Continuum. New York and London.

Caetano, M., Manzolli, J. Von Zuben, F. (2007). Self-Organizing Bio-Inspired Sound Transformation. Applications of Evolutionary Computing. LNCS, 4448, 477-487.

Camazine, S., Deneubourg, J-L, Franks, N., Sneyd, J., Theraulaz, G., Bonabeau, E. (2003). Self-Organization in Biological Systems. Princeton Studies in Complexity.

Casey, M. (2009). Soundspotting: A new kind of process? In R. Dean (ed.), The Oxford Handbook of Computer Music. New York, NY: Oxford University Press.

Chadabe, J. (1996). Electric Sound, The Past and promise of Electronic Music. Upper Saddle River, NJ: Prentice-Hall.

Dittrich, Ziegler & Banzhaf. (2001). Artificial chemistries, A review. Artificial Life, 7(3), 225-275.

Gold, R. (2007). The Plenitude. MIT Press, Cambirdge, MA.

Kandogan, E. (2000). Star coordinates: A multi-dimensional visualization technique with uniform treatment of dimensions. In Proceedings of the IEEE Information Visualization Symposium.

Kaufmann, S. (1995). At Home in the Universe: The Search for the Laws of Self-Organization and Complexity. Oxford University Press.

Kim, H.-G., Moreau, N., & Sikora, T. (2005). MPEG-7 Audio and Beyond: Audio Content Indexing and Retrieval. Chichester, UK: John Wiley & Sons.

Langton, C. (1997). Artificial Life: An Overview. Cambridge, MA: The MIT Press.

Lewin, R. (1992). Life at the Edge of Chaos. The University of Chicago Press, Chicago, IL.

Miranda, E. (1994). Music composition using cellular automata. Languages of Design, 2.

Reynolds, G. (1987). Flocks, Herds and Schools: A Distributed Behavioral Model. In SIGGRAPH 87 Conference Proceedings, Anaheim, CA.

Schmeder, A., Freed, A., and Wessel, D. (2010). Best Practices for Open Sound Control. In Linux Audio Conference, Utrecht, Holland.

Schwarz, D. (2000). A system for data-driven concatenative sound synthesis. In Proceedings of the International Conference on Digital Audio Effects, 97-102.

Tidemann, A., Ozturk, P. (2007). Self-organizing Multiple Models for Imitation: Teaching a Robot to Dance the YMCA. New Trends in Applied Artificial Intelligence, LNCS, 291-302.

Todd, S., Latham, W. (1992). Evolutionary Art and Computers. Academic Press.

Ulanowicz, R. (1979). Complexity, Stability, and Self-Organization in Natural Communities. Oecologia (Berl.), (43), 295-298.

Waldrop, M. (1994). Complexity, The Emerging Science at the Edge of Order and Chaos. Penguin Science Books, London, UK.