MediaScape: Towards a Video, Music, and Sound Metacreation

Arne Eigenfeldt; Miles Thorogood; Jim Bizzocchi; Philippe Pasquier

doi:10.7559/citarj.v6i1.129

Arne Eigenfeldt School of Contemporary Arts, Simon Fraser University | Vancouver | Canada
Miles Thorogood School of Interactive Arts and Technology, Simon Fraser University | Surrey | Canada
Jim Bizzocchi School of Interactive Arts and Technology, Simon Fraser University | Surrey | Canada
Philippe Pasquier School of Interactive Arts and Technology, Simon Fraser University | Surrey | Canada

DOI: https://doi.org/10.7559/citarj.v6i1.129

Keywords: Generative video, generative music, sound art, metacreation.

Abstract

We present a new media work, MediaScape, which is an initial foray into a fully interdisciplinary metacreativity. This paper defines metacreation, and we present examples of metacreative art within the fields of music, sound art, the history of generative narrative, and discuss the potential of the “open-documentary” as an immediate goal of metacreative video. Lastly, we describe MediaScape in detail, and present some future directions.

Author Biographies

Arne Eigenfeldt, School of Contemporary Arts, Simon Fraser University | Vancouver | Canada

Arne Eigenfeldt is a composer of live electroacoustic music, and a researcher into intelligent generative music systems. His music has been performed around the world, and his collaborations range from Persian Tar masters to contemporary dance companies to musical robots. He has presented his research at conferences and festivals such as ICMC, SMC, ICCC, EMS, EvoMusArt, GECCO, and NIME. He is a professor of music technology at Simon Fraser University, and is the co-director of the MAMAS (Metacreation Agent and Multi-Agent Systems) Lab.

Miles Thorogood, School of Interactive Arts and Technology, Simon Fraser University | Surrey | Canada

Miles Thorogood is a doctoral researcher at the School of Interactive Art and Technology, Simon Fraser University, exploring relationships between machine learning, artificial intelligence, soundscape, and creativity. His works have encompassed sound installation, locative media, and variety of generative a/v manifestations shown in Australia, Europe and North America. He has developed and run workshops and classes for creative coding in Canada and Australia. Published works include thematics of multi-agent systems, electronics hacking, artificial life, and generative systems. As well, he has been published internationally in leading technology and arts magazines.

Jim Bizzocchi, School of Interactive Arts and Technology, Simon Fraser University | Surrey | Canada

Jim Bizzocchi is a filmmaker currently working in video art and installation. His research interests include the aesthetics and design of the moving image, interactive narrative, and the development of computational video sequencing systems. He is interested in the effect of new technologies on cinematic visual expressions such as split-screens, layered imagery, image transitions, and stereoscopic cinema. His work in interactive narrative includes ongoing extension of the computational capabilities of his Re:Cycle video presentation and sequencing system.

Philippe Pasquier, School of Interactive Arts and Technology, Simon Fraser University | Surrey | Canada

Philippe Pasquier is Associate Professor at Simon Fraser University's School of Interactive Arts and Technology. He is both a scientist specialized in artificial intelligence and a multi-disciplinary artist. His contributions range from theoretical research in artificial intelligence, multi-agent systems and machine learning to applied artistic research and practice in digital art, computer music, and generative art. Philippe is a Chair and investigator of the AAAI series of international workshop on Musical Metacreation (MUME), the MUME-WE concerts series and the International workshop on Movement and Computation (MOCO), and the director of ISEA2015. He has co-authored over 100 peer-reviewed contributions, presented in forums ranging from the most scientifically rigorous to the most creatively arty.

References

Bandt, R. (2001). Sound Sculpture: Intersections in Sound and Sculpture in Australian Artworks. Craftsman House, Fine Art Publishing Limited.

Bizzocchi, J. (2007). Games and Narrative: An Analytical Framework. Loading: the Journal of the Canadian Game Studies Association, 1:1.

Bizzocchi, J. (2008) Ambient Video: The Transformation of the Domestic Cinematic Experience, in Small Tech: The Culture of Digital Tools, eds. Byron Hawk, David Rieder, and Ollie Oviedo. University of Minnesota Press, Minneapolis, MN.

Bizzocchi, J. (2011). Re: Cycle – A Generative Ambient Video Engine. Entertainment Computing – ICEC, Springer.

Bizzocchi, J., & Tanenbaum, J. (2012). Mass Effect 2 A Case Study in the Design of Game Narrative. Bulletin of Science, Technology & Society 32:5. http://dx.doi.org/10.1177/0270467612463796

Bootz, P. (2012). From OULIPO to Transitoire Observable The Evolution of French Digital Poetry. http://www.dichtung-digital.de/en/journal/archiv/?postID=248, retrieved January 14, 2014.

Burroughs, W. S., & Gysin, B. (1978).The Third Mind. Viking Press.

Cohen, H.(1995). The further exploits of AARON, painter. Stanford Humanities Review 4:2.

Cope, D. (1991). Computers and Musical Style. Madison, WI: A-R Editions.

Draves, S. (2008). Evolution and Collective Intelligence of the Electric Sheep. The Art of Artificial Evolution. Springer. http://dx.doi.org/10.1007/978-3-540-72877-1_3

Dubois, D., Guastavino, C., Raimbault, M. (2006). A cognitive approach to urban soundscapes: Using verbal data to access everyday life auditory categories. Acta Acustica united with Acustica, 92(6).

Eigenfeldt, A. (2008). Emergent rhythms through multi-agency in Max/MSP. Computer Music Modeling and Retrieval. Sense of Sounds. Springer. http://dx.doi.org/10.1007/978-3-540-85035-9_26

Eigenfeldt, A. (2011). Real-time composition as performance ecosystem. Organised Sound 16:2. http://dx.doi.org/10.1017/S1355771811000094

Eigenfeldt, A. (2012). More Than Four. Performed at Goldcorp Centre for the Arts, Vancouver, May 25 2012. http://youtu.be/ao_B0eW3gDI

Eigenfeldt, A., & Bahn, C. (2009). Bhatik. Performed at Western Front, Vancouver, April 13 2009. http://youtu.be/R-DQ5WPd4ao

Eigenfeldt, A., & Pasquier, P. (2010) Realtime Generation of Harmonic Progressions Using Constrained Markov Selection. Proceedings of the First International Conference on Computational Creativity, Lisbon.

Eigenfeldt, A, & Pasquier, P. (2011). Negotiated Content: Generative Soundscape Composition by Autonomous Musical Agents in Coming Together: Freesound. Proceedings of the Second International Conference on Computational Creativity, Mexico City.

Eigenfeldt, A., & Pasquier, P. (2012). Creative Agents, Curatorial Agents, and Human-Agent Interaction in Coming Together. Proceedings of Sound and Music Computing, Copenhagen.

Fargher, C, & Narushima, T. (2008). Knit Two Together: Art/Science Collaborations in BioHome. The Chromosome Knitting Project. The Scientific Imaginary in Visual Culture 5:181.

Galanter, P. (2003). What is Generative Art? Complexity theory as a context for art theory. GA2003–6th Generative Art Conference, Milan.

Gooding, M. (1995). A book of surrealist games. Boston, MA: Shambhala Redstone Editions.

Gunning, T. The Cinema of Attractions in "Early Cinema: Space, Frame, Narrative", ed. Thomas Elsaesser, BFI Publishing, London, 1990

Harrell, F. (2007). GRIOT's Tales of Haints and Seraphs: A Computational Narrative Generation System. Second Person: Role-Playing and Story in Games and Playable Media. MIT Press.

Harrell, F., & Chow, K. (2008). Generative visual renku: linked poetry generation with the griot system. Proceedings, electronic literature organization conference, Vancouver.

Hatala, M., Wakkary, R., & Kalantari L. (2005). Rules and Ontologies in Support of Real-Time Ubiquitous Application. Journal of Web Semantics 3.

Hayles, K. (2007). Electronic Literature: What is it? http://eliterature.org/pad/elp.html, retreived January 14, 2014.

Hiller, L., & Isaacson, L. (1979). Experimental Music; Composition with an Electronic Computer. Greenwood Publishing Group Inc.

Ihmels, T., & Riedel, J. (2007). Die Methodik der generativen Kunst.

Kapur, A., Eigenfeldt, A., Bahn, C., & Schloss, W. (2009). Collaborative composition for musical robots. Journal of Science and Technology of the Arts, 1:1. http://dx.doi.org/10.7559/citarj.v1i1.12

Krčadinac, U., Pasquier, P., Jovanovic, J., & Devedzic, V. (2013). Synesketch: An open source library for sentence-based emotion recognition. Affective Computing, IEEE Transactions on, 4(3).

Manovich, L., & Kratky, A. (2002). Soft cinema. ZKM Center for Art and Media, Karlsruhe.

Maxwell, J., Eigenfeldt, A., Pasquier, P., & Gonzalez, N. (2012). MusiCOG: A Cognitive Architecture for Music Learning and Generation. Proceedings of the 9th Sound and Music Computing Conference, Copenhagen.

Montfort, N. (2009). Curveship: an interactive fiction system for interactive narrating. Proceedings of the Workshop on Computational Approaches to Linguistic Creativity. Association for Computational Linguistics. http://dx.doi.org/10.3115/1642011.1642019

Montfort, N., y Pérez, R. P., Harrell, D. F., & Campana, A. (2013). Slant: A Blackboard System to Generate Plot, Figuration, and Narrative Discourse Aspects of Stories. Proceedings of the Fourth International Conference on Computational Creativity, Sydney.

Murch, W. (2007). Dense clarity–clear density. Ps1. org (Museum of Modern Art) 10.

Olofsson, F. (2013). low life. Musical Metacreation Weekend, Sydney (Old School Building, University of Sydney). Presented in association with ISEA 2013, curated by Oliver Bown. Sydney, Australia, June 15-16 2013.

Pachet, F. (2003). The continuator: Musical interaction with style. Journal of New Music Research 32:3. http://dx.doi.org/10.1076/jnmr.32.3.333.16861

Papadopoulos, G., & Wiggins, G. (1999). AI methods for algorithmic composition: A survey, a critical view and future prospects. AISB Symposium on Musical Creativity, Edinburgh.

Perez y Perez, R., Ortiz, O., Luna, W. A., Negrete, S., Pe-aloza, E., Castellanos, V., & Ávila, R. (2011). A System for Evaluating Novelty in Computer Generated Narratives. Proceedings of the Second International Conference on Computational Creativity, México City.

Philipsz, S. (2010). Lowlands. Tate, Britain (2010 Turner Prize winner). Curated by Helen Little and Katharine Stout, London.

Randel, D. (2003). The Harvard Dictionary of Music. Belknap Press.

Roma G., Herrera, P., & Serra, X. (2009). Freesound Radio: Supporting Music Creation by Exploration of a Sound Database. Computational Creativity Support Workshop CHI09, Boston.

Romero, J., & Machado, P. eds. (2008). The Art of Artificial Evolution: a Handbook on Evolutionary Art and Music. Springer. http://dx.doi.org/10.1007/978-3-540-72877-1

Rowe, R. (1992). Interactive Music Systems: Machine Listening and Composing. MIT press.

Seaman, B. (2002). Recombinant Poetics: Emergent Explorations of Digital Video in Virtual Space. New Screen Media: Cinema/art/narrative, London: BFI Pub.

Thorogood, M. (2007). Chatter and Listening: A Wireless Multi-node Sound Sculpture. Proceedings of the Australasian Computer Music Conference, Canberra.

Thorogood, M., & Pasquier, P. (2013). Computationally Generated Soundscapes with Audio Metaphor. Proceedings of the Fourth International Conference on Computational Creativity, Sydney.

Thorogood, M., & Pasquier, P. Impress: (2013) A machine learning approach to soundscape affect classification. Proceedings of the 13th International Conference on New Interfaces for Musical Expression, Daejeon.

Truax, B. (1998). Models and Strategies for Acoustic Design, Acoustic Ecology Conference.

Truax, B. (2008). Soundscape composition as global music: electroacoustic music as soundscape. Organised Sound 13:2. http://dx.doi.org/10.1017/S1355771808000149

Tzanetakis, G., & Cook, P. (2000). Marsyas: A framework for audio analysis. Organised sound 4:3. http://dx.doi.org/10.1017/S1355771800003071

Wardrip-Fruin, N., & Montfort, N. eds. (2003). The New Media Reader. The MIT Press.

Wardrip-Fruin, N. (2005). Playable media and textual instruments. Dichtung Digital 1.

Whalley, I. (2009). Software Agents in Music and Sound Art Research/Creative Work: Current State and a Possible Direction. Organised Sound 14:2. http://dx.doi.org/10.1017/S1355771809000260

Waschka, R. (2007). Composing with Genetic Algorithms: GenDash. Evolutionary Computer Music. Springer. http://dx.doi.org/10.1007/978-1-84628-600-1_6

Whitelaw, M. (2004). Metacreation. Art and Artificial Life. Cambridge: MIT Press.

Wilson, S., Cottle, D. & Collins, N. (2011). The SuperCollider Book. MIT Press.

Wooldridge, M. (2009). An Introduction to Multiagent Systems. Chichester: Wiley.

Wright, M., Freed, A. (1997). Open Sound Control: A New Protocol for Communicating with Sound Synthesizers. International Computer Music Conference. Thessaloniki, Greece.