User-Centred Design Actions for Lightweight Evaluation of an Interactive Machine Learning Toolkit

Francisco Bernardo, Mick Grierson, Rebecca Fiebrink

 
   

Abstract


Machine learning offers great potential to developers and end users in the creative industries. For example, it can support new sensor-based interactions, procedural content generation and end-user product customisation. However, designing machine learning toolkits for adoption by creative developers is still a nascent effort. This work focuses on the application of user-centred design with creative end-user developers for informing the design of an interactive machine learning toolkit. We introduce a framework for user-centred design actions that we developed within the context of an European Union innovation project, RAPID-MIX. We illustrate the application of the framework with two actions for lightweight formative evaluation of our toolkit—the JUCE Machine Learning Hackathon and the RAPID-MIX API workshop at eNTERFACE’17. We describe how we used these actions to uncover conceptual and technical limitations. We also discuss how these actions provided us with a better understanding of users, helped us to refine the scope of the design space, and informed improvements to the toolkit. We conclude with a reflection about the knowledge we obtained from applying user-centred design to creative technology, in the context of an innovation project in the creative industries.  


Keywords


User-centred Design; Interactive Machine Learning; Application Programming Interfaces; Toolkits; Creative Technology;

Full Text:

PDF

References


Abras, C., Maloney-Krichmar, D., & Preece, J. (2004). User-Centered Design. In W. Bainbridge (Ed.), Encyclopedia of Human-Computer Interaction (Vol. 37, pp. 445–56). Thousand Oaks: Sage Publications. http://doi.org/10.1.1.94.381

Amershi, S., Chickering, M., Drucker, S. M., Lee, B., Simard, P., & Suh, J. (2015). ModelTracker: Redesigning Performance Analysis Tools for Machine Learning. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems - CHI '15, 337–346. http://doi.org/10.1145/2702123.2702509

https://doi.org/10.1145/2702123.2702509

Bernardo, F., Tanaka, A., Fiebrink, R., Parkinson, A., Meala, S., & Bevilacqua, F. (2015). D2.1 User-Centred Design Methodology. Retrieved from https://rapidmix.goldsmithsdigital.com/wp-content/uploads/2016/02/D2.1-User-Centred-Design-Methodology1.pdf

Bernardo, F., Zbyszyński, M., Fiebrink, R., & Grierson, M. (2017). Interactive Machine Learning for End-User Innovation. In Proceedings of the Association for Advancement of ArtificiaI Intelligence Symposium Series: Designing the User Experience of Machine Learning Systems (pp. 369–375).

Bevilacqua, F., Bernardo, F., Mealla, S., & Fiebrink, R. (2015). D2.2 Design Guidelines for Prototyping. Retrieved from http://rapidmix.goldsmithsdigital.com/wp-content/uploads/2016/02/D2.2DesignGuidelineforPrototyping.pdf

Blackwell, A. F. (2017). End-User Developers – What Are They Like? In F. Paternò & V. Wulf (Eds.), New Perspectives in End-User Development (pp. 121–135). Cham: Springer International Publishing. http://doi.org/10.1007/978-3-319-60291-2_6

https://doi.org/10.1007/978-3-319-60291-2_6

Caramiaux, B., Montecchio, N., Tanaka, A., & Bevilacqua, F. (2014). Adaptive Gesture Recognition with Variation Estimation for Interactive Systems. ACM Transactions on Interactive Intelligent Systems, 4(4), 1–34. http://doi.org/10.1145/2643204

https://doi.org/10.1145/2643204

Cherry, E., & Latulipe, C. (2014). Quantifying the Creativity Support of Digital Tools through the Creativity Support Index. ACM Transactions on Computer-Human Interaction, 21(4), 1–25. http://doi.org/10.1145/2617588

https://doi.org/10.1145/2617588

Chowning, J. M. (1973). The Synthesis of Complex Audio Spectra by Means of Frequency Modulation. Journal of the Audio Engineering Society, 21, 1–10. http://doi.org/10.1038/srep01061

https://doi.org/10.1038/srep01061

Clarke, S. (2011). How Usable Are Your APIs? In A. Oram & G. Wilson (Eds.), Making Software: What Really Works, and Why We Believe It (1 edition, pp. 545–565). O'Reilly Media.

Cooper, A., Reimann, R., & Cronin, D. (2007). About Face 3: The Essentials of Interaction Design. Indianapolis, Indiana: Wiley.

Edwards, W. K., Bellotti, V., Dey, A. K., & Newman, M. W. (2003). Stuck in the Middle : The Challenges of User-Centered Design and Evaluation for Infrastructure. Chi 2003, 297–304. http://doi.org/10.1145/642611.642664

https://doi.org/10.1145/642611.642664

Fails, J. A., & Olsen, D. R. (2003). Interactive Machine Learning. In Proceedings of the 8th international conference on Intelligent user interfaces IUI 03 (pp. 39–45). http://doi.org/10.1145/604045.604056

https://doi.org/10.1145/604045.604056

Fiebrink, R., Cook, P. R., & Trueman, D. (2011). Human Model Evaluation in Interactive Supervised Learning. In Proceedings of the 2011 annual conference on Human factors in computing systems - CHI '11 (p. 147). http://doi.org/10.1145/1978942.1978965

https://doi.org/10.1145/1978942.1978965

Françoise, J., Schnell, N., & Bevilacqua, F. (2013). A multimodal probabilistic model for gesture-based control of sound synthesis. Proceedings of the 21st ACM International Conference on Multimedia - MM '13, 705–708. http://doi.org/10.1145/2502081.2502184

https://doi.org/10.1145/2502081.2502184

Hall, M. A., Frank, E., Holmes, G., Pfahringer, B., Reutemann, P., & Witten, I. H. (2009). The WEKA Data Mining Software: An Update. SIGKDD Explorations, 11(1), 10–18. http://doi.org/10.1145/1656274.1656278

https://doi.org/10.1145/1656274.1656278

Hartmann, B., Abdulla, L., Mittal, M., & Klemmer, S. R. (2007). Authoring Sensor-based Interactions by Demonstration with Direct Manipulation and Pattern Recognition. In Proceedings of the SIGCHI conference on Human factors in computing systems - CHI '07 (p. 145). http://doi.org/10.1145/1240624.1240646

https://doi.org/10.1145/1240624.1240646

Holtzblatt, K., Wendell, J. B., & Wood, S. (2004). Rapid Contextual Design: A How-To Guide to Key Techniques for User-Centered Design. San Francisco, CA: Morgan Kaufmann Publishers Inc.

Kapoor, A., Lee, B., Tan, D., & Horvitz, E. (2010). Interactive Optimization for Steering Machine Classification. Proceedings of the 28th international conference on Human factors in computing systems - CHI '10. New York, New York, USA, NY, USA: ACM Press. http://doi.org/10.1145/1753326.1753529

https://doi.org/10.1145/1753326.1753529

Katan, S., Grierson, M., & Fiebrink, R. (2015). Using Interactive Machine Learning to Support Interface Development Through Workshops with Disabled People. In CHI '15: Extended Abstracts on Human Factors in Computing Systems 2015. http://doi.org/10.1145/2702123.2702474

https://doi.org/10.1145/2702123.2702474

Lieberman, H., Paternó, F., Klann, M., Paternò, F., & Wulf, V. (2006). End-user development: An emerging paradigm. End User Development, 9, 1–8. http://doi.org/10.1007/1-4020-5386-X

https://doi.org/10.1007/1-4020-5386-X

McPherson, A. (2017). Bela: An embedded platform for low-latency feedback control of sound. The Journal of the Acoustical Society of America. http://doi.org/10.1121/1.4987761

https://doi.org/10.1121/1.4987761

Monk, A. (2007). Lightweight Techniques to Encourage Innovative User Interface Design. In L. E. Wood (Ed.), User Interface Design: Bridging the Gap from User Requirements to Design. CRC Press LLC.

Myers, B. A., & Stylos, J. (2016). Improving API usability. Communications of the ACM, 59(6), 62–69. http://doi.org/10.1145/2896587

https://doi.org/10.1145/2896587

Nielsen, J. (1994). Usability Engineering. San Francisco, CA: Morgan Kaufmann Publishers Inc.

Norman, D. A. (2013). The Design of Everyday Things. MIT Press.

Norman, D., & Draper, S. W. (1986). User Centered System Design: New Perspectives on Human-Computer Interaction. Hillsdale, NJ, USA: L. Erlbaum Associates Inc.

Parkinson, A., Zbyszyński, M., & Bernardo, F. (2017). Demonstrating Interactive Machine Learning Tools for Rapid Prototyping of Gestural Instruments in the Browser. In Web Audio Conference 2017 (pp. 1–2). Retrieved from https://qmro.qmul.ac.uk/xmlui/handle/123456789/26150

Patel, K., Bancroft, N., Drucker, S. M., Fogarty, J., Ko, A. J., & Landay, J. (2010). Gestalt: Integrated Support for Implementation and Analysis in Machine Learning. In Proceedings of the 23nd annual ACM symposium on User interface software and technology (pp. 37–46). http://doi.org/10.1145/1866029.1866038

https://doi.org/10.1145/1866029.1866038

Ritter, F. E., Baxter, G. D., & Churchill, E. F. (2014). Foundations for Designing User-Centered Systems: What System Designers Need to Know about People. London: Springer London Heidelberg New York Dordrecht.

https://doi.org/10.1007/978-1-4471-5134-0

Steiglitz, K. (1996). A Digital Signal Processing Primer: With Applications to Digital Audio and Computer Music. Addison-Wesley.

von Hippel, E. (1986). Lead Users: An Important Source of Novel Product Concepts. Management Science, 32(7), 791–805. http://doi.org/10.1017/CBO9781107415324.004

https://doi.org/10.1017/CBO9781107415324.004

Zbyszyński, M., Grierson, M., & Yee-king, M. (2017). Rapid Prototyping of New Instruments with CodeCircle. In NIME 2017 Proceedings of the International Conference on New Interfaces for Musical Expression (pp. 227–230).

Zbyszyński, M., Grierson, M., Yee-king, M., & Fedden, L. (2017). Write once run anywhere revisited: machine learning and audio tools in the browser with C++ and emscripten. In Web Audio Conference 2017 (pp. 1–5). Centre for Digital Music, Queen Mary University of London.




DOI: http://dx.doi.org/10.7559/citarj.v10i2.509

Refbacks

  • There are currently no refbacks.




Journal of Science and Technology of the Arts
Revista de Ciência e Tecnologia das Artes
ISSN: 1646-9798
e-ISSN: 2183-0088
Portuguese Catholic University | Porto


Esta revista científica é financiada por Fundos Nacionais através da FCT – Fundação para a Ciência e a Tecnologia


 Governo da República Portuguesa