Digital Creativity Centre

Universidade Catolica Portuguesa - School of Arts

Porto, Portugal
3 – 28 July 2017


How to Catch A Werewolf, Exploring Multi-Party Game-Situated Human-Robot Interaction

Lead-Organizers: Catharine Oertel, KTH (PI), Samuel Mascarenhas, INESC-ID, Zofia Malisz, KTH, José Lopes, KTH, Joakim Gustafson, KTH

In this project we will focus on the implementation of the roles of the “villager” and the “werewolves” using the IrisTK dialogue framework and the robot head Furhat. To be more precise, the aim of this project is to use multi-modal cues in order to inform the theory of mind model to drive the robot’s decision making process. Theory of mind is a concept that is related to empathy and it refers to the cognitive ability of modeling and understanding that others have different beliefs and intentions than our own. In lay terms, it can be described as “to put oneself into another’s shoes” and it is a crucial skill to properly play a deception game like “Werewolf”.

Participators: Yara Rizk, Cengiz Acarturk, Sena Büşra Yengeç, Mattias Bystedt, Eran Raveh

Full Project Description



KING’S SPEECH Foreign language: pronounce with style!

Principal investigators: Georgios Athanasopoulos*, Céline Lucas* and Benoit Macq* Candidates: Guillaume Gustin, Alessandro Cierro and Robin Guerit. * ICTEAM-ELEN - Université Catholique de Louvain, Belgium

The principal investigators are developing the GRAAL1 project which is concerned with developing a set of tools to facilitate self-training on foreign language pronunciation, with the first target being learning French. The goal of KING’S SPEECH is to develop new interaction modalities and evaluate them in combination with existing functionality aiming to better personalize GRAAL to the taste and specificities of each learner. This personalization will rely on a machine learning approach and an experimental set-up to be developed during eNTERFACE’17. The eNTERFACE’17 developments could be based on a karaoke scenario where the song is replaced by some authentic sentences (extracts of news, films, publicities, etc.). Applications like SingStar (Sony) or JustSing (Ubisoft) could also serve as a source of inspiration, e.g., using a smartphone as a microphone while interacting with avatars.

Participators: __

Full Project Description



The RAPID-MIX API: a toolkit for fostering innovation in the creative industries with Multimodal, Interactive and eXpressive (MIX) technology

Principal Investigators Francisco Bernardo, Michael Zbyszynski, Rebecca Fiebrink, Mick Grierson (EAVI – Embodied AudioVisual Interaction group, Goldsmiths University of London, Computing), Team Candidates Sebastian Mealla , Panos Papiotis (MTG/UPF – Music Technology Group, Universitat Pompeu Fabra), Carles Julia, Frederic Bevilacqua , Joseph Larralde (IRCAM – Institut de Recherche et Coordination Acoustique/Musique)

Members of the RAPID-MIX project are building a toolkit that includes a software API for interactive machine learning (IML),digital signal processing (DSP), sensor hardware, and cloud-based repositories for storing and visualizing audio, visual, and multimodal data. This API provides a comprehensive set of software components for rapid prototyping and integration of new sensor technologies into products, prototypes and performances.

We aim to investigate how developers employ and appropriate this toolkit so we can improve it based on their feedback. We intend to kickstart the online community around this toolkit with eNTERFACE participants as power users and core members, and to integrate their projects as demonstrators for the toolkit. Participants will explore and use the RAPID-MIX toolkit for their creative projects and learn workflows for using embodied interaction with sensors

Participators: José Raimundo, Helena Frijns, Mathias Nordvall, Jorge Nuno Coutinho

Full Project Description



End-to-End Listening Agent for Audio-Visual Emotional and Naturalistic Interactions

Principal Investigators: Kevin El Haddad (TCTS Lab - numediart institute - University of Mons, Belgium), Yelin Kim (Inspire Lab - University at Albany, State University of New York, USA), Hüseyin Çakmak (TCTS Lab - numediart institute - University of Mons, Belgium)
Team Members: Payton Lin (Research Center for Information Technology Innovation, Academia Sinica, Taiwan), Jaebok Kim (Human Media Interaction group, University of Twente, Enschede, Netherlands), Minha Lee (Human-Technology Interaction group - Technical University of Eindhoven, Eindhoven, Netherlands), and Yong Zhao (6VUB-NPU Joint AVSP Research Lab - Vrije Universiteit Brussel, Belgium & Northwestern Polytechnical University, China)

In this project, we aim at building a listening agent that would react with a naturalistic and human-like behavior and using nonverbal expressions to a user. The agent’s behavior will be modeled by and built on three main components: recognizing and synthesizing emotional and nonverbal expressions, and predicting the next expression to synthesize based on the currently recognized expressions. Its behavior will be rendered on a previously developed avatar which will also be improved during this workshop. At the end we should obtain functioning and efficient modules which ideally should work in real-time.

Participators: Louise Heron, Roberto Pulisci, Khalil El Harake, Nadine Hajj

Full Project Description



Cloud-based Toolbox for Computer Vision

Principal investigator: Dr. Sidi Ahmed MAHMOUDI from the Faculty of Engineering at the University of Mons. Belgium. Candidates: Dr. Fabian LECRON, PhD, Faculty of Engineering at the University of Mons. Belgium, Mohammed Amin BELARBI, PhD Student, Faculty of Exact sciences and Mathematics, University of Mostaganem, Algeria, Mohammed EL ADOUI, PhD Student, Faculty of Engineering, University of Mons, Belgium, Abdelhamid DERRAR, Student in Master University of Lyon, France, Pr. Mohammed BENJELLOUN, PhD, Faculty of Engineering, University of Mons, Belgium, Pr. Said MAHMOUDI, PhD, Faculty of Engineering, University of Mons, Belgium.

Nowadays, images and videos have been present everywhere, they can come directly from camera, mobile devices or from other peoples that share their images and videos. The latter are used to present and illustrate different objects in a large number of situations (public areas, airports, hospitals, football games, etc.). This makes from image and video processing algorithms a very important tool used for various domains related to computer vision such as video surveillance, human behavior understanding, medical imaging and database (images and videos) indexation methods. The goal of this project is develop an extension of our cloud platform (MOVACP) developed in the previous edition of eNTERFACE’16 workshop. The latter integrated several image and video processing applications. The users of this platform can use these methods without having to download, install and configure the corresponding software. Each user can select the required application, load its data and retrieve results, with an environment similar to desktop. Within eNTERFACE’17 workshop, we would like to improve and develop four main tools for our platform: 1. Integration of the major image and video processing algorithms that could be used by guests to perform their own applications. 2. Integration of machine learning methods (used for images and videos indexation) that exploit the uploaded data of users (is they accept of course) in order to improve the results precision. 3. Fast treatment of data acquired from IOT systems. 4. Development of an online 3D viewer that could be used for the visualization of 3D reconstructed medical images. 4. Fast treatment of data acquired from distant IoT systems.

Keywords cloud computing, image and video processing, video surveillance, medical imaging.

Participators: Lydia Aliane, lamia Bougara, Mohamed Amine Larhmam, Amine Lazouni

Full Project Description



Across the virtual bridge

Project Coordinators: Thierry RAVET (software design, motion signal processing, machine learning), Fabien GRISARD (software design, human-computer interface), Ambroise MOREAU (computer vision, software design), Pierre-Henri DE DEKEN (software design, game engine) - Numediart Institute, University of Mons, Belgium.
Team proposed: Mickael TITS (Motion Signal Processing, Machine learning), François ROCCA (computer vision, behaviour analysis), Radhwan BEN MADHKOUR (computer vision, software design), - Numediart Institute, University of Mons, Belgium, François ZAJEGA (3D design, Game engine) - ARTS2, Mons, Belgium.

The goal of the project is to explore different ways of creating interactions between people evolving in the real world (local players) and people evolving in a virtual representation of the same world (remote players). This latter one will be explored thanks to a virtual reality headset while local players will be geo-located through an app on a mobile device. Actions executed by remote players will be perceived by local players in the form of a sound or visual content and actions performed by local players will impact the virtual world as well. Local players and remote players will be able to exchange information with each other.
Keywords: Virtual world, mixed reality, computer-mediated communication .

Participators: __

Full Project Description



ePHoRt project: A telerehabilitation system for reeducation after hip replacement surgery

Principal investigators: Yves Rybarczyk (Nova University of Lisbon, Portugal), Arián Aladro (Universidad de las Américas, Ecuador), Mario Gonzalez (Health and Sport Science from University of Zaragoza - Spain), Santiago Villarreal (Universidad de las Américas - Quito, Ecuador), Jan Kleine Detersa (University of Twente in Human Media Interaction)

This project aims to develop a web-based system for the remote monitoring of rehabilitation exercises in patients after hip replacement surgery. The tool intends to facilitate and enhance the motor recovery, due to the fact that the patients will be able to perform the therapeutic movements at home and at any time. As in any case of rehabilitation program, the time required to recover is significantly diminished when the individual has the opportunity to practice the exercises regularly and frequently. However, the condition of such patients prohibits transportations to and from medical centres and many of them cannot afford a private physiotherapist. Thus, low-cost technologies will be used to develop the platform, with the aim to democratize its access. For instance, the motion capture system will be based on the Kinect camera that provides a good compromise between accuracy and price. The project will be divided into four main stages. First, the architecture of the web-based system will be designed. Three different user interfaces will be necessary: (i) one to record quantitative and qualitative data from the patient, (ii) another for the therapist consulting the patient’s performance and adapting the exercises accordingly, and (iii) for the physician having a medical supervision of the recovery process. Second, it will be essential to develop a module that performs an automatic assessment and validation of the rehabilitation activities, in order to provide a real-time feedback to the patient regarding the correctness of the executed movements. Third, we also intend to make use of a serious game and affective computing approaches, with the intention of motivating the user to perform the exercises for a sustainable period of time. Finally, an ergonomic study will be carried out, in order to evaluate the usability of the system.

Participators: Clément COINTE, Joao Rosas, Tiago Gonçalves

Full Project Description



Big Brother can you find, classify, detect and track us ?

Principal investigators: Marc Décombas, Jean Benoit Delbrouck (TCTS Lab - University of Mons, Belgium)

In this project, we will build a system that can detect, recognize objects or humans and describe them as much as possible on video. Objects may be moving as well as the people coming in and out of the visual field of the camera(s). Our project will be split into three main tasks :

  • detection and tracking
  • people re-identification
  • image/video captioning

The system should work in real time and should be able to detect people and follow them, re-identify them when they come back in the field and give a textual description of what each people is doing.

Participators: Pierre Marighetto, Pierrick Pamart, Max Cohen, Sohaib Laraba, Alessa Bandrabur, Amine Bellahsen, Hmayag Partamian

Full Project Description



Networked Creative Coding Environments

Principal investigators: Andrew Blanton, Digital Media Art at San Jose State University

As a part of ongoing research Andrew Blanton will present a workshop using Amazon Web Servers for the creation of networked art. The workshop will demonstrate sending data from Max/MSP to a Unix based Amazon Web Server and receiving data into a p5.js via websockets. The workshop will explore the critical discourse surrounding data as a borderless medium and the ideas and potentials of using a medium that can have global reach .

Participators: Luis Arandas, Sanjay Das

Full Project Description



Study of the reality level of VR simulations

Principal investigators: Andre Perrotta, UCP/CITAR
Team proposed:Pedro Pestana (UCP/CITAR), Jorge Cardoso (UCOIMBRA/CITAR), Celio Jonas Monteiro PhD student - UCP/CITAR, Armando Ramos PhD student - UCP/CITAR, Ricardo Ferreira - PhD student - UCP/CITAR, Bernardo Liborio Msc student - UCP/CITAR

We propose to develop a VR simulation based on 360o video, spatialized audio and force feedback using fans and motors, of near collision experiences of large vehicles on a frst person perspective, to be experienced by users wearing head-mounted stereoscopic VR gear in a MOCAP (motion capture) enabled environment that enables a one-to-one relationship between real and virtual worlds.


Full Project Description