You are here

IUALL (Interaction for Universal Access)

LOITER: training social skills of police officers (video at the end of this page)


There is a growing need for all members of our society to be able to interact with computer-based information. Authorities, for instance, increasingly expect people to be able to solve their own problems and to take the initiative to ask for support via web applications instead of being offered support by social workers visiting the area, as once was the case. Some people are less well equipped for modern society than others.

The IUALL (I, You, All) project starts from the idea that anybody should be able to interact with computer systems. Universal Access calls for systems that are designed also for people with, for instance, low-literacy skills, small children, those living outside their native country, the elderly and even those with severe disabilities. The project approaches this design challenge in two ways. First by investigating technology with social intelligence and second by investigating and producing tools for IT designers to help them design for all.

ICT with Social Intelligence.

Socially Intelligent technology can automatically sense and understand how people interact with each other and with technology. The project contributes to the field of social signal processing or human behavior understanding with algorithms that can detect social information automatically from multimodal cues taken from speech and video. The main domain in which this is investigated in this project is at the school playground. The project also contributes to the field of artificial intelligence, investigating and building spoken dialogue systems in the form of virtual humans that act as a coach or teacher to train social skills.

Using a human-centered design approach, a large part of the project investigates technology that can help people improve their participation in the society. Together with companies and various user groups in a local municipality, we develop and evaluate applications that increase collaboration and the feeling of safety in the community. This research combines Human-Computer Interaction research with the more technologically oriented strand of studies on socially intelligent technology. The various field studies provide insight for constructing a toolkit for inclusive design.

Biggest results so far

Monitoring group emotions

We have developed the crowd emotion monitor that evaluates the emotion of a group of people based on photos. It automatically analyzes the emotional expressions on people’s faces as they appear on images. The images can come from smartphone cameras or from cameras installed on fixed locations. Location tracking is done by using WiFi-sniffers to track the participants. Via a smartphone app the platform can share the resulted group emotion with stakeholders. Our first objective is to enhance people’s experiences during conferences. As a visitor of a conference the monitor can allow you to see the emotional states in various rooms. You can use this information to find out where the most interesting presentation is going on. More.

ICT science question: how can we reliable track the location of people’s smart phones in a crowd based on WiFi-sniffers? How can we automatically recognize emotional expressions on people’s faces from images that come from smartphones or fixed cameras? How should we store and compute the data so to monitor group emotions in real time? Our emotion monitor combines state-of-the-art sensing, event processing, reasoning and data communication technologies. 

Involved COMMIT/partners: Noldus, CWI, MondetDB, TNO, ESI, University of Twente, Vicar Vision.

Tikkertje 2.0 (video at the end of this page)

Interactive playgrounds are installations that combine the benefits of traditional playgrounds for children with advances in technology. In our demo we show the Interactive Tag Playground (ITP), a modern, interactive version of the century-old game called ‘Tagging’ or ‘Tikkertje’ in Dutch.With the Interactive Tag Playground we actively steer the interaction between players. To this end, players are being tracked using Kinects and additional body-worn sensors. In its most basic version we project differently colored circles around each player to indicate the role of tagger or runner. When these circles collide, a ‘tag’ is detected and the roles of the players switch. We also add novel interactive elements such as power-ups and bonuses. Apart from entertainment, the Interactive Tag Playground is also a tool to study how children interact with each other and with the environment. Our final aim is to automatically steer the interactions in such a way that all players remain engaged and physically active. More.

ICT science question: how can we best track the players and understand their interactive behavior? What kind of interactions are suitable at which moment during the play to maintain the engagement of players? How can players be motivated to adopt certain play styles such as cooperation or competition? 

Involved COMMIT/partners: KITT Engineering, Noldus and University Twente.

A serious game for training social skills of police officers (video at the end of this page)

We have developed a serious game called LOITER to train the social skills of police officers. Players of the game have to resolve a conflict with a group of loitering juveniles. Through playing this game, police trainees can improve their social awareness. Players interact with virtual juveniles in a 3Denvironment using a full-body immersive virtual reality system. The virtual juveniles use artificial intelligence to respond to the player according to theories from social psychology. Thus, the choices of the players in how to reason with the juveniles determine the outcome of the conflict. More.

ICT science question: one of our main challenges is to model human behaviour and let virtual characters respond believably to human users. To create such a model, we analyzed interactions between police officers and civilians. We also implemented an interactive story structure that lets people experience the influence of their own behaviour on others. This results in different scenarios with different story endings. Our approach is unique because of its combination of methodologies and the ability of the characters to explain the reasoning behind their actions.

Involved COMMIT/partners: Politie, T-XChange, University of Twente, Re-liOn.