Virtual worlds play an increasingly important role in our lives as places where you meet and make friends. Moreover, they influence the way we live, learn, communicate, heal, and entertain. When designed and applied appropriately they will have a strong positive influence our wellbeing. Wellbeing, or vitality, has at least three components: the physical body, the perception in the mind and the emotional state of the person. Current virtual worlds, however, exhibit poor affect and, therefore, do not offer a rich, emotional experience. This aims to enrich the determination of wellbeing to enable an increase the degree of wellbeing by changing a virtual world including the emotional perspective.
The ICT challenges of this project are:
- create a simple model to combine information of different observables to measure wellbeing, including the emotional state of a person. It is unclear how to combine qualitative and quantitative data provided by the intelligent sensors of the work packages in order to change the (virtual) world;
- create virtual worlds. At this moment little is known about the elements of a virtual world that contributes most to a specific form of behavior. The ICT challenge is to determine the kind of information that supports the targeted change of behavior most: which physical stimuli, which mind directions or emotional interventions, have the intended effect?
- introduce emotional aspects in virtual worlds. Most games are limited to large body movements, leaving the small and complicated face movement aside;
- most games are limited to large body movements, leaving the small and complicated face movement aside. The ICT challenge is to understand what small expressions in a (avatars) face will be associated with the right emotion.
Biggest results so far
Whiskers let your beloved ones feel your mood (video at the end of this page)
Although the Digital Age has made verbal communication over long distances extremely easy, non-verbal communication (like feelings as expressed by body language) has received much less attention. We develop new ways of non-verbal communication over long distances. Our demo consist of two ‘whiskers’, placed separate from each other. The demo looks like giant whiskers for humans that you can put in your home and that are connected via the internet to a whisker in the home of a beloved one. By touching and moving a whisker in your place, the whisker in the home of your friend or relative starts to move in the same way. This is an innovative way to communicate your feelings in a subtle way without being physically together. More.
ICT science question: what is a suitable medium for non-verbal communication using the internet? What is technically possible? What and how do people want to communicate non-verbally over long distances? Some existing products and concepts explore this idea, but generally use different modalities for the input and the output signal. For example, an input signal is translated into a blinking light as an output signal. Instead, in our prototype the input signal (giving a swing to one of the whiskers) and the output signal (the other whisker is swinging) are the same. This improves the feeling of being together.
Involved COMMIT/partners: Waag Society, HvA and University of Twente.
Learning with your body as interface
Learning in the 21st century puts high value on cooperation, personal involvement and creativity on the one hand, and digital skills on the other hand. In most of these digital skills the human body hardly plays a role, apart from typing on a keyboard. However, it is well known that the body does play an important role in learning. The mind uses and grows out of such bodily capacities.
Especially for the use at schools, we have developed three balance objects (Rolling Stairs, Cylinder and Seesaw) that help teenagers to learn with their body as an interface. Each object has sensors and is connected to an audio box.Teenagers can stand on an object and try to reach a balance. Depending on their movements, they hear different musical sound as a feedback. The sounds motivates them to change their posture.The objects evoke body awareness and emphasize the natural functionality of our bodies, like standing up, climbing, stretching and balancing. All objects aim to be silenced by the user, assilence is taken as the ideal state-of-mind to focus on the embodied self. More.
ICT science question: how to design interactive devices in which the body itself becomes the interface? What do we learn about the borders of the body while it interacts with these sensory devices? How easily does the body consider these objects as part of itself? Comparing our prototype objects with other devices, using the body as interface like Wii Fit Plus, the feedback loop that we use is unique. The participant is not only focused outwards, towards the screen, but the objects constantly lead the focus back to the sensation of the moving body itself.
Involved COMMIT/partners: Waag Society, Kennisnet and Marloeke van der Vlugt
Playing with emotions in the Moodroom
We have developed a Moodroom in which youngsters can express their emotions by moving their bodies. The Moodroom is an interactive installation that records bodily movements with a Kinect-sensor and translates them into colours, patterns, vibrations and sounds. The visual content is projected on three screens in front of the user. In the Moodroom youngsters can express and share their emotions without having to talk about them. The Moodroom is based on the principles of embodied cognition. Embodied cognition is an upcoming subject in both social psychology and neuropsychology that recognizes the mutual influence between our body, senses and cognitive functions. One aspect of this is communicating about emotions through visual cues, like colors and patterns. The goal of the Moodroom is to create a new way in which youngsters can get to know themselves and each other in an embodied learning environment. More.
ICT science question: how can sensory experiences be translated and used in a playful way in an interactive environment? First, we focus on abstract visual representations to arrive at a new ‘language’ of movements in which participants can express and communicate their emotions better than in words. We have the ambition to add other sensory experiences like sound and music.
Involved COMMIT/partners: Waag Society, Utrecht University and Bascule
Personalized interactive wall for elderly with dementia (video at the end of this page)
People suffering from dementia often feel confused and depressed. Some of them also display wandering behaviour. We build an interactive wall for people suffering from dementia. The wall uses computer vision to recognize the person in front of the wall and to recognize his or her behaviour and emotion. Based on the detected behaviour the wall then gives a personalized experience using video and music that the elderly like. Family members can upload content. The interaction with the wall may diminish the behavioural problems of dementia such as agitation, aggression, fear, depression and apathy. The wall also gives those who display wandering behaviour a virtual place to go to. More.
ICT science question: how to automatically recognize people, their behaviour and emotions? How to select appropriate audiovisual content automatically under demanding circumstances?
Involved COMMIT/partners: Utrecht University, HvA, UvA and CGI.
Send your loved one a gentle caress via a tactile sleeve (video at the end of this page)
We do not only use our sense of touch to explore the world around us, but also in social interaction. Social touch can help to reduce stress, communicate feelings and emotions, and is vitally important for the bonding between people. Unfortunately, our everyday digital communication via smart phones and computers completely lacks the sense of touch.
To overcome this disadvantage, we have developed a wearable, smart-textile sleeve, called TaSST (Tactile Sleeve for Social Touch). You can wear TaSST on your forearm and communicate through touch at a distance. The sleeve can register a gentle caress or a strong squeeze, and reproduce these touches as a vibrotactile pattern on a smart sleeve worn by somebody else. Our tactile sleeve helps to supports physical closeness between people separated by distance. More.
ICT science question: how can we use wearable haptic technology to communicate the important sense of touch?
Involved COMMIT/partners: HvA, University of Twente, Bartimeus and Elitac
An exercise game to reduce the risk of falling (video at the end of this page)
In many patients with neurological or orthopaedic disorders, but also in many healthy elderly, the ability to suddenly change the walking pattern is restricted. These people have a higher risk of falling, for example when trying to step over an obstacle. To reduce the falling risk, we have developed a fun and motivating exercise game. During an assessment on a walking belt we measure the adaptivity of a patient’s walking style. After defining a comfortable walking speed, we present a visual feedback of a target step length. The subject then needs to respond by taking smaller and larger steps respectively. The belt speed adjusts automatically to ensure a constant step frequency. We stimulate the patient to change his step frequency by controlled belt speed changes in combination with target step-lengths. Our assessment measures how well the patient performs adjustments in step lengths and step frequencies. The better the adjustment, the lower the risk of falling. More.
ICT science question: how can we improve the adaptability to patterns in personal data?
Involved COMMIT/partners: Motek Medical, CGI, HvA, University Utrecht, UvA, Waag Society and Digifit.
Virtual learning helps people with language problems in societal participation
We have designed a virtual learning environment that supports low-literates and immigrants with language problems to participate in the Dutch society. In this virtual environment, users can select prototypical situations in which they need information and communication skills in both formal and informal social settings. We have developed four scenarios: online banking, grocery shopping, visiting a service desk and chatting at a bus stop. The virtual environment uses virtual humanlike characters that graphically appear on a screen. The characters engage in a conversation, show gestures and have facial expressions like real humans. They either conduct conversations with the user or support him in doing participation exercises. More.
ICT science question: what is the best way to design an effective virtual learning environment? In particular, the learning environment should be able to deal with large differences in the input. The virtual learning environment should also adapt the offered content and the difficulty level over time.
Involved COMMIT/partners: TNO, TUDelft and Cinop.
Involving the end-user in the design process
Good product design contributes to good business. Unfortunately, many people have difficulties with using everyday technological products. To solve this problem, the end-user should be more involved in the design process. We have developed the Include Toolbox to support small and medium ICT-enterprises in applying ‘inclusive design methods’ in their product development. By involving the end-user in the design process, inclusive design tries to include as many users as possible. The user of the Include Toolbox first chooses a method tailored to the specific design phase and envisioned user group. Next, the user receives step-by-step support from preparation and execution of the method to analysis of the results. Several toolbox methods have already been applied, which resulted in two apps. The first app supports children around the neighbourhood, for example, by keeping them safe and helping them make new friends. The second app supports elderly to organize adhoc help when needed. More.
ICT science question: how can we bridge the gap between theory and practice of inclusive design? The tool needs to be sufficiently detailed to address all end-user groups in a meaningful way. At the same time the tool needs to be comprehensible and accessible for both experienced and inexperienced users.
Involved COMMIT/partners: TNO, Eagle Science
Socially adaptive apps (video at the end of this page)
Although social applications such as Twitter, Foursquare, Life360 and Facebook help to connect people, they can conflict with important personal values like privacy, independence and freedom. For example, when parents use GPS-tracking to know where their children are 24/7, this may be beneficial for family security, but it negatively impacts children’s privacy and independence. We have developed two mobile apps for increasing social participation of people in the neighbourhood while at the same time supporting users’ privacy, independence and freedom. The first app is designed for children between nine and twelve years old, to support them in their everyday activities (going to school, making new friends, etc.). The second app is designed for elderly people and their social network to make them feel safer and less dependent on others. More.
ICT science question: can the use of norms in social applications promote user values like independence better than the basic settings of existing applications like Facebook? How can social applications be designed to make them more usable to the special needs of vulnerable people such as children and elderly?
Involved COMMIT/partners: Sense Labs, BSS, TNO, and TUDelft.