You are here

Detection and interpretation of user behaviour

SenseI (Sensor based Engagement for Improved Health)


WP1 is concerned with the low-level detection of user’s physical, mental and social behaviours from low-level sensing data and the identification of higher-level physical, mental/emotional, and social states of the user, like fitness, objective and subjective fatigue, arousal, stress and social inclinations. WP1 consists of three main subprojects:

Assessment of Physical Aspects of Exercise

Algorithms for monitoring the ambulatory activity of the user based (mainly) on smartphone data will be developed for assessing both exercise type (categorization) and physical load, as well as for assessing the quality of specific exercise (notably walking and running).

Assessment of Mental/Emotional/Social Aspects of Exercise

Mental, emotional and social user states will be assessed via the smartphone through explicit verbal prompts from the user about how he or she feels (VU-FHMS), and by extracting the implicit information contained in the voice (UT). Social states and inclinations will be gleaned from (accelerometer and voice) data obtained from other users with whom the user interacts or doesn’t interact (HvA-DCMI). Models for mental, emotional and social state assessment will be developed and implemented on the smartphone for further testing and training. VU-FHMS, UT, and HvA-DCMI will collaborate extensively in integrating data from explicit and implicit verbal responses as well as accelerometer data in order to obtain a comprehensive assessment of the user’s mental and emotional state.

Low level sensor implementation

The aim of the present project is to create the processing environment for assessing the physical, mental and social state of the user. Initially, we will use the existing Sense platform on the phone to access local sensor data. This platform will be extended with additional sensor implementations for sport applications. This includes extra audio sensing, support for external sensors such as heart rate monitors and additional sensing modes for the motion sensors.

WP Leader: