Automatic recognition of people and their activities has very important social implications, because it is related to the extremely sensitive topic of civil liberties. Society needs to address this issue of automatic recognition and find a balanced solution that is able to meet its various needs and concerns. In the post 9/11 period, population security and safety considerations have given rise to research needs for identification of threatening human activities and emotional behaviours.
Timely identification of human intent is one of the most challenging areas of “all-hazards” risk assessment in the protection of critical infrastructure, business continuity planning and community safety. The “all-hazards” approach is used extensively by the public and private sector, including Public Safety Canada (PS Canada – formerly PSEPC), Emergency Management Ontario (EMO), US Federal Emergency Management Agency (FEMA) and US Department of Homeland Security (DHS).
This five-year research project addresses fundamental issues involved in the prevention of human-made disasters, namely the variable context-dependent, real-time detection/identification of potential threatening behaviour of humans, acting individually or in crowded environments. In order to better respond to the needs of the user sector, the proposed research will be carried on in conjunction with the users.
The proposed MUSES-SECRET project aims at the development and commercialization of new multimodal (video and infrared, voice and sound, RFID and perimeter intrusion) intelligent sensor technologies for location and socio-cultural context-aware security risk assessment and decision support in human-crowd surveillance applications in environments such as school campuses, hospitals, shopping centers, subways or railway stations, airports, sports and artistic arenas etc. Multisensor data fusion techniques will be investigated for the dynamic integration of the multi-thread flow of information provided by the heterogeneous network of surveillance sensors into a coherent multimodal model of the monitored human crowd. Real-time image processing and computer-vision algorithms will be studied for the identification, tracking and recognition of gait and other relevant body-language patterns of the human agents who can be deemed of interest for security reasons. Real-time signal processing algorithms will be designed for the identification and evaluation of environmental and human behaviour multimodal parameters (such as human gait, gestures, facial emotions, human voice, background sound, ambient light, etc.) that provide the contextual information for the specific surveillance activity. A multidisciplinary, context-aware, situation-assessment system, including human behaviour, cognitive psychology, multicultural sociology, learning systems, artificial intelligence, distributed multimedia and software design elements, will be ultimately developed for the real-time evaluation of the activity and emotional behaviour of the human subjects identified as being potentially of security interest in the monitored dynamic environment. The resulting system should provide efficient multi granularity-level function-specific feedback for the human users who are the final assessors and decision makers in the specific security monitoring situation.