As computers have become more powerful and ubiquitous, the essential nature of the relationship between humans and computers has evolved. Pervasive computing and network intensive technologies are enabling the development of a human-centric, symbiotic human-computer Partnership. The resulting emergent e-Society paradigm encompasses a variety of domains such as intelligent housing, semantic web, e-health, and e-commerce.
Modern network-intensive technologies provide the sine qua non key solutions for the efficient implementation of a large variety of human-centered ubiquitous computer applications in the emergent e-Society such as intelligent networked spaces for living (smart–houses), working, halth and elderly care, shopping, and entertainment. In order to augment the functionality and user-satisfaction of these network-intensive e-Society applications, we have to develop more accessible and unobtrusive, user-oriented and user friendly, human-computer interaction modalities allowing humans to seamlessly access information anytime and anywhere and to interact naturally and intuitively with these new intelligent spaces
One of the hottest areas of interest in intelligent buildings is the digital signage. This area has a strong industrial and commercial appeal. The dynamic digital signage industry is a fast growing industry with investments coming from important companies such as IBM (IBM/SCALA partnership), 3M Digital Signage, and Cisco (through the acquisition of Trivella Inc.) Using plasma and LCD screens that are usually centrally controlled and have changing contents, digital signage systems are used for advertisements, live television, news, way finding
and other property related information. While digital signage is, in many ways, a mature industry, interactive digital signage is still in its infancy.
The objective of this project is to develop new computer vision and voice recognition techniques that allow multimodal interactive capabilities to be added to networked digital signage systems through gestural (body posture and hand gesture) and voice human-computer interfaces.
The application potential spans a variety of network-intensive building environments such as intelligent buildings, art galleries and museums, movie complexes, libraries, shopping centers, smart health and elderly care facilities, and smart homes. The proposed solution is innovative both in its content and context. Multimodal, hand gestures and voice commands, human-computer communication has not been used before, to our knowledge, in designing intelligent networked building environments. The interest of our industrial and institutional users (see letters) testifies to the recognition of the proposed innovation and great practical application potential.
Hand gestures represent a very expressive and a powerful non-verbal context-dependent human communication modality [. The human hand is a complex, articulated object consisting of many connected parts and joints. With the latest advances in the fields of computer vision, image processing and pattern recognition, real-time vision-based hand gesture classification is becoming more and more feasible. We will study vision-based hand tracking and gesture classification, focusing on tracking the bare hand and recognizing hand gestures without the help of any markers and gloves. Our research will address both issues of hand gesture recognition: hand posture and hand gestures. Real-time signal processing algorithms will be designed for the identification and evaluation of environmental and human behaviour multimodal parameters, such as human body postures, human voice and background sound that provide the contextual information for the detection of human users to interact with the digital signage system.
The specific objectives of the proposed research program are: