The international consortium has set the goal to develop an intelligent, active model of auditory perception and experience in the context of listening and viewing.
The goal of the Two!Ears project is to improve computer models of human hearing, and thereby advance our understanding of human auditory information processing. The project tackles this problem with a new approach: the human listener is seen as a multimodal agent who develops a world model through interactive listening and viewing.
Up to now, computer models of human hearing have tended to focus on the evaluation of the signals received at the two ears – in other words, they are signal-driven. In the new Two!Ears approach, the prediction of human understanding and action will be improved by including hypothesis-driven processing. In such a scheme, world knowledge steers and improves the signal-driven evaluation. The system will be enabled to receive auditory events and integrate them with visual and proprioceptive information, for instance indications about the head orientation or the listener’s position in a room. It will thus attempt to describe an acoustic scene in the same way as a human listener does, in terms of primary perceptual constructs such as loudness, timbre and spatial extent. Additionally, the Two!Ears system will determine the meaning of the scene; for example, whether the perceived sounds come from a known or unknown speaker. The Two!Ears system will be realized on a robotic system which is capable of actively exploring its physical environment to orient itself and move around.
The system is based on an open architecture which allows easy modification and extension. This is considered a crucial approach to enable widespread use within the scientific community of the auditory models and cognitive functions developed within the project. Two!Ears will have considerable impact on future developments in information and communication technologies where understanding and responding to sound is relevant. In addition, research in adjacent areas such as biology, medicine, perception and cognitive psychology will benefit from the outcomes of the project. As an example, hearing aid algorithms for listening in situations with many competing speakers might benefit from the insights of the project.
|The Two!Ears project has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration over a period of three years (2013-16). The international consortium comprised, next to the DTU Hearing Systems, eight research institutes from the EU and the US.
Read more here: