NEuroNet Roadmap
[Preliminary Draft]

First Report on the Use of Neural Networks and Other Computational Intelligence Techniques in Intelligent Multimedia Systems

                           S Kollias, F Piat and A Drossopoulos, ICCS-NTUA, 3rd June 1999




Being involved in the European TMR project Physta, whose goal is to develop an intelligent system to recogniseemotions based on the integration of facial and vocal expressions [4,5], we are particularly aware of the potential of Neural Networks (NNs) for intelligent Human-Computer Interaction (HCI). Making HCI more natural to humans and more intelligent is one of the greatest, fundamental challenges of multimedia applications. Humans naturally communicate through a variety of channels and intelligence is required in order to interpret humans’ messages, at both within- and between-channel levels of processing. The voice and the face are the two privileged channels of communication, and therefore applications to face and voice processing are emphasised in what follows. On another hand, the proliferation of data available to users is so important that information retrieval, data transmission and browsing efficiency are crucial issues of today’s computer use. These applications also represent a great potential for the use of NNs, as mentioned below.
 

Face processing:

Voice processing: Music processing: Multi-modal integration for HCI: Multimedia databases Browsing Human-Machine Interaction using Agents (dialog-based interactions)

References

[1] Special Issue on Artificial Neural Network Applications, Proceedings of the IEEE, vol. 84, no. 10, Oct. 1996.

[2] Special Issue Part One: Multimedia Signal Processing, Proceedings of the IEEE, vol. 86, no. 5, May 1998.

[3] Special Issue Part Two: Multimedia Signal Processing, Proceedings of the IEEE, vol. 86, no. 6, June 1998.

[4] R. Cowie, E. Douglas-Cowie, N. Tsapatsoulis, G. Votsis, S. Kollias, W. Fellenz and J. Taylor, “Emotion recognition in human-computer inter-action,” submitted for publication to the IEEE Signal Processing Magazine, January 1999.

[5] Development of Feature Representations from Emotionally coded Facial Signals and Speech, Report for TMR Physta project, Research contract FMRX – CT97 – 0098 (DG12-BDCN)

[6] S.Y. Kung and J.N. Hwang, “Neural Networks for Intelligent Multimedia Processing”, Special Issue Part Two: Multimedia Signal Processing, Proceedings of the IEEE, vol. 86, no. 6, June 1998.

[7] Modulate Project Report, Education Multimedia Project, 1999-2001.

Industrial Clubs in Greece