The AVIR proposal originates from the demand for new solutions allowing common users to easily access, store and retrieve relevant audio-visual information from the vast amounts of resources at their disposal. The next generation of television systems will be connected to many sources of information and entertainment (TV and radio from air, cable or satellite, video and audio libraries, video tape/disk recorders, Internet). Literally hundreds of channels will soon be offered to the user, which could be disoriented by this overload of information.

Users will not pay for just more extra channels, but will appreciate if the content in the channels is easily accessible and, more importantly, can be easily selected according to the user's personal interest. This can only be achieved if the broadcaster delivers meta-data describing the actual content in sufficient detail enabling for automatic handling by agents residing on the end user's system. AVIR will investigate on novel procedures for automatic analysis and indexing of audio-visual information, specifically meant to support consumer services.

The objective of this project is to investigate and experiment end-to-end solutions for delivering new added value services on top of digital video broadcast services, which will enable a better exploitation of multimedia information resources by non-IT experts.

As a result the project will deliver a prototype service user platform and will demonstrate its feasibility on a broadcast delivery chain. It takes into account extraction of high quality meta-data and electronic delivery of meta-data associated to audio-visual content, including adaptation of consumer receivers and recorders towards a personalized multimedia repository.

Intelligent agents based on a user interest profile will help the user to browse and access most relevant programmes via an intelligent, personal electronic guide. A low cost, high capacity home storage device, will also be used to increment the capabilities of the consumer system. Thanks to the received descriptors, advanced retrieval features can be implemented on the stored assets and, in combination with the user’s profile, automatic recording feature is possible. A visual navigation system, a search engine and agents will help the user identify video material of interest on the home video-recorder, transforming it into a personal multimedia repository.

The project will be end-user driven, focusing on the user interaction aspects of the consumer system. However, support for such applications is needed throughout the whole service delivery chain. Descriptors will be generated by the content and service provider. AVIR will investigate on novel procedures for automatic analysis and indexing of audio-visual information, specifically meant to support consumer services.

The indexing process will generate a set of content descriptors (or meta-data), according to an open standard specification to guarantee full interoperability. We will therefore contribute to existing standardisation bodies to define new standards or improve existing ones (MPEG-7, DVB, SMPTE, W3C RDF etc.).

The meta-data will be streamed, multiplexed and distributed along with the associated content. The service consumer system will extract these descriptors from the channels that support the service according to the defined standard. Thus an 'intelligent program guide' will help the user in finding interesting audio-visual material, exploiting the descriptors to search through currently transmitted or stored programs.

The project consortium is made of companies and organisations bringing together knowledge and expertise in all fields required by the scope of the project and encompassing users, research institutes, service providers and manufacturers.

LEP will perform analysis of video with temporal and object segmentation.

UNIBS will perform research on cross-modal indexing, with joint analysis of audio and video information, to obtain high level semantic descriptors.

UPMC will focus on speech-to text conversion, meta-data structure and the query engine.

RAI will experiment the provision of the broadcast service with meta-data via a DVB satellite channel, crucial for the demonstration phase.

Tomorrow Internet AG / TV Spielfilm will experiment a service for electronic provision of descriptors about TV programmes for the electronic guide.

Tecmath will be responsible for the multimedia database issues for the retrieval system.

Philips Research (co-ordinator) will develop the consumer system, including the user interaction on the receiver and the storage system, based on latest technology for video tape and disk recorders.

BBC Archive (sponsor) will provide assets from its video repositories to test automatic indexing procedures for the generation of meta-data.