Presented at MPEG-7 Workshop, 16 February 1998, Brussels
(Previous to the 9th ACTS Concertation Meeting)
1. Introduction
The transition from analog to digital video is about to bring the expected convergence of television, computer and communication, with many possibilities for delivery of multimedia interactive services. However, current broadcast services only feature an EPG where interaction is limited to getting title of current and next program, if available. Metadata can be defined as control and descriptive elements associated to media content. In order to allow searches on programs, more structured metadata are needed in the broadcast streams. One of the goals of the STORit project is to provide low-cost solutions for enriching the service offered to consumers by the addition of metadata on the broadcast streams. This will allow more interactivity with the receiving system, without needing a return channel, and when a home storage device is available, more attractive features can be offered to consumers. Unfortunately there is no adequate standard to be used for metadata describing audio-visual content. MPEG7, formally called "Multimedia Content Description Interface", wishes to address this problem by providing a complete and yet flexible standard for the generation of metadata in all fields of multimedia content production and distribution.
2. Professional Applications
Nowadays television and film archives store a vast amount of multimedia material in several different formats. The metadata is manually produced and maintained by the archives departments for internal documentation and retrieval purposes and is normally stored with proprietary formats. A major requirement of the television production community is the ability to exchange programme materials across organisations, platforms and applications. The usage of metadata creates the opportunity to include content-based retrieval features into new and existing databases. There is therefore a huge potential interest in an international standard that could ensure interoperability between video archive operators and a wider diffusion of the data to the professional and the general public.
MPEG-7 started intending to make audiovisual material "as searchable as text is today". In this case search is assumed to exploit a multimedia database where content and descriptors are stored in digital form. The search task can also be executed remotely in a networked environment. On a larger scale this kind of services takes place according to a "pull" model, and the procedure often uses the Internet WWW paradigm. An AHG is currently defining the requirements for different applications.
The EBU and SMPTE, the two major international television standard bodies, have decided to join their efforts by forming a Task Force for Harmonised Standards for the exchange of television program material. One of the most important findings of the Task Force was that to easily handle various schemes and support television production, a new class of descriptive and supporting data connected to programs should be defined. Their task was not to define the standard for metadata itself, but basically to determine the user requirements, which MPEG7, after establishing a liaison with the Task Force, is likely to take into consideration in the context of professional video production applications.
DAVIC divides content metadata into two categories. Content management metadata is used in the service provider system for content transfer and control. Navigational metadata is used to enable usage of the content, as controlled by the client and server applications. Navigational metadata might also be distributed to the general public. For broadcast services this data could be encapsulated in the stream together with the associated content. Therefore the requirement is that this set of metadata must be easily stream-able, either in a separate stream or in multiplex along with the source material.
3. Consumer Applications
Surprisingly the last part of the distribution chain, concerned with the service provided to consumers, so far seems not to have drawn the attention of people working on metadata and description schemes. This is the area on which the ACTS STORit project will focus its activities. Why meta-data at home? For example to let users get information about the available programmes and interact with or program the system to search programmes that fit their personal profile. A system to allow the user to quickly and efficiently find the programme of interest is becoming more attractive as the number of available channels increases. Currently consumers normally get information about the programmes via printed magazines. Let us consider available mechanisms to electronically deliver descriptors to end-users.
Analog systems - Current systems for analog television services providing some sort of textual information are typically inserted in the Vertical Blanking Interval lines of the normal PAL or NTSC signal. In the U.S. this normally contains closed captions, i.e. subtitling service, while in Europe a more extended service is provided according to the Teletext standard. The description of the content of a TV program might be found on some of the pages of the teletext service, though the correct page number must be found as this is not standardised. There are however new solutions that aim at developing a sort of EPG for analog TV services as well. An example is the NextView standard (ETS 300 707) and will use hidden pages (from A00 to FFF) or packets (see ETS 300 708) in addition to the existing teletext transmission. The presence of private solutions in the market stresses the interest for such applications. In Germany, for example, TV Spielfilm has introduced a new service for delivery of descriptors about TV programs. The information is sent on the VBI channel and can be received with a small set top box, called I-box. The subscribers can also enjoy a personalised guide: the user can insert some data in an interest profile, based on which the machine will decide which programs are interesting for the user.
Digital Systems - In Europe the DVB consortium has introduced a standard called Service Information DVB-SI, (ETS 300 468), which is used for the Electronic Program Guides of Set Top Boxes. Similarly, the PSIP standard (Program and System Information Protocol) is adopted by ATSC. DVB-SI provides identification of services and events for the user and is organised in six tables. By making use of the information contained in these tables, especially the Service Description and the Event Information Table, an EPG can show program related information. An open standard for enabling users to search in a video database is a very relevant issue. DVB-SI was used as main example for MPEG7 at the opening seminar by L. Chiariglione, chairman of MPEG. Applications based on DVB-SI descriptors have been studied in the ACTS SMASH project: the results say that the current standard is insufficient. Several descriptors are missing, some relevant ones are not used or optional, the structure is not optimal, therefore extensions to this standard would be needed.
4. The ACTS STORit project
The STORit project aims at providing low cost solutions for enriching the typical broadcast services with additional data. The provision of metadata and links to other media in addition to the television signal will enable a richer and more fulfilling experience for those who wish to go beyond the superficial entertainment capabilities of these media. For the broadcaster, not only will it provide a domestic framework enabling the delivery of new program services, enhanced through the provision of more fulsome information about future programmes, including recommendations about programs on topics related to that of the currently-viewed programme, and an ability to provide supplementary information about a programme’s subject.
In order to satisfy the interests of each user, we foresee the usage of low-cost home storage devices as opposed to the realisation of networked solutions with on demand video servers: while the cost of storage decreases in a fast pace, the cost of bandwidth does not, therefore the costs of full interactive services as VOD systems will remain prohibitive still for some time. A broadband delivery channel such as DVB is ideally suited to deliver the service information commonly used by many users. Such information can be tailored locally to the wishes of each user. Intelligent software agents, based on user interest profiles, will take care of filtering relevant information on behalf of the user and help him/her to find interesting programmes from the ones which are on-air, scheduled, or stored on the system. The extracted descriptors will be stored in a database with references to the original video material. Basically this involves operations of filtering and search on broadcast programs, where both strongly benefit from the same sort of meta-information.
The future home videorecorder assumes therefore a new role as a home personal multimedia repository, allowing consumers to access personalised video programmes. Furthermore, the typical linear model of a broadcast program can become, thanks to the intermediating role of the storage device, an interactive system for e.g. infotainment applications. Such an enhanced consumer device will allow in particular IT illiterates to master the information overload and have a more user-friendly approach to information access and retrieval, which can enrich their vision of entertainment.
5. Technical aspects related to metadata
In order to realise this kind of services, the audio-visual material must be analysed and processed before being distributed (see fig.1) to generate the meta-information necessary for the retrieval operation. We will therefore study and develop new techniques for (semi-)automatic content extraction from audio-visual material, in form of key-information such as text, key-frames, video clips and will define procedures for the operators to generate metadata during the production process. The additional data will include production attributes, a formal representation of the script and cues for linking the content. The descriptor scheme will be developed in accordance with standards as MPEG7. This auxiliary information can be kept in a database by the content provider or the service provider, or created in real-time for live transmissions. The meta-data will be then streamed and distributed with the original content. Suitable protocols and formats for the encapsulation of meta-information in audio-visual streams will be defined.
One of the main objectives of the project is to develop a content description interface for home storage applications. The aim of such an interface and the underlying structure of metadata are to enhance the identification of material for viewing, recording or storage management and to fully describe the relationships and links to related material. Such a common interface specification would ensure that any level or subset of this structure could be interpreted correctly by automatic agents and by user interfaces.
On the consumer storage system, important challenges are represented by the modules for user profiling, filtering and indexing which should all be automated. Furthermore, easeness of use will be a key feature in the development of procedures for easy retrieval of content from the storage device, and management of stored material, much of which will be for one time usage.
As far as standards are concerned, the STORit project considers it as an important task to monitor and seek alliances with other projects that deal with similar technologies. We will consider contributions to international standardisation (such as MPEG7, W3C RDF) with requirements for AV consumer application. Most of them however almost exclusively look at the paradigm where all storage is at the providers side and thus not in the consumer’s home. We intend to specifically address the issues arising from storage at the user’s home and especially address the interoperability of services and interfaces. Will MPEG7 address these issues? And will it come in time to satisfy the requests of service providers and consumers? The MPEG7 standard will only be ready in the year 2001. At present the existing DVB-SI specification provides a basic framework of meta-information and describes a means of delivery. Perhaps extension of DVB-SI or a specific section of DVB-Data can be thought of for a short-term solution. We seek cooperation with interested parties (broadcasters, TV guide publishers, equipment manufacturers) to bring a viable solution in the short term.
Very important will be also the development of a suitable API. We intend to adopt where possible open international standards. Suitable API elements might be found in the DVB-SI API, the DSM-CC API, the MHEG-5 API, the Java embedded system API, and the MPEG-2 Section Filter API. In this respect we aim at co-operation with organisations which consider the home environment, such as DAVIC (High level protocols and In Home Digital Network) and the DVB MHP (Multimedia Home Platform).