This paper has been proposed for ECMAST'96 (May'96) by De Lang, Lagendijk, Persoon
but has NOT been selected by the ECMAST Programme Committee.
Authors: D. De Lang (Philips Research Eindhoven), R.L. Lagendijk (Technical University Delft), E.H.J. Persoon (Philips Research Eindhoven)
With the progression of advanced communication networks and systems, more and more services will be provided to the domestic consumer in digital format. Besides the currently emerging linear-viewing services such as video-on-demand and pay-television, a rapidly increasing number of non-television like information services will be provided such as digital catalogues and interactive multimedia documents, including text, audio and images. The feature of these typically interactive services is that they are often viewed in a non-linear fashion. Browsing through catalogues and other documents is a rather typical way of consuming the information contained therein. In order to make this possible for the domestic user in the future, three issues should be taken into consideration.
In the first place, local storage of the information being browsed should be provided whenever possible. There are several reasons for using local storage as opposed to continuously retrieving the required information over the network. The user can have fast access to the data independently of the network load, information can be downloaded during off-peak hours thus reducing costs, and re-transmission of the same information can be avoided. However, the capacity of existing (disk) storage systems does not match the needs of future multimedia information well. Therefore attention should be given to a storage architecture for domestic use that can satisfy those future needs.
A second issue that needs to be solved is a way to manage the (locally) stored information. Filing, retrieving, protecting and browsing systems tailored to the storage system should be developed, such that even within huge volumes of information specific pieces can be located easily. Existing navigation systems are all based on prior annotation of the information or search processes on textual information. It is widely recognised that there is a need for intelligent management and search methods for the visual information in multimedia documents and in digital video and image libraries. The problems at hand are that textual annotation can usually not made beforehand because of the complexity of this task and that visual information will be transmitted and stored in a compressed format hampering even the most simple image analysis operations.
The third issue is that should be investigated the way the domestic user wishes to access and use the delivered services. There are many new services that can be envisaged and for each service there are many possible ways in which the user may want to interact with and use the service. Therefore a study is needed on the possible services that can be realised using the combination of digital information provided by a network and the local storage of such information in the home. Next trial experiments must be carried out to find out how user wants to interact with those services.
In the European project SMASH (Storage for Multimedia Application Systems in the Home) investigations and verifications are carried out of domestic storage systems that integrate disk storage and tape storage into a single transparent system. In this way the advantages of disk storage (fast and random access) and tape storage (huge capacity, cheap per stored bit) can be effectively combined into a single system.
In this paper we will present the first ideas that have originated in the SMASH project. Since the project only started in September 1995 it is not possible to present already concrete results. Therefore the paper should be looked upon as a presentation of the projects' objectives and way in which we plan to reach those objectives.
In the introduction this was listed as the first topic that will be addressed in the project. At this moment, there is no storage architecture available that satisfies the future requirements for the multimedia services that will be delivered to the home. A first and important example is the variety of distributive services that will be delivered through the digital satellites and the associated services that will be provided through the digital CATV systems. There is the wish of the service providers to be able to download information to the homes in addition to the linear programmes that will be delivered through those channels. The bandwidth that is available through one transponder or one PAL channel is approximately 40 Mbit (5 Mbyte) per second. If such a channel is used to download one hour of information, a total storage capacity is needed of around 40 Gigabytes. One can see that both the required transfer speed (of 5 Mbytes per second) as well as the required storage capacity (of 40 Gigabytes) yields difficult requirements to a storage system for home use.
Besides those "down load" requirements there are requirements associated with the use of the downloaded information by the consumer. Ideally the consumer should be able to access the stored information in a very interactive way with fast response times. The only solution we have at this moment is to use high end fast Hard Disk Drives, which can reach a bandwidth of 40 Mbits per second. However if we would like to install 40 Gigabytes of such hard disk storage in the home the cost price would be by far prohibitive for consumer use. Therefore we propose a storage concept that consists of one Hard Disk Drive (HDD) and one Magnetic Tape Drive (MTD): the combination of the two storage units is called the Combo system.
Although the precise specifications of this Combo system are not yet finalised, one should think about an HDD of around 1 to 2 Gigabytes and a MTD of around 50 Gigabytes. As to the required bandwidth we would like that the "download" bandwidth is limited to some lower bandwidth than can be achieved through one transponder channel.
To make storage devices that can handle high bit rates is in principle possible but usually their is a significant price tag for such devices. Moreover one should take into account that the required bit rates are "sustained" bitrates and not "burst rates". To give an example, many hard disks have high burst rates but when it comes to sustained bit rates usually those are rather low. At this moment special so called A/V HDD's are being sold that offer higher sustained bit rates but again those drives are more expensive. Besides the performance of the basic storage system, there is also the operating system and file management system which will cause the sustained bandwidth to be rather limited. For example in the MS-DOS system the sustained bandwidth when writing files to an HDD is limited by the necessity to update the FAT (File Allocation Table) during the writing of the file. This update of the FAT affects the "sustained bit rate" significantly. Therefore we are currently studying what should be the preferable sustained bit rate which the Combo should be able to handle. This bit rate should definitely follow from mutual co-ordination with service providers that wish to download services to the home.
To the user the HDD and MTD should act as a transparent system. This means that the organisation and allocation of files should be done automatically by the Combo. Consequently, frequently and in a transparent manner files will have to be transferred between the HDD and MTD in a background manner, while at the same time the user may access (other) information from the HDD. This will put additional requirements on how the HDD and MTD are connected and how they are controlled.
The second important issue that needs to be resolved in the SMASH project is the way to manage the (locally) stored information. Three different aspects of this management are currently considered, namely:
Clearly, these technical issues need to be answered satisfactorily in order to realise the desired functionalities and user applications (see Section 3) on the Combo system described in the above Section 1.
Although this choice is not yet definitive, the current view of the Combo interconnection to other consumer devices is that it will be realised by the emerging standard IEEE 1394 external interconnect bus. The dominant reasons for selecting this bus are the isochronous data transfer service supported by this bus and the required compatibility with the DVB external interface. Through the IEEE 1394 interface, the Combo system may operate as slave of a PC or a set top box, or may operate (in a limited fashion) as master of other consumer equipment. The IEEE 1394 isochronous packets have been defined for DVC and MPEG data, while the definition for audio and computer data is under consideration. For the connection of the Combo system to the network, several options are considered, such a satellite distribution channels or ATM-based two-way communication systems based on HCF, FTTC or FTTH.
The SMASH Combo system will store a huge amount of multimedia information. To navigate through this information, especially where visual information is concerned, is a challenge in itself. Some services may provide table of contents including visual information pointers, such as the Electronic Programme Guide (EPG), but in general this information has to be extracted from the recorded information itself upon recording time. Efficient ways of browsing through large amounts of stored visual data is strongly related to emerging "digital video and image libraries". In the US, several projects under the Digital Library Initiative (DLI) sponsored by NSF, NASA and ARPA, concentrate on all issues related to digital libraries, among which the browsing of video and still picture information.
Notwithstanding the strong relation between the navigation of digital libraries and the visual search in the Combo system, additional requirements need to be taken into account for the search engine to be developed, due to the specific Combo architecture and interfacing. For instance, most bulk data will be stored on the tape-component of the Combo system. The hard disk of the Combo system should be used only for storing information that realises/improves the performance of the browsing process. To enable users to browse through the large amount of data stored on the Combo system, appropriate key information must be available or extracted. The user (interface) must be able to access this key information with short waiting time. The kind of key information that should be extracted depends on the sort of information that is coming in. Many types of information may come with already extracted key information, such as a table of contents. Availability and extraction of key information (key frames) from image sequences and still pictures is a less clear cut situation and requires special attention in the SMASH project.
In the current view on the system's concept, the extraction of key information takes
place on-line, while an off-line postprocessing step is needed to organise the extracted
key information. Various approaches for extraction key information are being considered.
These methods can be classified into methods for (i) extraction key information or key
frames from video sequences, (ii) extraction of key information and clustering thereof for
still pictures.
Video sequences can be represented well by a selection of important frames, or key frames. Key frames may refer to a single specific shot, or to a more shots from a single scene. The identification and extraction of key frames must be carried out on (JPEG/MPEG/...) compressed data. Methods for key frame finding, such as shot change detection are, however, content-based and typically work on the DCT-level (DC coefficients) or motion-vector level. Therefore a partial decoding of the incoming streams will be unavoidable in the visual search engine. Key frame extraction may sometimes lead to a huge number of extracted frames. Either an on-line regulation mechanism is required to control the extracted amount of information, or some off-line postprocessing is needed to prune the extracted information to a more reasonable amount. Storing of the key frames could be done in the form of subsampled frames, e.g. DC-images.
For extracting the key-information out of still pictures or key frames, feature extraction methods have been proposed. In this approach, the indexing of images is done by using some characteristic low-level features (texture, colour, shape), so that a search can be carried out on these features. Methods for feature extraction out of DCT-compressed image sets have been proposed for JPEG images. Most methods require partial decoding up to DCT-coefficient level.
The security of information stored on the Combo system is strongly related to conditional access definition in DVB/DAVIC for video services and to copyright issues for a wide variety of other services involving for instance computer programmes. Clearly the SMASH project cannot solve security issues just by itself. For this reason main efforts are towards developing a security concept that builds on earlier and current projects (such as the European projects ACCOPI, OKAPI, TALISMAN) and world-wide (standardisation) efforts (DAVIC, DVB, US Capstone project).
The main security issue to be addressed for the Combo system is that the mass storage device must be able to record data from several sources which need different security mechanisms. Services delivered from commercial service providers clearly need different protection (conditional access, copy protection) than that from for instance a camcorder. Protection mechanisms in the Combo must guarantee that service providers can keep control over locally stored data and that copyrights are not violated, but at the same time the increase in complexity of the Combo system to realise such functionalities should be relatively small.
The current view on the concept for the security of the Combo system is a protection mechanism in which service providers and consumers can keep control over their data even when stored on tape. This concept may include advanced data labelling techniques (somewhat like a successor of the serial copy management system (SCMS) for digital audio recordings) and encryption techniques. Notwithstanding our current view on the protection concept, the SMASH project will be heavily dependent on international standardisation efforts and political developments in this area.
Arising from the 'regular' use of local storage, the SMASH Combo supports a number of rather basic features, like the storing ('recording') and retrieving ('playing') of digital data. Next to that the combination of the random accessible hard disk unit and the huge capacitative tape drive offers features, that surpasses all known possibilities of regular storage devices like single hard disk units or tape drives.
Apart from well known playback functions such as Play, Pause, Fast Forward, Rewind and setting the playback speed, which are rather complex to implement with a compression standard like MPEG, advanced features like bookmarking, fast visual search, and link management have been investigated within.
To be more specific:
The ability of the SMASH device to use the combination of a magnetic tape drive (offering a huge storage capacity) and a (random accessible) hard disk offers extensive opportunities to record all kinds of data and media simultaneously in an intelligent way. For example while recording a movie on the tape drive, the SMASH device will be able to store an incoming multimedia call (limited in storage capacity) on the hard disk, coming from an answering machine application running on the set-top box. The recording process can be either a real-time process or a file download. In the first case the magnetic tape drive will be able to support the transfer rate of the incoming bit-stream; since the transfer rate of the magnetic tape drive is fixed, a solid-state buffer is used to accommodate several incoming bitrates. In the second case, requirements are less strict.
Flexible bitrates can be supported by the use of the hard disk as an extensive buffer for the tape unit. Both simultaneous recording of several data streams as well as simultaneous recording and playback is possible.
Tapes can be changed manually during real-time playing or recording. The hard disk offers the necessary buffer or pre-load functionality.
The table of contents of several tapes could be stored on the hard disk. To enable automatic management of different tapes: each tape must be given a code and the application running on the STB must be able to tell the user which tape it must insert in the Combo. Pointers within the file to other files can be supported. The efficiency of tape use increases, by rearranging of content, garbage removal, and removal of stuffing overhead.
Since access time on the tape can be long, methods to overcome this lack of responsiveness in the users' perception have been investigated. Likely possibilities are: presenting the user a low resolution version of some content cached on the hard disk and anticipation by the hard disk on a filed profile of the regular use.
Visual searching methods can be used for the tape content, since key frames can be stored on the hard disk to act as a visual table of content.
Extensive literature study of current and future use of storage requirements in the home have resulted in the following overview:
Figure 1: Future position of SMASH applications and services
Figure 1 reflects the conclusion of our study to the frequency of use and the tenability of associated content of storage applications. The area occupied by the circles is an indication for the required storage capacity.
Besides this the requirements for hugh storage capacity, the user interface of the operation, i.e. storing, retrieving and clearing, will set stringent requirements to the Combo System. At this time a preliminary design of the UI is being finalised and will be tested in March. The results of the user tests, resulting in the final functional requirements and overall architecture of the system are expected in April, and will be presented at the ECMAST conference in May.
contact: Dr. E.H.J. Persoon, project leader SMASH (AC018)
Philips Research Eindhoven, Prof. Holstlaan 4, 5656 AA Eindhoven, The Netherlands
phone: +31 40 27 42486, fax: +31 40 27 44648, e-mail: persoon@natlab.research.philips.com