D4.3 Datasets for CANTATA project

In the context of the European CANTATA project, partners involved in multi content analysis validation methods combined their efforts to create a webpage to share knowledge about datasets (sets & metadata & ground truth & metrics...) for three different domains: surveillance, consumer electronic and medical. Datasets should fall into three categories (Surveillance, Medical, Consumer Applications)

Please follow the template below to provide detailed information:

If you want to add a new dataset, or if you have any comments, please contact Cédric Marchessoux ().


PETS 2000 Dataset:

  • PETS 2001
    • Websitehttp://www.cvg.cs.rdg.ac.uk/PETS2001/pets2001-dataset.html
    • Description of Dataset: Outdoor people and vehicle tracking (two synchronised views; includes omnidirectional and moving camera). PETS'2001 consists of five separate sets of training and test sequences, i.e. each set consists of one training sequence and one test sequence. All the datasets are multi-view (2 cameras) and are significantly more challenging than for PETS'2000 in terms of significant lighting variation, occlusion, scene activity and use of multi-view data.
    • Description of Ground Truth/Metadata: Tracking information on image plane and ground plane can be found at: http://www.cvg.cs.rdg.ac.uk/PETS2001/ANNOTATION/
    • Contextual info: Camera Calibration provided
    • Results from metrics and ground truth: Centroid and bounding box coordinates on image plane, object class (person, vehicle, other), position on ground plane and object orientation.
    • Comments:
    • Information on Copyright: Free download from website
    • Contact person from Cantata: Dimitrios Makris,
PETS 2001 Dataset 1:
PETS 2001 Dataset 2:
PETS 2001 Dataset 3:
PETS 2001 Dataset 4:
PETS 2001 Dataset 5:

  • PETS 2002- VISOR BASE: Moving People
    • Website: http://www.cvg.cs.rdg.ac.uk/PETS2002/pets2002-db.html
    • Description of Dataset: Indoor people tracking (and counting). Two training and four testing sequences consist of people moving in front of a shop window. Sequences are provided as both MPEG movie format and as individual JPEG images.
    • Description of Ground Truth/Metadata: People tracking, counting and activity recognition.
    • Contextual info: No calibration
    • Results from metrics and ground truth:  How many people are passing in front of the shop window,  how many people stop and look into the window,  how many people are looking into the window at each instant (frame) in time,  the trajectories of people passing in front of the store,  the time spent per frame (processing time):  a histogram of the microseconds spent processing each frame.
    • Comments:
    • Information on Copyright: Free download from website
    • Contact person from Cantata: Dimitrios Makris,
PETS 2002 - VISOR BASE Dataset:

  • PETS-ICVS'2003 - FGnet
    • Website: http://www.cvg.cs.rdg.ac.uk/PETS-ICVS/pets-icvs-db.html
    • Description of Dataset: Smart meeting, that includes facial expressions, gaze and gesture/action. The environment consists of three cameras: one mounted on each of two opposing walls, and an omnidirectional camera positioned at the centre of the room. The dataset consists of four scenarios.
    • Description of Ground Truth/Metadata: a) Eye positions of people in Scenarios A, B and D. (every 10th frame is annotated). b)  Facial expression and gaze estimation for Scenarios A and D, Cameras 1-2. c) Gesture/action annotations for Scenarios B and D, Cameras 1-2.
    • Contextual info: Camera Calibration provided.
    • Results from metrics and ground truth: For each frame, the requirement is to perform:face localisation (centre location of eyes), recognition of facial expression, recognition of face/hand gesture, estimation of face/head direction (gaze),  recognition of actions.
    • Comments:
    • Information on Copyright: Free download
    • Contact person from Cantata: Dimitrios Makris,
PETS-ICVS'2003 - FGnet Dataset:

  • VS-PETS'2003 - INMOVE
    • Website: http://www.cvg.cs.rdg.ac.uk/VSPETS/vspets-db.html
    • Description of Dataset: Outdoor people tracking - football data (three synchronised views). The datasets consists of football players moving around a pitch.
    • Description of Ground Truth/Metadata: Tracking information on image plane for camera 3 can be found at: http://www.cvg.cs.rdg.ac.uk/VSPETS/Camera3Xml.zip.  An AVI file of the ground truth for camera view 3 is also available at http://www.cvg.cs.rdg.ac.uk/VSPETS/Cam3_Gt.avi
    • Contextual info
    • Results from metrics and ground truth: The location of each player on the pitch, for each frame of the sequence. For each player, the bounding box (with origin bottom left) in pixels should be determined.  The position of the player is defined as the middle bottom of the bounding box (in pixels).
    • Comments:
    • Information on Copyright: Free download from website
    • Contact person from Cantata: Dimitrios Makris,
VS-PETS'2003 - INMOVE Camera Setting:
VS-PETS'2003 - INMOVE Dataset Views: 

PETS-ECCV'2004 - CAVIAR Datasets:

PETS'2006 - ISCAPS Dataset:

PETS'2007 - REASON Dataset:

BEHAVE dataset with ground truth bounding boxes:

i-LIDS Abandoned Package Scenario:
i-LIDS Doorway Surveillance Scenario:
i-LIDS Parked Vehicle Scenario Scenario:
i-LIDS Sterile Zone Scenario:
IBM Indoor Dataset:


Backgrounds  Signal: masses        Signals: micro calcifications  

    • Website: http://www.MyPACS.net

    Description of dataset: MyPACS.net is still free, and it now has over 16,500 teaching files contributed by 14,000 registered users.  With 75,000 key images categorized by anatomy and pathology, you can quickly find examples of any disease.   The web-based viewer has been improved with more PACS-like features, and it still works instantly in your browser, requiring nothing to download:

    1. You can now send DICOM directly from your PACS to MyPACS.net.  Full studies are anonymized and sent through your firewall to MyPACS.net, where you can share them with anybody.  See http://www.mypacs.net/dar.
    2. You can now adjust windows/levels on every image in MyPACS, and scroll through series as if you were on a PACS.
    3. Users can rate cases, and highly rated cases show up first in search results.
    4. It's now easier to export your cases to PowerPoint.   Just select "PowerPoint" from the case options menu.
    5. Need a second opinion?  Upload images to MyPACS.net and select "solicit opinions". Hundreds of volunteer radiologists will be invited to comment on your case.
    6. Want to participate in international outreach?  Sign up as a volunteer, and you will receive email whenever cases matching your expertise are submitted by physicians in developing nations.

    • Description of Ground Truth/Metadata: Description of Ground Truth/Metadata: Description of the pathology by medical doctors.
    • Contextual info: environment conditions (calibration, scene...): Medical modality described: Brand and acquisition conditions
    • Results from metrics and ground truth: none


    1. Cranium and Contents (1205)
    2. Face and Neck (398)
    3. Spine and Peripheral Nervous System (504)
    4. Skeletal System (3433)
    5. Heart (160)
    6. Chest (894)
    7. Gastrointestinal (1271)
    8. Genitourinary (800)
    9. Vascular/Lymphatic (416)
    10. Breast (62)
    11. Other (458) 
    • Information on Copyright: MyPACS.net is still free, you need to be registered.
    • Contact person from Cantata: Cédric Marchessoux ()

      Consumer Applications