1,140 audio-visual-inertial recordings of people interacting with containers (e.g. pouring a liquid in a cup; shaking a food box). 15 containers; 3 filling levels; 3 types of filling. RGB, depth, and infrared images from 4 views; multi-channel audio from an 8-element circular microphone array.
10,216 RGB images automatically sampled from the three fixed views of the public videos recordings of the CORSMAL Container Manipulation dataset, and capturing cups (4) and drinking glasses (4) as containers under different lighting and background conditions. Containers are completely visible or occluded by the person's hand.
An image-based dataset for category-level 6D object pose and size estimation with 138,240 pseudo-realistic composite RGB-D images of hand-held containers on top of 30 real backgrounds (mixed-reality set) and 3,951 RGB-D images selected from the CORSMAL Container Manipulation dataset.
Auditory and tactile signals of a Kuka IIWA robot with an Allegro hand holding a plastic container containing different materials. The robot manipulates the container with vertical shaking and rotation motions. The data consists of force/pressure measurements on the Allegro hand using a Tekscan tactile skin sensor, auditory signals from a microphone, and the joints data of the IIWA robot and the Allegro hand joints.
21 audio recordings of a person manipulating containers (e.g. pouring a liquid in a cup; shaking a food box). Recordings acquired with a different hardware setup, and environmental conditions. The dataset allows one to further validate audio-based classification models on a different settings.
219 configurations of 6 people manipulating and handing over 16 objects between each other in pairs. Objects include 4 drinking cups, 1 drinking glass, 1 mug, 1 food box, 1 pitcher and 8 common household objects. Data includes videos, poses (joints) and force sensors, all synchronised.
Jointly with Prof. Tamim Asfour's team at Karlsruhe Institute of Technology (SECONDHANDS EU H2020 project).
18 Convolutional Neural Networks (ResNet) for filling level classification from an RGB image. Models were trained with 6 different strategies, and under 3 data splits.
3 convolutional neural networks (ResNet-18) trained to classify the filling type and level of a container, used within the real-to-simulation framework. Reference publication: Towards safe human-to-robot handovers of unknown containers.
Convolutional Neural Networks specialised for classifying action, content type, and content level.
Classic ML models such as kNN, Support Vector Machine, and Random Forest are also included.
Reference publication: Audio Classification of the Content of Food Containers and Drinking Glasses.
5 Convolutional Neural Networks (ResNet-18 and ResNet-50) trained on CIFAR-10, CIFAR-100, ImageNet-100 and ImageNet using PRIME. One network is robustly trained on ImageNet-100 by combining DeepAugment + PRIME. Reference publication: PRIME: A few primitives can boost robustness to common corruptions.
An image-based multi-branch convolutional neural network re-trained on the CHOC dataset for the task of category-level 6D pose estimation on real hand-occluded containers.
