FreeMan Documentation

Overall Structure

FreeMan consists of 2 subsets corresponding to 30FPS & 60FPS, respectively. Each subset data includes RGB videos, camera parameters, 2D & 3D human keypoints, SMPL parameters and bounding boxes. File structure is shown as below. Data are grouped into different folders by data types.
yyyymmdd_xxxxxxxx_subjNN is the unique name for each session. yyyymmdd refer to capture date, xxxxxxxx refer to uuid of the session, NN refer to subject id.

              
              FreeMan
                ├── 30FPS
                │   ├── bbox2d
                │   │   ├── yyyymmdd_xxxxxxxx_subjNN.npy
                │   │   └── ...
                │   ├── keypoints2d
                │   │   ├── yyyymmdd_xxxxxxxx_subjNN.npy
                │   │   └── ...
                │   ├── keypoints3d
                │   │   ├── yyyymmdd_xxxxxxxx_subjNN.npy
                │   │   └── ...
                │   ├── motions
                │   │   ├── yyyymmdd_xxxxxxxx_subjNN_view0.npy
                │   │   ├── ...
                │   │   ├── yyyymmdd_xxxxxxxx_subjNN_view8.npy
                │   │   └── ...
                │   ├── cameras
                │   │   ├── yyyymmdd_xxxxxxxx_subjNN.json
                │   │   └── ...
                │   └── videos
                │        ├── yyyymmdd_xxxxxxxx_subjNN
                │        │   ├── cameras.json
                │        │   ├── chessboard.pkl
                │        │   └── vframes
                │        │       ├── c01.mp4
                │        │       ├── ...
                │        │       └── c08.mp4
                │        ├── ...
                ├── 60FPS
                │   ├── ...
                
            

Dataset Splits

FreeMan provides train, val, test splits. Session ID of each splits are stored in train.txt, val.txt and test.txt, respectively. Each line of these text files includes a session ID in corresponding subset.

RGB Frames

For each session, videos are stored in videos folder. Each video is named as cXX.mp4, where XX is the camera id. cameras.json and chessboard.pkl are camera parameters and chessboard data for calibration, respectively. Videos are in resolution of 1920 * 1080 and have framerate of 30/60 FPS.

Camera Parameters

Cameras in FreeMan follows Pinhole Camera pattern. Camera parameters for all sessions are stored in cameras and are stored in JSON format. cameras.json contains 8 camera parameters, each of which is a dictionary with 6 keys: name, size, matrix, rotation, translations and distortions. Parameters follow OpenCV convention and are explained as below:

              
              [
              {
                "name": "c01",
                "size": [
                    frame width,
                    frameheight
                ],
                "matrix": [
                    [
                        focal length x,
                        0.0,
                        center point x
                    ],
                    [
                        0.0,
                        focal length y,
                        center point y
                    ],
                    [
                        0.0,
                        0.0,
                        1.0
                    ]
                ],
                "rotation": [
                    Rotation1,
                    Rotation2,
                    Rotation3
                ],
                "distortions": [
                    P1,
                    P2,
                    P3,
                    P4,
                    P5
                ],
                "translation": [
                    translation_x,
                    translation_y,
                    translation_z
                ]
              },
              {

              },
              ...
              ]
              
            

Pose Annotations

Skeleton

Up to now, FreeMan provides keypoint annotations in COCO format, which is widely used and defines 17 human joints. Both 2D & 3D pose annotations are stored in numpy format. 2D keypoints for a session are in shape of [C, F, J, 2], where C is camera index, F is frame index, J is joint index, 2 is x, y coordinates. 3D keypoints for a session are in shape of [F, J, 3], where F is frame index, J is joint index, 3 is x, y, z coordinates.
2D & 3D are stored in keypoints2d & keypoints3d folders, respectively. Each file is named as yyyymmdd_xxxxxxxx_subjNN.npy, where yyyymmdd refer to capture date, xxxxxxxx refer to uuid of the session, NN refer to subject id.

SMPL parameters are stored in motions folder. Each file is named as yyyymmdd_xxxxxxxx_subjNN_viewX.npy, where yyyymmdd refer to capture date, xxxxxxxx refer to uuid of the session, NN refer to subject id, X refer to camera id. Each file is in shape of [F, 85], where F is frame index, 85 is SMPL parameters.