Alpha test before public (#3)

* Revise getting_started

* add README files

* add mvpose doc

* [Fix] Fix doc, dockerfile, scripts and github action (#1)

* Test codecov

* [Fix] Fix outdata URL, and advices for docs (#5)

* Add docs and benchmark for MvP (#2)

* [Add] Add download links for checkpoints

* [Fix] Fix hyperlinks, fix default dirs and filenames in configs and scripts

* [Add] Add a small test set for inference/demo

* Add docs and benchmark for MVPose and MVPose tracking (#6)

* [Fix] Update docs for sperson in getting started (#7)

* Revise mvp inference

* Bump version to v0.5.0

Co-authored-by: lijiaqi2 <[email protected]>
Co-authored-by: gaoyang3 <[email protected]>
Co-authored-by: LazyBusyYang <[email protected]>
Co-authored-by: wqyin <[email protected]>
Co-authored-by: yinwanqi <[email protected]>
Co-authored-by: jiaqiAA <[email protected]>

Author: 5 people

.github/workflows/build.yml

@@ -6,6 +6,7 @@ name: build
 on:
   push:
     branches:
+     - alpha_test # remove it after public
      - main
     paths-ignore:
       - 'README.md'
@@ -23,56 +24,54 @@ concurrency:
   cancel-in-progress: true
 
 jobs:
-  build_cuda101:
+  build_cuda102:
     runs-on: ubuntu-18.04
-    strategy:
-      matrix:
-        python-version: [3.7]
-        torch: [1.7.0, 1.8.0]
-        include:
-          - torch: 1.7.0
-            torchvision: 0.8.1
-          - torch: 1.8.0
-            torchvision: 0.9.0
     defaults:
       run:
         shell: bash -l {0}
-
+    container:
+      image: openxrlab/xrmocap_runtime:ubuntu1804_x64_cu102_py38_torch181_mmcv153
+      credentials:
+        username: ${{secrets.DOCKERHUB_USERNAME}}
+        password: ${{secrets.DOCKERHUB_PWD}}
     steps:
       - uses: actions/checkout@v2
-      - uses: conda-incubator/setup-miniconda@v2
-        with:
-          activate-environment: xrmocap_env
-          python-version: ${{matrix.python-version}}
-          auto-activate-base: false
-      - name: Prepare test data
+      - name: Show conda env
         run: |
-          # TODO: prepare test data or remove this step
-      - name: Upgrade pip
-        run: pip install pip --upgrade
-      - name: Install ffmpeg
-        run: |
-          conda install ffmpeg
-          ffmpeg -version
-      - name: Install PyTorch
+          source /root/miniconda3/etc/profile.d/conda.sh && conda deactivate
+          conda info -e
+          conda activate openxrlab
+          conda list | grep torch
+          conda list | grep mmcv
+      - name: Prepare test data
         run: |
-          conda install pytorch==${{matrix.torch}} torchvision==${{matrix.torchvision}} cudatoolkit=10.1 -c pytorch
-      - name: Install MMCV
+          sh scripts/download_test_data.sh
+      - name: Prepare weight
         run: |
-          pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cpu/torch${{matrix.torch}}/index.html
-          python -c 'import mmcv; print(mmcv.__version__)'
-      - name: Install other dependencies
-        run: pip install -r requirements.txt
+          sh scripts/download_test_data.sh
+          mkdir weight
+          cd weight
+          wget https://download.openmmlab.com/mmdetection/v2.0/faster_rcnn/faster_rcnn_r50_fpn_1x_coco/faster_rcnn_r50_fpn_1x_coco_20200130-047c8118.pth
+          wget https://download.openmmlab.com/mmpose/top_down/hrnet/hrnet_w48_coco_wholebody_384x288_dark-f5726563_20200918.pth
+          wget https://download.openmmlab.com/mmtracking/mot/faster_rcnn/faster-rcnn_r50_fpn_4e_mot17-half-64ee2ed4.pth
+          wget https://download.openmmlab.com/mmtracking/mot/reid/tracktor_reid_r50_iter25245-a452f51f.pth
+          cd ..
       - name: Build and install
-        run: rm -rf .eggs && pip install -e .
+        run: |
+          rm -rf xrmocap.egg-info
+          source /root/miniconda3/etc/profile.d/conda.sh && conda activate openxrlab
+          pip install xrprimer
+          pip install -e .
       - name: Run unittests and generate coverage report
         run: |
+          source /root/miniconda3/etc/profile.d/conda.sh && conda activate openxrlab
           coverage run --source xrmocap -m pytest tests/
           coverage xml
           coverage report -m
       - name: Upload coverage to Codecov
-        uses: codecov/codecov-action@v2
+        uses: codecov/codecov-action@v3
         with:
+          token: ${{ secrets.CODECOV_TOKEN }} # deprecated after public
           files: ./coverage.xml
           flags: unittests
           env_vars: OS,PYTHON

.github/workflows/publish-to-pypi.yml

@@ -19,4 +19,4 @@ jobs:
       - name: Publish distribution to PyPI
         run: |
           pip install twine
-          twine upload dist/* -u __token__ -p ${{ secrets.pypi_password }}
+          twine upload dist/* -u __token__ -p ${{ secrets.PYPI_PASSWORD }}

.pre-commit-config.yaml

@@ -37,6 +37,7 @@ repos:
     rev: v2.1.0
     hooks:
       - id: codespell
+        args: ["--ignore-words-list", "mot"]
   - repo: https://github.com/myint/docformatter.git
     rev: v1.3.1
     hooks:

Dockerfile

@@ -42,21 +42,19 @@ RUN . /root/miniconda3/etc/profile.d/conda.sh && \
     conda activate openxrlab && \
     pip install pre-commit interrogate coverage pytest && \
     pip install mmcv-full==1.5.3 -f https://download.openmmlab.com/mmcv/dist/cu102/torch1.8.1/index.html && \
-    pip install xrprimer -i https://repo.sensetime.com/repository/pypi/simple && \
+    pip install xrprimer && \
     pip cache purge
 
 # Install build requirements
 RUN . /root/miniconda3/etc/profile.d/conda.sh && \
     conda activate openxrlab && \
-    wget http://10.4.11.59:18080/resources/XRlab/requirements/xrmocap/build.txt && \
-    pip install -r build.txt && rm build.txt && \
+    pip install -r https://github.com/openxrlab/xrmocap/blob/main/requirements/build.txt && \
     pip cache purge
 
 # Install test requirements
 RUN . /root/miniconda3/etc/profile.d/conda.sh && \
     conda activate openxrlab && \
-    wget http://10.4.11.59:18080/resources/XRlab/requirements/xrmocap/test.txt && \
-    pip install -r test.txt && rm test.txt && \
+    pip install -r https://github.com/openxrlab/xrmocap/blob/main/requirements/test.txt && \
     pip cache purge
 
 # Install mmhuman3d
@@ -73,3 +71,11 @@ RUN . /root/miniconda3/etc/profile.d/conda.sh && \
     pip uninstall opencv-python opencv-python-headless -y && \
     pip install opencv-python-headless && \
     pip cache purge
+
+# Clone xrmocap and install
+RUN . /root/miniconda3/etc/profile.d/conda.sh && \
+    conda activate openxrlab && \
+    cd /workspace && \
+    git clone https://github.com/openxrlab/xrmocap.git && \
+    cd xrmocap && pip install -e . && \
+    pip cache purge

README.md

@@ -1,5 +1,13 @@
 # XRMocap
 
+<div align="left">
+
+[![actions](https://github.com/openxrlab/xrmocap/workflows/build/badge.svg)](https://github.com/openxrlab/xrmocap/actions)
+[![codecov](https://codecov.io/gh/openxrlab/xrmocap/branch/main/graph/badge.svg)](https://codecov.io/gh/openxrlab/xrmocap)
+[![PyPI](https://img.shields.io/pypi/v/xrmocap)](https://pypi.org/project/xrmocap/)
+[![LICENSE](https://img.shields.io/github/license/openxrlab/xrmocap.svg)](https://github.com/openxrlab/xrmocap/blob/main/LICENSE)
+
+</div>
 
 ## Introduction
 
@@ -9,7 +17,7 @@ XRMoCap is an open-source PyTorch-based codebase for the use of multi-view motio
 
 If you are interested in single-view motion capture, please refer to [mmhuman3d](https://github.com/open-mmlab/mmhuman3d) for more details.
 
-**TODO:[Insert demo here]**
+https://user-images.githubusercontent.com/26729379/187710195-ba4660ce-c736-4820-8450-104f82e5cc99.mp4
 
 A detailed introduction can be found in [introduction.md](./docs/en/tutorials/introduction.md).
 
@@ -62,7 +70,7 @@ Supported datasets:
 
 ## Getting Started
 
-Please see [getting_started.md](docs/getting_started.md) for the basic usage of XRMoCap.
+Please see [getting_started.md](docs/en/getting_started.md) for the basic usage of XRMoCap.
 
 ## License
 

configs/mvp/README.md

@@ -19,32 +19,35 @@ We provide the config files for MvP: [Direct multi-view multi-person 3d pose est
 
 ## Results and Models
 
-We evaluate MvP on 3 popular benchmarks.
-
-[Explain common setting and metrics]
+We evaluate MvP on 3 popular benchmarks, report the Percentage of Correct Parts (PCP) on Shelf and Campus dataset, Mean Per Joint Position Error (MPJPE), mAP and recall on CMU Panoptic dataset.
 
 ### Campus
 
-[Explain specific setting, metrics and other details (finetuned from pretrained cmu panoptic xxx)]
+MvP for Campus fine-tuned from the model weights pre-trained with 3 selected views in CMU Panoptic dataset is provided. Fine-tuning with the model pre-train with CMU Panoptic HD camera view 3, 6, 12 gives the best final performance on Campus dataset.
 
 | Config | Campus  | Download |
 |:------:|:-------:|:--------:|
-| - | - | [model]() | [log]() |
+| [mvp_campus.py](./campus_config/mvp_campus.py) | 96.7 | [model](https://openxrlab-share.oss-cn-hongkong.aliyuncs.com/xrmocap/weight/mvp/xrmocap_mvp_campus-e6093968_20220831.pth) |
 
 
 ### Shelf
 
-[Explain specific setting, metrics and other details (finetuned from pretrained cmu panoptic xxx)]
+MvP for Shelf fine-tuned from the model weights pre-trained with 5 selected views in CMU Panoptic dataset is provided. The 5 selected views, HD camera view 3, 6, 12, 13 and 23 are the same views used in VoxelPose.
 
 | Config | Shelf  | Download |
 |:------:|:-------:|:--------:|
-| - | - | [model]() | [log]() |
+| [mvp_shelf.py](./shelf_config/mvp_shelf.py)  | 97.1 | [model](https://openxrlab-share.oss-cn-hongkong.aliyuncs.com/xrmocap/weight/mvp/xrmocap_mvp_shelf-22d1b5ed_20220831.pth)  |
 
 
 ### CMU Panoptic
 
-[Explain view selection, training details etc.]
+MvP for CMU Panoptic trained from stcratch with pre-trained Pose ResNet50 backbone is provided. The provided model weights were trained and evaluated with the 5 selected views same as VoxelPose (HD camera view 3, 6, 12, 13, 23).  A checkpoint trained with 3 selected views (HD camera view 3, 12, 23) is also provided as the pre-trained model weights for Campus dataset fine-tuning.
+
+| Config | AP25 | AP100 | Recall@500 | MPJPE(mm) |Download |
+|:------:|:----:|:----:|:---------:|:--------:|:--------:|
+| [mvp_panoptic.py](./panoptic_config/mvp_panoptic.py) | 91.5 | 97.9 | 99.85 |16.45 | [model](https://openxrlab-share.oss-cn-hongkong.aliyuncs.com/xrmocap/weight/mvp/xrmocap_mvp_panoptic_5view-1b673cdf_20220831.pth) |
+| [mvp_panoptic_3cam.py](./panoptic_config/mvp_panoptic_3cam.py) | 54.7 | 95.1 | 98.83 |30.55 | [model](https://openxrlab-share.oss-cn-hongkong.aliyuncs.com/xrmocap/weight/mvp/xrmocap_mvp_panoptic_3view_3_12_23-4b391740_20220831.pth)  |
+
+### Pose ResNet50 Backbone
 
-| Config | AP25 | MPJPE(mm) | Download |
-|:------:|:----:|:---------:|:--------:|
-| - | - | - | [model]() | [log]() |
+All the checkpoints provided above were trained on top of the pre-trained [Pose ResNet50](https://openxrlab-share.oss-cn-hongkong.aliyuncs.com/xrmocap/weight/mvp/xrmocap_pose_resnet50_panoptic-5a2e53c9_20220831.pth) backbone weights.

configs/mvp/campus_config/mvp_campus.py

@@ -33,7 +33,7 @@
     end_epoch=25,
     pretrained_backbone='',
     model_root='./weight',
-    finetune_model='convert_model_best_31223.pth.tar',
+    finetune_model='xrmocap_mvp_panoptic_3view_3_12_23.pth.tar',
     resume=False,
     lr_decay_epoch=[10],
     inference_conf_thr=[0.0],

configs/mvp/panoptic_config/mvp_panoptic.py

@@ -30,7 +30,7 @@
     weight_decay=1e-4,
     optimizer='adam',
     end_epoch=200,
-    pretrained_backbone='convert_pose_resnet50_panoptic.pth.tar',
+    pretrained_backbone='xrmocap_pose_resnet50_panoptic.pth.tar',
     model_root='./weight',
     finetune_model=None,
     resume=False,

configs/mvp/panoptic_config/mvp_panoptic_3cam.py

@@ -30,7 +30,7 @@
     weight_decay=1e-4,
     optimizer='adam',
     end_epoch=200,
-    pretrained_backbone='convert_pose_resnet50_panoptic.pth.tar',
+    pretrained_backbone='xrmocap_pose_resnet50_panoptic.pth.tar',
     model_root='./weight',
     finetune_model=None,
     resume=False,

configs/mvp/shelf_config/mvp_shelf.py

@@ -33,7 +33,7 @@
     end_epoch=30,
     pretrained_backbone='',
     model_root='./weight',
-    finetune_model='convert_model_best_5view.pth.tar',
+    finetune_model='xrmocap_mvp_panoptic_5view.pth.tar',
     resume=False,
     lr_decay_epoch=[30],
     inference_conf_thr=[0.0],