Installation — Ego-Exo4D-HM

0. Install the Ego-Exo4D dataset

Download the Ego-Exo4D dataset (see their downloader docs, and the Download page here) — you'll need takes.json, captures.json, the takes/ directory with trajectory/gopro_calibs.csv per take, and annotations/ego_pose/ if you want to run evaluation. Then point the codebase at it:

export EGOEXO4D_DATASET=/path/to/EgoExo4d_dataset

slahmr/macros.py reads this at import time and raises immediately if it's missing, so every shell you run any pipeline script from needs this exported first.

1. Clone the repository and submodules

git clone --recursive https://github.com/Abhiram824/egoexo4d_human_meshes.git
cd egoexo4d_human_meshes
git submodule sync --recursive && git submodule update --init --recursive

submodule sync ensures submodule URLs/commits (e.g. third-party/ViTPose, pinned to a fork with an EGL rendering fix) match what's recorded in this repo.

2. System prerequisites

unzip (needed by download_models.sh, which doesn't check whether extraction succeeded) and a C/C++ toolchain + nvcc (needed to build detectron2, neural-renderer-pytorch, and lietorch from source).

3. Set up the environment

bash install_conda.sh

Or, with a virtualenv instead of conda:

bash install_pip.sh

If your conda is new enough to require accepting Terms of Service before conda create will run:

conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/main
conda tos accept --override-channels --channel https://repo.anaconda.com/pkgs/r

Why the install script looks the way it does

Useful if you're diffing against an older version or troubleshooting a from-scratch install:

Symptom	Cause	Fix
`ModuleNotFoundError: No module named 'torch'` building detectron2 (via PHALP or HaMeR)	pip's build isolation hides the already-installed torch from detectron2's `setup.py`	`--no-build-isolation` on the detectron2, PHALP, and HaMeR install lines
Same error for detectron2 specifically, even with `--no-build-isolation` on the PHALP line	newer pip doesn't propagate a command-line `--no-build-isolation` flag down to transitively resolved git dependencies (detectron2 is only discovered via PHALP's own `setup.py`)	install detectron2 standalone, explicitly, immediately before the PHALP line
`ModuleNotFoundError: No module named 'pip'` building chumpy	chumpy's `setup.py` does `import pip` directly, which fails in an isolated build env without pip	`--no-build-isolation` on `pip install -r requirements.txt`
`ModuleNotFoundError: No module named 'pkg_resources'` building mmcv, or `missing the 'build_editable' hook` installing HaMeR	very recent setuptools dropped `pkg_resources`, but HaMeR's/our own editable installs need setuptools ≥64 (PEP 660)	pin `setuptools==68.2.2` — the last version with both
`numpy.dtype size changed, may indicate binary incompatibility` importing mmpose/xtcocotools	unpinned numpy resolves to 2.x, breaking xtcocotools's compiled extension (built for numpy 1.x ABI)	`pip install numpy==1.26.4` as the last step, after `projectaria-tools` (which itself pulls numpy>=2 if installed after)
`json.decoder.JSONDecodeError: Unterminated string` parsing Aria calibration (cooking/bike takes only)	newer `vrs` builds truncate long tag values in their plain-text summary dump, corrupting the calib JSON the code parses out of it	pin `vrs=1.3.0`
`pip's dependency resolver ... opencv-python requires numpy>=2 ... incompatible`	expected side effect of the numpy 1.26.4 re-pin above	harmless warning, not a failure — this pipeline doesn't break at runtime with numpy 1.x

Finally, a small hack to make HaMeR consistent with SLAHMR's own path conventions:

sed -i '5s/".\/_DATA"/os.path.abspath(f"{__file__}\/..\/..\/..\/..\/_DATA")/' hamer/hamer/configs/__init__.py

4. Download models

./download_models.sh

or, equivalently:

gdown https://drive.google.com/uc?id=1GXAd-45GzGYNENKgQxFQ4PHrBp8wDRlW
unzip -q slahmr_dependencies.zip
rm slahmr_dependencies.zip

All models and checkpoints unpack into _DATA (body_models/{smpl,smplh}, humor_ckpts, vitpose_ckpts, droid.pth).

5. HaMeR's own checkpoints

A separate ~6GB download, no auth wall. fetch_demo_data.sh extracts relative to wherever you run it (hamer/_DATA/) — but the sed patch in step 3 points HaMeR's own code at the repo root's _DATA/, so merge them in:

cd hamer
bash fetch_demo_data.sh
cd ..
cp -rn hamer/_DATA/hamer_ckpts _DATA/
cp -rn hamer/_DATA/data _DATA/
cp -rn hamer/_DATA/vitpose_ckpts/* _DATA/vitpose_ckpts/
rm -rf hamer/_DATA

6. License-gated models (manual)

Two model files aren't included in any automated download and must be obtained by hand — both are one-time account registrations, not automatable:

MANO — register at mano.is.tue.mpg.de, download the MANO model, and place only MANO_RIGHT.pkl at _DATA/data/mano/MANO_RIGHT.pkl (the directory exists but is empty after step 5). MANO_LEFT.pkl is not needed — HaMeR only ever instantiates a right-hand model; left-hand crops are flipped into right-hand space before inference and flipped back after.
SMPL-X — register at smpl-x.is.tue.mpg.de (a separate account from MANO's), download the SMPL-X neutral model, and place it at _DATA/body_models/smplx/SMPLX_NEUTRAL.npz. This is needed even though the pipeline's body model is SMPL+H, not SMPL-X: BodyModel.__init__ borrows SMPL-X's hand PCA basis to fill in what SMPL+H's own model doesn't include. Without this file, everything through triangulation (stage 3) still works — it only fails once you reach the SLAHMR optimizer (stage 4), so it's easy to miss until then.

Verify the install with a test run

Once everything above is in place, confirm it works end-to-end on one short take:

export CUDA_VISIBLE_DEVICES=0
export EGL_DEVICE_ID=0
export PYOPENGL_PLATFORM=egl
unset DISPLAY WAYLAND_DISPLAY

python scripts/run_pipeline.py --video cmu_bike02_4 --device_num 0 --log-time

cmu_bike02_4 is a good choice: it's short (110 frames, ~13-14 minutes end-to-end on a single GPU) but still exercises the Aria/egocentric path (bike takes use Aria, same as cooking takes), so a clean run touches every stage — undistortion, camera params, per-camera detection/pose, HaMeR+MANO hand mesh, multi-view triangulation, and SLAHMR smooth_fit optimization with mesh rendering.

A successful run ends with:

Stitched 5 video(s) into outputs/cmu_bike02_4/final_vis
Pipeline completed successfully.
Timing log saved to: outputs/cmu_bike02_4/time_log.json

The presence of outputs/cmu_bike02_4/time_log.json is the project's success sentinel. You should also have 5 rendered videos in outputs/cmu_bike02_4/final_vis/*.mp4 with real, non-trivial file sizes — if any fail to render (in particular, pyrender/EGL errors), double-check the GPU environment variables above; a container without proper GPU/EGL passthrough (e.g. rootless Docker substitutes like udocker) can complete every other stage but fail specifically at mesh rendering.