Motion capture used to need a suit, markers or a pricey app. InstantHMR does it from a plain webcam or your phone camera, right inside ComfyUI. It finds every person in the frame and recovers a full 3D body (pose, shape, keypoints and mesh) in about 1 millisecond per person on a desktop GPU.
The new ComfyUI-InstantHMR node pack wraps it in three nodes, including a live capture node that can drive a VRM or Mixamo character, stream to VSeeFace or Blender over VMC, and record takes. This guide covers install, the five checkpoints and their licences, all three workflows, live phone capture over HTTPS, exports, and fixes.
- What: a 20-32M parameter 3D human mesh recovery model from Naturalpad Studio, in ComfyUI via jtydhr88/ComfyUI-InstantHMR.
- Output: the same MHR body as ComfyUI’s native SAM 3D Body nodes, so you can render meshes, OpenPose 2D/3D and silhouettes, and export animated GLB or BVH.
- Files: five checkpoints (
instanthmr_nanotoinstanthmr_medium) from Hugging Face intoComfyUI/models/detection/. - Licence catch: four checkpoints are CC BY-NC 4.0 (non-commercial). Only
instanthmr_medium_commercialis Apache-2.0, at lower accuracy. - Needs: ComfyUI 0.34.0 or newer (native SAM 3D Body nodes). For phone capture, run ComfyUI over HTTPS.
New to ComfyUI? My ComfyUI beginner guide covers install, models, nodes and fixes in one place.
What InstantHMR Is
HMR stands for human mesh recovery: estimating a full 3D body from a normal 2D image. InstantHMR is a tiny, fast HMR model from Naturalpad Studio, released in early November 2026 with a paper (HAL id hal-05854387) and training code at Naturalpad-studio/InstantHMR.
For each person it predicts:
- Pose and body shape
- 70 keypoints in 2D and 3D
- A full MHR mesh (the body model used by Meta’s SAM 3D Body)
Speed is the selling point: about 1 ms per person on a desktop GPU and about 5 ms on a phone NPU, according to the project. That is fast enough for live preview, where the heavier SAM 3D Body is better for offline quality.
Because the output matches ComfyUI’s native SAM 3D Body format (MHR_POSE_DATA), everything downstream (smoothing, rendering, GLB and BVH export) uses core ComfyUI nodes.
The 5 Checkpoints and Licences
| File | Params | Backbone | 3DPW PA-MPJPE (lower is better) | Licence |
|---|---|---|---|---|
instanthmr_nano.safetensors |
20.0M | HGNetV2-B0 | 46.6 mm | CC BY-NC 4.0 |
instanthmr_tiny.safetensors |
20.3M | HGNetV2-B1 | 44.0 mm | CC BY-NC 4.0 |
instanthmr_small.safetensors |
24.4M | HGNetV2-B2 | 41.7 mm | CC BY-NC 4.0 |
instanthmr_medium.safetensors (default) |
32.2M | HGNetV2-B4 | 40.0 mm | CC BY-NC 4.0 |
instanthmr_medium_commercial.safetensors |
32.2M | HGNetV2-B4 | 50.7 mm | Apache-2.0 |
Which to pick:
- Personal projects, best accuracy:
instanthmr_medium. - Weak GPU or laptop live preview:
instanthmr_nanoortiny. - Anything commercial (paid client work, a product, monetised channel assets):
instanthmr_medium_commercial. It was trained on data that allows commercial use, which costs some accuracy. The node pack code and the MHR body model are Apache-2.0.
PA-MPJPE is the average joint error after aligning the body, measured on the 3DPW benchmark. A 10 mm gap is a real drop in accuracy, so test the commercial checkpoint on your own footage before committing to it.
Install the Nodes and Models
Step 1: Update ComfyUI
InstantHMR builds on the native SAM 3D Body nodes that arrived in ComfyUI 0.34.0. RT-DETR, SAM3 and MoGe nodes, used in the shipped graphs, are also core. See how to update ComfyUI.
Step 2: Install the node pack
cd ComfyUI/custom_nodes
git clone https://github.com/jtydhr88/ComfyUI-InstantHMR
Restart ComfyUI. If it shows up in ComfyUI Manager on your build, installing from there works too.
Step 3: Download the checkpoints
Get them from jtydhr88/ComfyUI-InstantHMR on Hugging Face and put them in ComfyUI/models/detection/:
cd ComfyUI/models
hf download jtydhr88/ComfyUI-InstantHMR instanthmr_medium.safetensors --local-dir detection
Swap the file name for instanthmr_medium_commercial.safetensors if you need commercial use. Your folder should look like:
ComfyUI/models/detection/
instanthmr_medium.safetensors
Step 4: SAM 3D Body loader (for some exports)
The body_mesh and bones_only render styles and BVH export need a SAM 3D Body loader on the same graph. Get sam_3d_body_dinov3_bf16.safetensors from Comfy-Org/sam-3d-body and place it where your ComfyUI’s SAM 3D Body loader looks for it (the native template shows the folder).
The 3 Nodes Explained
| Node | What it does | Key inputs / output |
|---|---|---|
| Load InstantHMR Model | Loads a checkpoint from models/detection |
checkpoint name |
| Run InstantHMR Prediction | One MHR body per person box | Person boxes from RT-DETR Detect (class person) or SAM3 Video Track; fov. Output: MHR_POSE_DATA |
| InstantHMR Live Capture | Live 2D/3D preview from a webcam or phone, drives a rigged character, records clips | model, detector, camera, recording, character, vmc; advanced detect_every, fov. Output: VIDEO |
Settings that matter:
- fov: the camera’s vertical field of view in degrees. At 0 it guesses (focal length = 1.05x the image diagonal). For video, chain Load MoGe Model and the MoGe FoV node for a measured value.
- detect_every: run the person detector every N frames. 2-3 keeps live preview smooth.
- smoothing: 0.5 on the native smoothing node is the suggested starting value for removing jitter.
With no boxes connected, the whole frame is treated as one person, which is fine for single-person webcam use.
Image, Video and Live Workflows
Three example graphs ship in example_workflows/:
| Workflow | Pipeline | Use it for |
|---|---|---|
instanthmr_image.json |
RT-DETR boxes on a photo, meshes drawn over it | Testing, pose reference images |
instanthmr_video.json |
SAM3 tracking, MoGe FoV, smoothing, rendered video and a SCAIL GLB | Turning a video clip into animation |
instanthmr_live.json |
Runs the pipeline on a clip recorded with Live Capture | Webcam mocap takes |
Drag a JSON onto the canvas, pick your checkpoint in the loader, load an image or video, and click Run. For several people in one video, keep the SAM3 tracking step so each person keeps the same index across frames.
The OpenPose renders are handy as control images for pose LoRAs, such as the ones in our MOSS-Image control LoRA guide.
Live Capture From a Phone (HTTPS)
The InstantHMR Live Capture node has two camera sources:
- This device: the browser webcam on the PC running ComfyUI. Works on
localhostwith no extra setup. - Phone / other device: shows a QR code; scan it on a phone on the same network to stream its camera.
Browsers only allow camera access on localhost or over HTTPS, so phone capture needs ComfyUI to run with a certificate:
- Create a self-signed certificate and include your PC’s LAN address (for example 192.168.1.20) in
subjectAltName. With OpenSSL:openssl req -x509 -newkey rsa:2048 -nodes -days 365 \ -keyout comfy.key -out comfy.crt -subj "/CN=comfyui" \ -addext "subjectAltName=IP:192.168.1.20" - Start ComfyUI listening on your network with TLS:
python main.py --listen 0.0.0.0 --tls-keyfile comfy.key --tls-certfile comfy.crt - Open
https://192.168.1.20:8188on the PC and accept the certificate warning once. Do the same on the phone after scanning the QR code.
Recorded takes save to ComfyUI/input/instanthmr_live/.
Security: --listen 0.0.0.0 makes ComfyUI reachable by every device on your network. Only do it on a network you trust, never forward the port, and close it when you finish. Keep your node packs patched too; see our ComfyUI-Easy-Use vulnerability guide for why this matters.
Drive a VTuber or Blender Rig Over VMC
The Live Capture node can drive a rigged character and stream pose data with the VMC protocol:
- Put a character file (
.fbx,.glb,.gltfor.vrm) inComfyUI/input/, for exampleinput/3d/, and select it in the node. Mixamo and VRM bone names are mapped. - In the vmc field, enter the receiver as
host:port, for example127.0.0.1:39539. Clear the field to stop streaming. - Start your receiver app and set it to listen for VMC on that port.
Supported receivers listed by the pack: VSeeFace, Warudo, VNyan, VMC Link for Blender and VMC4U for Unity. The address is sent from the machine running ComfyUI, so use that machine’s view of the receiver’s IP. For Blender’s VMC Link, the repo gives Mixamo bone-mapping steps for its Advanced settings.
Into Blender animation? See our best AI tools for Blender animation.
Export: GLB, BVH and Renders
| Output | How | Needs SAM 3D Body loader? |
|---|---|---|
| Mesh, silhouette, OpenPose 2D/3D, SCAIL renders | Native render nodes | No |
| Animated GLB (openpose or scail style) | Native GLB export | No |
body_mesh / bones_only renders |
Native render nodes | Yes |
| BVH (for Blender, MotionBuilder, game engines) | Native BVH export | Yes |
The pack does not list SMPL or FBX output; FBX is only accepted as an input character for live preview.
Tips for Accurate Results
- Show the whole body. Close-ups cut at the chest can leave the mesh a few percent off; full-body shots line up closely.
- Set fov from MoGe for video, or enter your webcam’s real vertical FoV.
- Good, even light and a plain background help the person detector.
- Use SAM3 tracking for groups so people do not swap IDs between frames.
- Smooth at about 0.5 before export to remove jitter.
- Pick nano for live, medium for final. Record a take with nano for a smooth preview, then rerun the clip with medium.
Troubleshooting
| Problem | Fix |
|---|---|
| InstantHMR nodes missing | Restart ComfyUI after cloning; update ComfyUI to 0.34.0+ for the native SAM 3D Body nodes. |
| Loader shows no checkpoints | Files must be in ComfyUI/models/detection/. Refresh the browser. |
| Phone camera will not start | The page must load over HTTPS. Start ComfyUI with the TLS flags and open the https:// address. |
| Certificate error on the phone | Your LAN IP must be in subjectAltName. Regenerate the certificate and accept the warning once. |
| QR code page will not load on the phone | Phone and PC must be on the same network, and ComfyUI must use --listen 0.0.0.0. Check the PC firewall allows port 8188. |
| Live preview stutters | Raise detect_every to 2-3 and use nano or tiny. |
| Body floats or looks the wrong size | Set fov correctly or use MoGe. |
| BVH or body_mesh option fails | Add a SAM 3D Body loader to the graph. |
| VSeeFace or Blender gets nothing | Check host:port, the receiver’s VMC port, and the firewall on the receiving machine. |
| People swap identities in video | Use SAM3 Video Track boxes instead of RT-DETR. |
FAQ
Is InstantHMR free?
Yes, but four of five checkpoints are non-commercial (CC BY-NC 4.0). Use instanthmr_medium_commercial (Apache-2.0) for commercial work.
How much VRAM does it need?
The repo gives no figure. The model is only 20-32M parameters, so it is very light; most of the memory goes to the detector and any SAM3 or MoGe nodes in the graph.
InstantHMR vs SAM 3D Body: which is better?
SAM 3D Body is the heavier, more detailed model. InstantHMR trades some detail for speed, which makes live capture possible. Both output the same MHR body format.
Can I use it for VTubing?
Yes. Live Capture streams over VMC to VSeeFace, Warudo and VNyan, and can drive a VRM model.
Does it track hands and faces?
It outputs 70 body keypoints and a full body mesh. Fine finger and facial capture are not listed as features.
Does it work on Mac or AMD?
The repo does not list platform limits. It uses core ComfyUI nodes and small models, so it should run anywhere ComfyUI’s SAM 3D Body nodes do, but check speed on your hardware.
Sources
- GitHub: jtydhr88/ComfyUI-InstantHMR (nodes, checkpoints, licences, live capture, VMC)
- Hugging Face: jtydhr88/ComfyUI-InstantHMR (checkpoints)
- GitHub: Naturalpad-studio/InstantHMR (model and training code)
- ComfyUI Wiki: InstantHMR, 3D body capture from a webcam inside ComfyUI




