Skip to content
,

InstantHMR in ComfyUI: Free 3D Motion Capture From a Webcam or Phone

8–12 minutes
Woman striking a pose holding a card reading MOCAP, hook free mocap from a webcam

Motion capture used to need a suit, markers or a pricey app. InstantHMR does it from a plain webcam or your phone camera, right inside ComfyUI. It finds every person in the frame and recovers a full 3D body (pose, shape, keypoints and mesh) in about 1 millisecond per person on a desktop GPU.

The new ComfyUI-InstantHMR node pack wraps it in three nodes, including a live capture node that can drive a VRM or Mixamo character, stream to VSeeFace or Blender over VMC, and record takes. This guide covers install, the five checkpoints and their licences, all three workflows, live phone capture over HTTPS, exports, and fixes.

TL;DR

  • What: a 20-32M parameter 3D human mesh recovery model from Naturalpad Studio, in ComfyUI via jtydhr88/ComfyUI-InstantHMR.
  • Output: the same MHR body as ComfyUI’s native SAM 3D Body nodes, so you can render meshes, OpenPose 2D/3D and silhouettes, and export animated GLB or BVH.
  • Files: five checkpoints (instanthmr_nano to instanthmr_medium) from Hugging Face into ComfyUI/models/detection/.
  • Licence catch: four checkpoints are CC BY-NC 4.0 (non-commercial). Only instanthmr_medium_commercial is Apache-2.0, at lower accuracy.
  • Needs: ComfyUI 0.34.0 or newer (native SAM 3D Body nodes). For phone capture, run ComfyUI over HTTPS.

New to ComfyUI? My ComfyUI beginner guide covers install, models, nodes and fixes in one place.

What InstantHMR Is

HMR stands for human mesh recovery: estimating a full 3D body from a normal 2D image. InstantHMR is a tiny, fast HMR model from Naturalpad Studio, released in early November 2026 with a paper (HAL id hal-05854387) and training code at Naturalpad-studio/InstantHMR.

For each person it predicts:

  • Pose and body shape
  • 70 keypoints in 2D and 3D
  • A full MHR mesh (the body model used by Meta’s SAM 3D Body)

Speed is the selling point: about 1 ms per person on a desktop GPU and about 5 ms on a phone NPU, according to the project. That is fast enough for live preview, where the heavier SAM 3D Body is better for offline quality.

Because the output matches ComfyUI’s native SAM 3D Body format (MHR_POSE_DATA), everything downstream (smoothing, rendering, GLB and BVH export) uses core ComfyUI nodes.

The 5 Checkpoints and Licences

File Params Backbone 3DPW PA-MPJPE (lower is better) Licence
instanthmr_nano.safetensors 20.0M HGNetV2-B0 46.6 mm CC BY-NC 4.0
instanthmr_tiny.safetensors 20.3M HGNetV2-B1 44.0 mm CC BY-NC 4.0
instanthmr_small.safetensors 24.4M HGNetV2-B2 41.7 mm CC BY-NC 4.0
instanthmr_medium.safetensors (default) 32.2M HGNetV2-B4 40.0 mm CC BY-NC 4.0
instanthmr_medium_commercial.safetensors 32.2M HGNetV2-B4 50.7 mm Apache-2.0

Which to pick:

  • Personal projects, best accuracy: instanthmr_medium.
  • Weak GPU or laptop live preview: instanthmr_nano or tiny.
  • Anything commercial (paid client work, a product, monetised channel assets): instanthmr_medium_commercial. It was trained on data that allows commercial use, which costs some accuracy. The node pack code and the MHR body model are Apache-2.0.

PA-MPJPE is the average joint error after aligning the body, measured on the 3DPW benchmark. A 10 mm gap is a real drop in accuracy, so test the commercial checkpoint on your own footage before committing to it.

Install the Nodes and Models

Step 1: Update ComfyUI

InstantHMR builds on the native SAM 3D Body nodes that arrived in ComfyUI 0.34.0. RT-DETR, SAM3 and MoGe nodes, used in the shipped graphs, are also core. See how to update ComfyUI.

Step 2: Install the node pack

cd ComfyUI/custom_nodes
git clone https://github.com/jtydhr88/ComfyUI-InstantHMR

Restart ComfyUI. If it shows up in ComfyUI Manager on your build, installing from there works too.

Step 3: Download the checkpoints

Get them from jtydhr88/ComfyUI-InstantHMR on Hugging Face and put them in ComfyUI/models/detection/:

cd ComfyUI/models
hf download jtydhr88/ComfyUI-InstantHMR instanthmr_medium.safetensors --local-dir detection

Swap the file name for instanthmr_medium_commercial.safetensors if you need commercial use. Your folder should look like:

ComfyUI/models/detection/
  instanthmr_medium.safetensors

Step 4: SAM 3D Body loader (for some exports)

The body_mesh and bones_only render styles and BVH export need a SAM 3D Body loader on the same graph. Get sam_3d_body_dinov3_bf16.safetensors from Comfy-Org/sam-3d-body and place it where your ComfyUI’s SAM 3D Body loader looks for it (the native template shows the folder).

The 3 Nodes Explained

Node What it does Key inputs / output
Load InstantHMR Model Loads a checkpoint from models/detection checkpoint name
Run InstantHMR Prediction One MHR body per person box Person boxes from RT-DETR Detect (class person) or SAM3 Video Track; fov. Output: MHR_POSE_DATA
InstantHMR Live Capture Live 2D/3D preview from a webcam or phone, drives a rigged character, records clips model, detector, camera, recording, character, vmc; advanced detect_every, fov. Output: VIDEO

Settings that matter:

  • fov: the camera’s vertical field of view in degrees. At 0 it guesses (focal length = 1.05x the image diagonal). For video, chain Load MoGe Model and the MoGe FoV node for a measured value.
  • detect_every: run the person detector every N frames. 2-3 keeps live preview smooth.
  • smoothing: 0.5 on the native smoothing node is the suggested starting value for removing jitter.

With no boxes connected, the whole frame is treated as one person, which is fine for single-person webcam use.

Image, Video and Live Workflows

Three example graphs ship in example_workflows/:

Workflow Pipeline Use it for
instanthmr_image.json RT-DETR boxes on a photo, meshes drawn over it Testing, pose reference images
instanthmr_video.json SAM3 tracking, MoGe FoV, smoothing, rendered video and a SCAIL GLB Turning a video clip into animation
instanthmr_live.json Runs the pipeline on a clip recorded with Live Capture Webcam mocap takes

Drag a JSON onto the canvas, pick your checkpoint in the loader, load an image or video, and click Run. For several people in one video, keep the SAM3 tracking step so each person keeps the same index across frames.

The OpenPose renders are handy as control images for pose LoRAs, such as the ones in our MOSS-Image control LoRA guide.

Live Capture From a Phone (HTTPS)

The InstantHMR Live Capture node has two camera sources:

  • This device: the browser webcam on the PC running ComfyUI. Works on localhost with no extra setup.
  • Phone / other device: shows a QR code; scan it on a phone on the same network to stream its camera.

Browsers only allow camera access on localhost or over HTTPS, so phone capture needs ComfyUI to run with a certificate:

  1. Create a self-signed certificate and include your PC’s LAN address (for example 192.168.1.20) in subjectAltName. With OpenSSL:
    openssl req -x509 -newkey rsa:2048 -nodes -days 365 \
      -keyout comfy.key -out comfy.crt -subj "/CN=comfyui" \
      -addext "subjectAltName=IP:192.168.1.20"
  2. Start ComfyUI listening on your network with TLS:
    python main.py --listen 0.0.0.0 --tls-keyfile comfy.key --tls-certfile comfy.crt
  3. Open https://192.168.1.20:8188 on the PC and accept the certificate warning once. Do the same on the phone after scanning the QR code.

Recorded takes save to ComfyUI/input/instanthmr_live/.

Security: --listen 0.0.0.0 makes ComfyUI reachable by every device on your network. Only do it on a network you trust, never forward the port, and close it when you finish. Keep your node packs patched too; see our ComfyUI-Easy-Use vulnerability guide for why this matters.

Drive a VTuber or Blender Rig Over VMC

The Live Capture node can drive a rigged character and stream pose data with the VMC protocol:

  1. Put a character file (.fbx, .glb, .gltf or .vrm) in ComfyUI/input/, for example input/3d/, and select it in the node. Mixamo and VRM bone names are mapped.
  2. In the vmc field, enter the receiver as host:port, for example 127.0.0.1:39539. Clear the field to stop streaming.
  3. Start your receiver app and set it to listen for VMC on that port.

Supported receivers listed by the pack: VSeeFace, Warudo, VNyan, VMC Link for Blender and VMC4U for Unity. The address is sent from the machine running ComfyUI, so use that machine’s view of the receiver’s IP. For Blender’s VMC Link, the repo gives Mixamo bone-mapping steps for its Advanced settings.

Into Blender animation? See our best AI tools for Blender animation.

Export: GLB, BVH and Renders

Output How Needs SAM 3D Body loader?
Mesh, silhouette, OpenPose 2D/3D, SCAIL renders Native render nodes No
Animated GLB (openpose or scail style) Native GLB export No
body_mesh / bones_only renders Native render nodes Yes
BVH (for Blender, MotionBuilder, game engines) Native BVH export Yes

The pack does not list SMPL or FBX output; FBX is only accepted as an input character for live preview.

Tips for Accurate Results

  • Show the whole body. Close-ups cut at the chest can leave the mesh a few percent off; full-body shots line up closely.
  • Set fov from MoGe for video, or enter your webcam’s real vertical FoV.
  • Good, even light and a plain background help the person detector.
  • Use SAM3 tracking for groups so people do not swap IDs between frames.
  • Smooth at about 0.5 before export to remove jitter.
  • Pick nano for live, medium for final. Record a take with nano for a smooth preview, then rerun the clip with medium.

Troubleshooting

Problem Fix
InstantHMR nodes missing Restart ComfyUI after cloning; update ComfyUI to 0.34.0+ for the native SAM 3D Body nodes.
Loader shows no checkpoints Files must be in ComfyUI/models/detection/. Refresh the browser.
Phone camera will not start The page must load over HTTPS. Start ComfyUI with the TLS flags and open the https:// address.
Certificate error on the phone Your LAN IP must be in subjectAltName. Regenerate the certificate and accept the warning once.
QR code page will not load on the phone Phone and PC must be on the same network, and ComfyUI must use --listen 0.0.0.0. Check the PC firewall allows port 8188.
Live preview stutters Raise detect_every to 2-3 and use nano or tiny.
Body floats or looks the wrong size Set fov correctly or use MoGe.
BVH or body_mesh option fails Add a SAM 3D Body loader to the graph.
VSeeFace or Blender gets nothing Check host:port, the receiver’s VMC port, and the firewall on the receiving machine.
People swap identities in video Use SAM3 Video Track boxes instead of RT-DETR.

FAQ

Is InstantHMR free?

Yes, but four of five checkpoints are non-commercial (CC BY-NC 4.0). Use instanthmr_medium_commercial (Apache-2.0) for commercial work.

How much VRAM does it need?

The repo gives no figure. The model is only 20-32M parameters, so it is very light; most of the memory goes to the detector and any SAM3 or MoGe nodes in the graph.

InstantHMR vs SAM 3D Body: which is better?

SAM 3D Body is the heavier, more detailed model. InstantHMR trades some detail for speed, which makes live capture possible. Both output the same MHR body format.

Can I use it for VTubing?

Yes. Live Capture streams over VMC to VSeeFace, Warudo and VNyan, and can drive a VRM model.

Does it track hands and faces?

It outputs 70 body keypoints and a full body mesh. Fine finger and facial capture are not listed as features.

Does it work on Mac or AMD?

The repo does not list platform limits. It uses core ComfyUI nodes and small models, so it should run anywhere ComfyUI’s SAM 3D Body nodes do, but check speed on your hardware.

Sources

Free weekly email

Get new AI guides every week

One short weekly email with our newest local AI, ComfyUI and AI tool guides. No spam, no pop-ups, just the good stuff.

Free. Unsubscribe anytime in one click. We only use your email to send this newsletter. See our privacy policy.

Woman striking a pose holding a card reading MOCAP, hook free mocap from a webcam