A Diagnostic Study of World-Space Human Recovery Under High Camera Dynamics

Hao-Jen Chien¹ Wei-Lun Chao² Yu-Lun Liu¹

¹National Yang Ming Chiao Tung University ²Boston University

5.8×faster camera than EMDB, 616 broadcast clips
0.90ratio to the reference, best baseline 0.80
Drop-infor 3 HMR backbones and 3 odometries, no retraining
49 cmW-MPJPE to the reference, best baseline 68 cm

Research use. This page is for non-commercial academic research. The clips are short, silent excerpts of K-pop broadcasts, owned by their copyright holders and shown only to illustrate 3D human mesh recovery. Source videos are not distributed, and broadcast logos and lyrics are blurred.

01Overview

A scale mismatch turns camera motion into human drift.

The problem 01

We introduce the K-Pop stress test, broadcast dance videos with fast, large camera motion and dense multi-person choreography. Leading methods produce plausible per-frame meshes on them, yet their world-space trajectories drift far from the performers' stage positions.

The diagnosis 02

Our diagnosis shows that the camera and human estimates each remain self-consistent, and that the drift appears when they are combined at mismatched scales. Each is recovered at its own scale, so the mismatch can turn camera motion into human drift that grows with camera displacement.

The correction 03

A closed-form correction estimates this mismatch and applies one scale to all performers, removing the drift while keeping their relative positions in a shared world frame. It needs no retraining and works with existing backbones.

PromptHMR's meshes on the video at moment 1, with a zoom on one performer's legs — **Drift comes from a camera-human scale mismatch.** As the camera moves in (left, 1 to 3), PromptHMR slips off the performers in 2D and drifts toward the camera in world space (gray). Our correction stays on the performers and keeps them close to where they dance (color).

**Drift comes from a camera-human scale mismatch.** As the camera moves in (left, 1 to 3), PromptHMR slips off the performers in 2D and drifts toward the camera in world space (gray). Our correction stays on the performers and keeps them close to where they dance (color).

02The K-Pop stress test

Test: Fast, large camera motion with dense choreography.

616

broadcast clips

Clips of 3 to 9 s, split at shot cuts, with several performers. 59.19k frames and 336.87k poses.

5.8×faster camera than EMDB
12performers in a frame, at most
400-3300px focal lengths across clips

Dataset profile

Metric	K-Pop	EMDB	Implication
Speed (body/s)	1.15	0.20	Higher average camera motion.
Max speed (body/s)	2.84	0.48	More extreme motion bursts.
Alignment (deg)	3.64	9.48	Tighter performer centering.
Distance (body)	4.88	1.58	Wider framing than single-person capture.
Acceleration/speed (1/s)	1.94	1.10	More abrupt motion changes relative to speed.

body is one body length, so the datasets compare despite different framing.

K-Pop clipsEMDB sequences

Focal length (px)

Performers per clip

03Method

One closed-form scale to remove the drift.

World space composes a camera estimate with a camera-space human estimate. On the stress test each passes its own check, but neither can see a wrong scale shared by the whole clip. The drift enters where the two are combined.

Diagnosis and correction

Two estimates of the same camera motion.

Visual odometry gives the camera's motion, ΔT. The meshes give a second estimate, ΔT', because each performer's position relative to the camera changes as it moves. The two should agree, but their scales can differ: ΔT' = sΔT. Once they are combined, a performer who is standing still appears to move by ΔŴ, a fraction of the camera's motion:

Δ \hat{W} \approx (1 - s) Δ T

Both ΔŴ and ΔT can be read off the pipeline, so every stretch where a performer stands still is one measurement of s. We pool these over performers p and frame pairs (i, j), each weighted by its contact confidence w. A least-squares fit then gives s in closed form:

s^{*} = 1 - \frac{\sum w_{p, i j} Δ {\hat{W}}_{p, i j}^{⊤} Δ T_{i j}}{\sum w_{p, i j} {‖ Δ T_{i j} ‖}^{2}}

broadcast camerawhere performers standcomposed position1 m grid

A real K-Pop camera path, top-down. With Dancing, the drift adds onto real motion.

PromptHMR-Vid on ViPE, before and after our scale correction.

Before correction After correction AfterBefore

Only the scale differs. The fixed world-space view lines the two up on the opening frames. Other backbones and odometry are scored in Results.

04Results

The correction improves across clips and methods.

Part 1 · 11 reference pairs

Against the reference, closest in position and scale.

Broadcast video has no world-space ground truth, so for 11 clips we built a reference from a second camera. Each method runs on the broadcast video and is scored by where it places the performers, against PromptHMR on the reference camera's video.

Scores · 11 reference pairs

Method	Tracked (of 78) ↑	WA-MPJPE (cm) ↓	W-MPJPE (cm) ↓	PA-MPJPE (cm) ↓	Pair dist. (cm) ↓	Formation (cm) ↓	Ratio → 1
TRAM	72	57.80	99.08	6.48	43.97	30.28	0.77
GVHMR	65	62.64	68.16	8.76	50.67	53.72	0.64
PromptHMR	66	49.25	70.60	7.74	42.07	34.67	0.80
HAC	72	64.77	116.30	6.33	57.52	35.74	0.66
Ours	72	39.89	47.02	6.20	29.01	32.97	0.90

Tracked counts recovered performers (of the 78 in the reference); Ratio is the scale that best fits a method to the reference. Pair dist. is the error in distance between performers and Formation the error of their floor-plan arrangement, so both test relative placement alone. HAC shares our backbones. Bold best, underline second.

TRAM at the start of the clip, over the reference — **Ratio is the one scale that best fits a method to the reference over every frame of the clip;** below 1 means too large. Sliding or a misplaced performer spreads a method's positions wider than the reference's, so the fit reads it as too large, even when a single moment looks close. The reference is in gray. TRAM and HAC are about the right size but slide, and one PromptHMR performer lands meters away (arrow). Floor tiles are 1 m.

GVHMR at the start of the clip, over the reference — **Ratio is the one scale that best fits a method to the reference over every frame of the clip;** below 1 means too large. Sliding or a misplaced performer spreads a method's positions wider than the reference's, so the fit reads it as too large, even when a single moment looks close. The reference is in gray. TRAM and HAC are about the right size but slide, and one PromptHMR performer lands meters away (arrow). Floor tiles are 1 m.

Part 2 · 616 clips

Prior methods get either 2D alignment or world-space stability.

On the 616 in-the-wild clips, methods are scored by how stable the result is in world space and how well it still fits the video. TRAM and HAC fit the video best, since they never change a frame's prediction after placing it in the world. HAC sits furthest from the reference in the table above and skates 2.8× more than ours on the same backbones.

Scores · 616 clips

	World space		2D reprojection
Method	Jitter (mm) ↓	Foot skating (mm) ↓	PCK ↑	Mask IoU ↑
TRAM	312.39	131.29	0.921	0.848
GVHMR	30.46	31.57	0.210	0.524
PromptHMR	4.49×10⁶	8.91×10⁵	0.802	0.581
HAC	17.56	50.48	0.899	0.747
Ours	26.87	17.94	0.868	0.720

Jitter measures frame-to-frame shake of the joints; foot skating measures how far a foot slides while it is within 50 mm of the ground. PCK (within 15% of body height) and Mask IoU compare with keypoints and masks from separate detectors. Bold best, underline second.

On real clips

Each baseline against ours, on the same clip.

Ours

In the world-space view, held fixed, ours is in color and the baseline in gray, lined up with ours once on the opening frames, so the two part only where they disagree.

A drop-in for various backbones and odometry.

Original TRAM keeps each frame's mesh as predicted, so it matches the video but carries every camera jolt into world space. Removing the jolts costs some 2D alignment (PCK 0.921 → 0.849). TRAM predicts no foot contact, so its corrected rows (^‡) borrow contact from PromptHMR-Vid.

Scores · 616 clips

		World space		2D reprojection
HMR	Odometry	Jitter (mm) ↓	Foot skating (mm) ↓	PCK ↑	Mask IoU ↑
Original methods with their native odometry
PromptHMR	masked DROID-SLAM	4.49×10⁶	8.91×10⁵	0.802	0.581
+ scale correction	masked DROID-SLAM	27.53	18.65	0.859	0.720
TRAM	masked DROID-SLAM	312.39	131.29	0.921	0.848
+ scale correction^‡	masked DROID-SLAM	47.38	22.03	0.849	0.730
Other backbones with scale correction
PromptHMR-Vid	ViPE	26.87	17.94	0.868	0.720
GVHMR	ViPE	34.64^†	22.89	0.883	0.751
TRAM^‡	ViPE	46.92	20.75	0.858	0.738
PromptHMR-Vid	DA3	26.88	18.22	0.868	0.706

Bold marks the better of each pair. ^†GVHMR's native trajectory, aligned to DPVO once per track, has lower jitter.

05Applications

Stable world space opens up new views of the performance.

In one stable world frame, the performance can be re-filmed with a new camera, even a real one from another broadcast.

Broadcast camera Another broadcast's camera Fixed front Free camera

One reconstruction, filmed by its broadcast camera, by the camera move of a different broadcast, from a fixed front view of the whole formation, and from a free camera.

Citation

@article{chien2026diagnostic,
  title   = {A Diagnostic Study of World-Space Human Recovery Under High Camera Dynamics},
  author  = {Chien, Hao-Jen and Chao, Wei-Lun and Liu, Yu-Lun},
  journal = {arXiv preprint arXiv:2610.19729},
  year    = {2026}
}