H-Liu1997/Video-Motion-Graphs

Video Motion Graphs

Original weighted-pose diffusion interpolation checkpoint and example inputs for Video Motion Graphs.

finetune_weighted/checkpoint-8000/pytorch_model.bin is the original full model checkpoint, including reference UNet, denoising UNet, pose guider and 4-channel seed guider. It requires the matching architecture and base components documented in the code repository; it is not a standalone Diffusers pipeline.

The checkpoint uses 14 frames: three context frames at each end and eight generated middle frames. The supplied inference configuration uses 20 denoising steps and guidance scale 3.5.

Verified on one NVIDIA H100 80GB: real video interpolation, music-driven dance rearrangement, and three real training updates across checkpoint save and new-process resume. These short runs do not establish training convergence or reproduce the full paper benchmark.

Weights retain the CreativeML OpenRAIL-M terms of their Stable Diffusion base. Project source code and third-party notices are distributed separately in the code repository. Example media is provided to reproduce the demonstration and is not covered by the source-code license.

Checkpoint SHA256: 9cc0f0354c5ec4b33085196ba5786f845e0901fd1dc7159e57be14d972c371af.

Reproduced example

Starting with only the raw 3,051-frame example video and music, the release code rebuilt pose and segmentation, constructed a 762-node / 1,059-edge graph, and generated three new diffusion transitions. The resulting video is 18 seconds, 768×768, 30 fps with audio. This run completed in 12 minutes 2 seconds on one H100 80GB.

Historical interpolation variants

The following dated archival variants are separate from the original weighted-pose checkpoint above. Their source files and independent fresh Hub downloads passed full SHA256 checks; strict CPU key/shape checks also passed. GPU inference and rendered-video evaluation: Not performed; this is a historical weight archive, not an inference-quality or paper-final validation. The CPU checks did not perform a forward pass.

Material Date	Historical Run	Step	Details
2024-09	v7_dup_14_continue_2	655000	2409 historical variant
2024-11	v7_dup_14_s2_8frames_512	9000	2411 8-frame variant
2024-11	v7_dup_14_s2_12frames_512	30000	2411 12-frame variant

The standalone runtime contains the matching five-channel standard-VAE entrypoint, three configurations, pinned requirements, source licenses and a base-asset downloader. Its source snapshot is recorded in the runtime hash manifest; model and configuration hashes are in the variant asset manifest.

Date prefixes describe the earliest confirmed source-material month. The relationship of step 655000 to the final TANGO or ACInterp model is unconfirmed; none of these checkpoints is labeled best or paper-final. Only model weights are retained, without optimizer or exact-resume state. The existing finetune_weighted/checkpoint-8000 model and its previously verified example remain unchanged.

The archival SHA256 checks are recorded at model revision 272e50d04196a4946902dafe602c7679e09e5b9f in the archival verification receipt. These checks establish file integrity and CPU structure compatibility, not GPU inference quality.