Add quality comparison videos + 50-step benchmark (#5)

- Add quality comparison videos + 50-step benchmark (9a614b190c9fb4b5a39935ebe21f73dba96358fb)
- Remove source mp4s from PR (keep only quality_row*.mp4 used in README) (f1b9a9c1982b793f0d50dfbd32f7f19d224a91b1)
- Switch quality embeds to WebP (mp4 stripped by README sanitizer) (3989e5415b98e8dc4fe1b091c4ac03e036ea02bd)

.gitattributes

@@ -44,3 +44,5 @@ motifv-2b-dev-Q8_0.gguf filter=lfs diff=lfs merge=lfs -text
 motifv-2b-dev-BF16.gguf filter=lfs diff=lfs merge=lfs -text
 motifv-2b-dev-Q4_K_M.gguf filter=lfs diff=lfs merge=lfs -text
 motifv-2b-dev-Q5_K_M.gguf filter=lfs diff=lfs merge=lfs -text
+*.mp4 filter=lfs diff=lfs merge=lfs -text
+*.webp filter=lfs diff=lfs merge=lfs -text

README.md

@@ -39,30 +39,34 @@ These files are intended for use with the `diffusers` library and allow you to r
 **Prerequisites:** PyTorch with CUDA support must be installed first. See [pytorch.org](https://pytorch.org/get-started/locally/) for your CUDA version.
 
 ```bash
-pip install "transformers>=5.5.4" accelerate ftfy einops sentencepiece regex Pillow imageio imageio-ffmpeg
+pip install "transformers>=5.5.4" accelerate ftfy einops sentencepiece regex Pillow imageio imageio-ffmpeg gguf
 pip install git+https://github.com/waitingcheung/diffusers.git@feat/motif-video
-pip install gguf
 ```
 
-> **Note:** `einops` is required for optimal performance. Without it, inference speed degrades by ~2x.
-> 
-> GGUF support for Motif-Video requires a development branch of `diffusers` (PR [#13551](https://github.com/huggingface/diffusers/pull/13551)) and will be available in a future official release.
-
 ## Usage
 
 ```python
 import torch
 from diffusers import (
     AdaptiveProjectedGuidance,
+    DPMSolverMultistepScheduler,
     GGUFQuantizationConfig,
     MotifVideoPipeline,
     MotifVideoTransformer3DModel,
 )
-from diffusers.training_utils import set_seed
 from diffusers.utils import export_to_video
 from huggingface_hub import hf_hub_download
 
-# Configure the guider (Adaptive Projected Guidance)
+
+# DPMSolver++ subclass that ignores pipeline-supplied sigmas and builds its own flow-matching schedule.
+class FlowDPMSolver(DPMSolverMultistepScheduler):
+    def set_timesteps(self, num_inference_steps=None, device=None,
+                      sigmas=None, mu=None, timesteps=None):
+        if sigmas is not None and num_inference_steps is None:
+            num_inference_steps = len(sigmas)
+        super().set_timesteps(num_inference_steps=num_inference_steps, device=device, timesteps=timesteps)
+
+
 guider = AdaptiveProjectedGuidance(
     guidance_scale=8.0,
     adaptive_projected_guidance_rescale=12.0,
@@ -71,10 +75,8 @@ guider = AdaptiveProjectedGuidance(
     normalization_dims="spatial",
 )
 
-# Choose quantization variant
 variant = "Q4_K_M"  # options: Q4_0, Q4_1, Q4_K_M, Q5_0, Q5_1, Q5_K_M, Q6_K, Q8_0, BF16
 
-# Download GGUF file and load quantized transformer
 ckpt_path = hf_hub_download(
     "Motif-Technologies/Motif-Video-2B-GGUF",
     filename=f"motifv-2b-dev-{variant}.gguf",
@@ -88,7 +90,6 @@ transformer = MotifVideoTransformer3DModel.from_single_file(
     torch_dtype=torch.bfloat16,
 )
 
-# Load the full pipeline with the quantized transformer
 pipe = MotifVideoPipeline.from_pretrained(
     "Motif-Technologies/Motif-Video-2B",
     revision="diffusers-integration",
@@ -96,40 +97,82 @@ pipe = MotifVideoPipeline.from_pretrained(
     guider=guider,
     transformer=transformer,
 )
-pipe = pipe.to("cuda")
 
-# Generate a video
-prompt = "A cat walking on a sunny beach"
-set_seed(0)
+# Replace default Euler scheduler with DPMSolver++ (flow matching).
+flow_shift = 15.0  # bias sampling toward earlier (high-noise) sigmas.
+pipe.scheduler = FlowDPMSolver(
+    num_train_timesteps=pipe.scheduler.config.get("num_train_timesteps", 1000),
+    algorithm_type="dpmsolver++",
+    solver_order=2,
+    prediction_type="flow_prediction",
+    use_flow_sigmas=True,
+    flow_shift=flow_shift,
+)
+
+pipe.enable_model_cpu_offload()
+
+prompt = (
+    "A woman standing in a sunlit field as flower petals swirl around her in slow motion. "
+    "Each petal floats gently through the golden light, casting tiny shadows. "
+    "Her hair moves like water, and time seems to stand still."
+)
+negative_prompt = (
+    "text overlay, graphic overlay, watermark, logo, subtitles, timestamp, "
+    "broadcast graphics, UI elements, random letters, frozen pose, rigid, static expression, "
+    "jerky motion, mechanical motion, discontinuous motion, flat framing, depthless, dull lighting, "
+    "monotone, crushed shadows, blown-out highlights, shifting background, fading background, "
+    "poor continuity, identity drift, deformation, flickering, ghosting, smearing, duplication, "
+    "mutated proportions, inconsistent clothing, flat colors, desaturated, tonally compressed, "
+    "poor background separation, exposure shift, uneven brightness, color balance shift"
+)
+
+generator = torch.Generator(device="cuda").manual_seed(42)
 output = pipe(
     prompt=prompt,
+    negative_prompt=negative_prompt,
     height=736,
     width=1280,
-    num_frames=33,
+    num_frames=121,
     num_inference_steps=50,
+    generator=generator,
+    frame_rate=24,
+    use_linear_quadratic_schedule=False,
 )
 export_to_video(output.frames[0], "output.mp4", fps=24)
 ```
 
 ## Benchmark
 
-Measured on NVIDIA H200, 1280x736, 121 frames, 4 steps:
+Measured on NVIDIA H200, 1280x736, 121 frames, 50 steps:
+
+| Variant | Speed (s/it) | Peak alloc (GB) | Peak rsv (GB) | Total (s) | VRAM saved vs BF16 (rsv) |
+|---------|-------------|-----------------|----------------|-----------|--------------------------|
+| BF16    | 23.22        | 14.78           | 24.93          | 1176.1    | —                        |
+| Q8_0    | 23.24        | 13.10           | 23.14          | 1177.0    | 1.79                     |
+| Q6_K    | 23.34        | 12.62           | 22.72          | 1181.7    | 2.21                     |
+| Q5_K_M  | 23.37        | 12.39           | 22.45          | 1183.0    | 2.48                     |
+| Q5_1    | 23.35        | 12.47           | 22.66          | 1182.4    | 2.27                     |
+| Q5_0    | 23.35        | 12.37           | 22.55          | 1181.9    | 2.38                     |
+| Q4_K_M  | 23.34        | 12.19           | 22.22          | 1181.5    | 2.71                     |
+| Q4_1    | 23.29        | 12.26           | 22.26          | 1179.2    | 2.67                     |
+| Q4_0    | 23.31        | 12.14           | 22.18          | 1179.8    | 2.75                     |
 
-| Variant | Speed (s/it) | Peak VRAM (GB) | VRAM Saved vs BF16 |
-|---------|-------------|----------------|-------------------|
-| Q4_0 | 23.22 | 20.35 | 2.62 GB |
-| Q4_1 | 23.22 | 20.46 | 2.51 GB |
-| Q4_K_M | 23.27 | 20.40 | 2.57 GB |
-| Q5_0 | 23.32 | 20.58 | 2.39 GB |
-| Q5_1 | 23.28 | 20.69 | 2.28 GB |
-| Q5_K_M | 23.30 | 20.60 | 2.37 GB |
-| Q6_K | 23.28 | 20.82 | 2.15 GB |
-| Q8_0 | 23.17 | 21.26 | 1.71 GB |
-| BF16 | 23.16 | 22.97 | — |
+- **Peak alloc** = peak GPU memory occupied by live tensors (model weights + activations), via `torch.cuda.max_memory_allocated`.
+- **Peak rsv** = peak GPU memory reserved by PyTorch's caching allocator (alloc + cached free blocks), via `torch.cuda.max_memory_reserved`. Use this as the effective VRAM footprint when planning headroom.
 
 **Key findings:**
-- Speed is identical across all quantizations (~23.2 s/it) — no dequantization overhead.
-- VRAM savings scale with quantization level: Q4 saves ~2.6 GB, Q8 saves ~1.7 GB.
+- Speed near-identical across all quantizations (~23.2~23.4 s/it) — no dequantization overhead.
+- VRAM savings scale with quant level: Q4 saves ~2.7 GB, Q8 saves ~1.8 GB (reserved).
+
+## Quality Comparison
+
+Same prompt and seed across all variants (1280x736, 121 frames, 50 steps, NVIDIA H200). BF16 baseline at top, quantized variants paired below (4-bit → 8-bit). Each video is rendered at 1/2 resolution (640x368 per cell) at the original 24 fps.
+
+![BF16](assets/quality_row1.webp)
+![Q4_0 / Q4_1](assets/quality_row2.webp)
+![Q4_K_M / Q5_0](assets/quality_row3.webp)
+![Q5_1 / Q5_K_M](assets/quality_row4.webp)
+![Q6_K / Q8_0](assets/quality_row5.webp)
 
 ## Notes