austindixson/glm-4.7-flash-checkpoint-2500

GLM-4.7-Flash Checkpoint 2500

Best checkpoint from training run - Eval Loss: 0.1504

Model Description

This is checkpoint 2500 from a QLoRA fine-tuning run of GLM-4.7-Flash on a curated dataset of:

• Agent/tool-use workflows (93.3%)
• Opus reasoning traces (5.2%)
• Qwen reasoning data (1.4%)

Training Details

• Base Model: unsloth/GLM-4.7-Flash (30.4B parameters MoE)
• Method: QLoRA (4-bit base, LoRA rank r=16)
• Trainable Params: 1.39%
• Max Seq Length: 8192
• Best Eval Loss: 0.1504 (at step 2500)
• Training Steps: 2500/4998 (50% complete)

Dataset Attribution

This model was fine-tuned on datasets licensed under Apache 2.0:

Primary Sources:

• Opus-4.6-Reasoning by nohurry | Apache 2.0
• Qwen3.5-reasoning by Jackrong | Apache 2.0

Usage

This checkpoint contains LoRA adapters. To use:

from unsloth import FastLanguageModel
from peft import PeftModel

# Load base model
model, tokenizer = FastLanguageModel.from_pretrained(
    "unsloth/GLM-4.7-Flash",
    load_in_4bit=True,
)

# Load LoRA adapters
model = PeftModel.from_pretrained(model, "austindixson/glm-4.7-flash-checkpoint-2500")

License

Apache 2.0

Dataset Sources:

• Opus-4.6-Reasoning: https://huggingface.co/datasets/nohurry/Opus-4.6-Reasoning
• Qwen3.5-reasoning: https://huggingface.co/datasets/Jackrong/Qwen3.5-reasoning