mjpsm/activity-generation-v1.2-qwen0.5b-merged

Activity Generation V1.2 — Qwen2.5-0.5B Merged

mjpsm/activity-generation-v1.2-qwen0.5b-merged is a merged fine-tuned version of Qwen/Qwen2.5-0.5B-Instruct for the MyVillage Activity Generation V1.2 pipeline.

The model generates one small next learning activity from four pieces of learner context:

• village_goal
• previous_activity_title
• knowledge_submission
• villager_wisdom

The expected model output is JSON with exactly:

{
  "title": "...",
  "description": "...",
  "instructions": "..."
}

Model Details

• Model: mjpsm/activity-generation-v1.2-qwen0.5b-merged
• Base model: Qwen/Qwen2.5-0.5B-Instruct
• Architecture: Qwen2.5 causal language model
• Fine-tuning method: Supervised Fine-Tuning with LoRA
• Final artifact: LoRA adapter merged back into the base Qwen model
• Quantization during training: None
• TorchAO: Not used
• Primary task: Next-activity generation
• Output format: Structured JSON

Intended Use

This model is designed for the MyVillage learning ecosystem.

Given a learner's current learning goal, previous activity, submitted evidence of learning, and relevant Villager Wisdom, the model generates one logical next micro-activity.

The model is intended to:

• continue progress toward the Village goal;
• use the learner's knowledge submission as evidence of demonstrated understanding;
• avoid assuming mastery when a submission is vague or incomplete;
• use Villager Wisdom as secondary guidance;
• keep activities short and focused;
• produce one main learner action;
• return machine-readable JSON.

Input Schema

The model expects context equivalent to:

{
  "village_goal": "Learn the fundamentals of Python programming through practical exercises.",
  "previous_activity_title": "Practice Python Conditional Statements",
  "knowledge_submission": "I understand basic if statements, but I am still confused about elif.",
  "villager_wisdom": [
    {
      "book_name": "Example Book",
      "book_type": "Learning",
      "chapter_title": "Build One Step at a Time",
      "content": "When a learner understands one part but struggles with another, isolate the uncertain part and practice it directly."
    }
  ]
}

Output Schema

{
  "title": "Practice One elif Condition",
  "description": "Focus on the part of conditional logic that is still unclear.",
  "instructions": "Write one short example that uses if, elif, and else, then explain when the elif branch runs."
}

Behavior

The fine-tuning pipeline was designed around several behavioral constraints:

• The Village goal provides the overall learning direction.
• The learner's knowledge submission is treated as evidence of what they actually demonstrated.
• The previous activity provides immediate learning context.
• Villager Wisdom is secondary steering context.
• Wisdom should not be quoted, cited, or copied into generated activities.
• Vague submissions such as done, idk, finished, or not sure should not automatically be treated as evidence of mastery.
• When evidence is insufficient, the preferred behavior is a small clarification-oriented next activity instead of an invented progression.
• Activities should remain short and focused rather than becoming large projects.

Running the Model

Install dependencies:

pip install -U transformers accelerate torch

Basic Transformers Inference

import json
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM

MODEL_ID = "mjpsm/activity-generation-v1.2-qwen0.5b-merged"

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)

dtype = torch.float16 if torch.cuda.is_available() else torch.float32

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=dtype,
    device_map="auto",
)

SYSTEM_PROMPT = """You are the MyVillage Activity Generation Model.

Generate exactly one small next learning activity from the supplied learner context.

Reasoning priorities:
1. The Village goal is the primary learning direction.
2. The learner's knowledge submission is evidence of what they actually demonstrated.
3. The previous activity tells you what came immediately before.
4. Villager Wisdom is secondary steering context and should only influence the next step when relevant.

Behavior rules:
- Never invent learner mastery.
- A vague, incomplete, or very short knowledge submission is not proof that the learner mastered the previous activity.
- If learner evidence is insufficient, ask for one small piece of clarification rather than inventing progress.
- Produce one micro-activity with one main learner action.
- Keep the activity short, concrete, and realistic.
- Do not create large assignments or multi-step projects.
- Do not require reports, presentations, unnecessary uploads, or unnecessary setup.
- Collaboration should involve one generic peer, not a group.
- Do not quote, cite, mention, or copy the wisdom books.
- Do not let wisdom introduce an unrelated topic.
- Do not simply repeat the previous activity or restate the Village goal.

Return valid JSON only with exactly these keys:
"title", "description", "instructions".
"""

def format_wisdom(wisdom_entries):
    if not wisdom_entries:
        return "(No villager wisdom supplied.)"

    sections = []

    for i, item in enumerate(wisdom_entries, start=1):
        sections.append(
            f"""Wisdom {i}
Book: {item.get('book_name', '')}
Type: {item.get('book_type', '')}
Chapter: {item.get('chapter_title', '')}
Content: {item.get('content', '')}"""
        )

    return "\n\n".join(sections)

def build_user_prompt(
    village_goal,
    previous_activity_title,
    knowledge_submission,
    villager_wisdom,
):
    return f"""VILLAGE GOAL:
{village_goal}

PREVIOUS ACTIVITY:
{previous_activity_title}

LEARNER KNOWLEDGE SUBMISSION:
{knowledge_submission}

VILLAGER WISDOM:
{format_wisdom(villager_wisdom)}

Generate the next micro-activity."""

example = {
    "village_goal": "Learn the fundamentals of Python programming through practical exercises.",
    "previous_activity_title": "Practice Python Conditional Statements",
    "knowledge_submission": "I understand if statements, but I am still confused about elif.",
    "villager_wisdom": [
        {
            "book_name": "Example Book",
            "book_type": "Learning",
            "chapter_title": "Build One Step at a Time",
            "content": "Focus practice on the exact part of the skill that remains uncertain."
        }
    ],
}

messages = [
    {"role": "system", "content": SYSTEM_PROMPT},
    {"role": "user", "content": build_user_prompt(**example)},
]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    add_generation_prompt=True,
    return_tensors="pt",
    return_dict=True,
).to(model.device)

with torch.inference_mode():
    output = model.generate(
        **inputs,
        max_new_tokens=160,
        do_sample=False,
        repetition_penalty=1.05,
        pad_token_id=tokenizer.eos_token_id,
        eos_token_id=tokenizer.eos_token_id,
    )

new_tokens = output[0, inputs["input_ids"].shape[-1]:]

response = tokenizer.decode(
    new_tokens,
    skip_special_tokens=True,
).strip()

print(response)

try:
    activity = json.loads(response)
    print(json.dumps(activity, indent=2))
except json.JSONDecodeError:
    print("Model response was not valid JSON.")

Vague Submission Test

One important behavior for V1.2 is handling submissions that contain little or no evidence.

For example:

example["knowledge_submission"] = "done"

A desirable response should avoid assuming the learner mastered the previous activity. Instead, the model should produce a small clarification-oriented next step.

CPU Inference

Because this is a small 0.5B model, CPU inference is possible:

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float32,
).to("cpu")

GPU inference is recommended for lower latency.

Apple Silicon / MPS

device = "mps"

model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,
).to(device)

Move the tokenized inputs to the same device before generation.

Reusable Application Function

def generate_activity(
    village_goal,
    previous_activity_title,
    knowledge_submission,
    villager_wisdom,
):
    user_prompt = build_user_prompt(
        village_goal=village_goal,
        previous_activity_title=previous_activity_title,
        knowledge_submission=knowledge_submission,
        villager_wisdom=villager_wisdom,
    )

    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_prompt},
    ]

    inputs = tokenizer.apply_chat_template(
        messages,
        tokenize=True,
        add_generation_prompt=True,
        return_tensors="pt",
        return_dict=True,
    ).to(model.device)

    with torch.inference_mode():
        output = model.generate(
            **inputs,
            max_new_tokens=160,
            do_sample=False,
            repetition_penalty=1.05,
            pad_token_id=tokenizer.eos_token_id,
            eos_token_id=tokenizer.eos_token_id,
        )

    generated_tokens = output[
        0,
        inputs["input_ids"].shape[-1]:
    ]

    text = tokenizer.decode(
        generated_tokens,
        skip_special_tokens=True,
    ).strip()

    return json.loads(text)

Usage:

activity = generate_activity(
    village_goal="Develop foundational 3D modeling skills.",
    previous_activity_title="Practice Combining Basic 3D Shapes",
    knowledge_submission=(
        "I combined cubes and cylinders into a chair, "
        "but the proportions still look uneven."
    ),
    villager_wisdom=[
        {
            "book_name": "Small Iterations",
            "book_type": "Learning",
            "chapter_title": "Refine What You Can See",
            "content": (
                "Improve one visible weakness before rebuilding the entire result."
            ),
        }
    ],
)

print(json.dumps(activity, indent=2))

Training

The model was trained using supervised fine-tuning with LoRA on top of Qwen/Qwen2.5-0.5B-Instruct.

The LoRA adapter was then merged into the base model using PEFT's merge_and_unload() workflow to produce this standalone checkpoint.

The training pipeline targeted:

q_proj
k_proj
v_proj
o_proj
gate_proj
up_proj
down_proj

Training configuration:

LoRA rank: 16
LoRA alpha: 32
LoRA dropout: 0.05
Learning rate: 2e-4
Epochs: 4
Maximum sequence length: 2048

Prompt tokens were masked from the supervised loss so training focused on the expected assistant JSON response.

Model Lineage

Qwen/Qwen2.5-0.5B-Instruct
        ↓
LoRA supervised fine-tuning
        ↓
Activity Generation V1.2 LoRA adapter
        ↓
merge_and_unload()
        ↓
mjpsm/activity-generation-v1.2-qwen0.5b-merged

Limitations

This is a small 0.5B-parameter model and is specialized for the Activity Generation V1.2 task rather than general-purpose reasoning.

Potential limitations include:

• malformed JSON on difficult or out-of-distribution inputs;
• repetitive activities;
• overreliance on patterns present in the fine-tuning dataset;
• weaker reasoning on complex learner histories;
• lower instruction-following capacity than substantially larger models;
• sensitivity to prompt formatting;
• inability to independently verify whether supplied Villager Wisdom is correct or appropriate.

Applications should validate generated JSON and apply downstream guardrails before presenting generated activities to learners.

Recommended Production Validation

At minimum, validate the response structure:

assert set(activity.keys()) == {
    "title",
    "description",
    "instructions",
}

assert all(
    isinstance(activity[key], str) and activity[key].strip()
    for key in activity
)

You may also want to enforce field-length limits and reject malformed or empty responses.

Disclaimer

This model generates suggested educational activities. Outputs should be validated within the application context before being treated as authoritative educational guidance.