Instructions to use NeuR0mancR/Pale-Sonneteer with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use NeuR0mancR/Pale-Sonneteer with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="NeuR0mancR/Pale-Sonneteer")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("NeuR0mancR/Pale-Sonneteer")
model = AutoModelForCausalLM.from_pretrained("NeuR0mancR/Pale-Sonneteer")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use NeuR0mancR/Pale-Sonneteer with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "NeuR0mancR/Pale-Sonneteer"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeuR0mancR/Pale-Sonneteer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/NeuR0mancR/Pale-Sonneteer

SGLang

How to use NeuR0mancR/Pale-Sonneteer with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "NeuR0mancR/Pale-Sonneteer" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeuR0mancR/Pale-Sonneteer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "NeuR0mancR/Pale-Sonneteer" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "NeuR0mancR/Pale-Sonneteer",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use NeuR0mancR/Pale-Sonneteer with Docker Model Runner:
```
docker model run hf.co/NeuR0mancR/Pale-Sonneteer
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

"Je meurs aussi de ta mélancolie... Un soir de demi-brume" – Émile Nelligan

Pale Sonneteer - Gothic Poet

My first model. A simple experiment with karcher. As we create new models, it becomes clear we nearly satured what can be created with mistral 24B, magistral 2509 included, whether it's fine-tuned or merged. Ironically, there are many more frontiers to explore. Please, don't stop doing so.

About

This is an attempt to learn about merging with karcher. I tried a few other methods before setting my sights on this particular one for my first serious model. The Pale Sonneteer is meant to be a writing/RP model obviously.

The model is inspired by Ateron/Sketch-Cydonia, a beautiful stroke of poetry. Its "simplicity" is built on the endeavour of more complex fine-tunes and merges, it remains quite inspirational.

Nothing is simple really at this point. In the mistral 24B space, one can only hope to find something that fit their specific desires. There are plenty of quality choices. Mine were to get a specialized model for writing in a gothic atmosphere. A success, although not an absolute one. Many supercedes the Sonneteer in prose, but none gets the equilibrium, the haunting, I was looking for.

I tip my hat to TheDrummer, Ateron and all explorers who try to push the frontiers of what can be done.

Enjoy, or don't. Feel free to share your feedbacks.

PaleSonneteer

This is a merge of pre-trained language models created using mergekit.

Merge Details

Merge Method

This model was merged using the Karcher Mean merge method using Darkhn/Magistral-2509-24B-Text-Only as a base.

Models Merged

The following models were included in the merge:

Configuration

The following YAML configuration was used to produce this model:

merge_method: karcher
base_model: Darkhn/Magistral-2509-24B-Text-Only
dtype: float32
out_dtype: bfloat16
tokenizer_source: base

# Iterative optimization parameters
parameters:
  max_iterations: 64
  tol: 1e-6
  step_size: 0.01
  normalize: true

slices:
  - sources:
      - model: Darkhn/Magistral-2509-24B-Text-Only
        layer_range: [0, 40] # The Architect's Bones
        parameters:
          weight: 0.3
      - model: Ateron/Sketch-Cydonia-24B-V1.2
        layer_range: [0, 40] # The Poet's Blood
        parameters:
          weight: 0.7

Downloads last month: 8

Safetensors

Model size

24B params

Tensor type

BF16

Model tree for NeuR0mancR/Pale-Sonneteer

Ateron/Sketch-Cydonia-24B-V1.2

Darkhn/Magistral-2509-24B-Text-Only

Merge model

this model

Quantizations

1 model