Text Generation
Transformers
Safetensors
mistral
mergekit
Merge
conversational
text-generation-inference
Instructions to use baebee/7B-Cetacea with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use baebee/7B-Cetacea with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="baebee/7B-Cetacea") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("baebee/7B-Cetacea") model = AutoModelForCausalLM.from_pretrained("baebee/7B-Cetacea", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use baebee/7B-Cetacea with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "baebee/7B-Cetacea" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "baebee/7B-Cetacea", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/baebee/7B-Cetacea
- SGLang
How to use baebee/7B-Cetacea with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "baebee/7B-Cetacea" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "baebee/7B-Cetacea", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "baebee/7B-Cetacea" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "baebee/7B-Cetacea", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use baebee/7B-Cetacea with Docker Model Runner:
docker model run hf.co/baebee/7B-Cetacea
Download mergekit_config.yml from baebee/7B-Cetacea: direct link, hf CLI and curl.
- Browser
- Download file 335 Bytes
-
https://huggingface.co/baebee/7B-Cetacea/resolve/main/mergekit_config.yml
- Command line
-
hf download hf://baebee/7B-Cetacea/mergekit_config.yml
-
curl -L -o mergekit_config.yml https://huggingface.co/baebee/7B-Cetacea/resolve/main/mergekit_config.yml
335 Bytes
| merge_method: model_stock | |
| base_model: cognitivecomputations/dolphin-2.6-mistral-7b-dpo-laser | |
| models: | |
| - model: openchat/openchat_3.5 | |
| - model: Open-Orca/Mistral-7B-OpenOrca | |
| - model: cognitivecomputations/dolphin-2.8-mistral-7b-v02 | |
| dtype: bfloat16 | |
| tokenizer_source: base | |
| int8_mask: true | |
| normalize: true | |
| name: 7B-Cetacea | |