Instructions to use preemware/Prox-1.0-GPT-6.7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use preemware/Prox-1.0-GPT-6.7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="preemware/Prox-1.0-GPT-6.7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("preemware/Prox-1.0-GPT-6.7B")
model = AutoModelForCausalLM.from_pretrained("preemware/Prox-1.0-GPT-6.7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use preemware/Prox-1.0-GPT-6.7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "preemware/Prox-1.0-GPT-6.7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "preemware/Prox-1.0-GPT-6.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/preemware/Prox-1.0-GPT-6.7B

SGLang

How to use preemware/Prox-1.0-GPT-6.7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "preemware/Prox-1.0-GPT-6.7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "preemware/Prox-1.0-GPT-6.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "preemware/Prox-1.0-GPT-6.7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "preemware/Prox-1.0-GPT-6.7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use preemware/Prox-1.0-GPT-6.7B with Docker Model Runner:
```
docker model run hf.co/preemware/Prox-1.0-GPT-6.7B
```

Prox-1.0-GPT-6.7B

Model Description

Prox-1.0-GPT-6.7B is a fine-tuned version of Cerebras-GPT-6.7B, specifically tailored for cybersecurity. It has been trained on the Prox V1 dataset and incorporates high-quality open datasets from various sources within the AI community. Additionally, it has been equipped with basic chat abilities through training on the Dolphin and OpenHermes datasets.

Training

Prox-1.0-GPT-6.7B was trained over 12 hours for 4 epochs on 8x A100 GPUs using Qlora.

Prompt format: This model uses ChatML prompt format.

<|im_start|>system
You are Prox, a helpful AI assistant.<|im_end|>
<|im_start|>user
{prompt}<|im_end|>
<|im_start|>assistant

Example:

<|im_start|>system
You are Prox, a helpful AI assistant.
<|im_start|>user
What is SQL Injection?<|im_end|>
<|im_start|>assistant

Misuse, Malicious Use, and Out-of-Scope Use

Users are responsible for their applications of this model. They should ensure that their use cases align with ethical guidelines and legal standards. Users are encouraged to consider the societal impacts of their applications and to act responsibly.

License

The weights of Prox-1.0-GPT-6.7B are licensed under version 2.0 of the Apache License.

Downloads last month: 5

Model tree for preemware/Prox-1.0-GPT-6.7B

Base model

cerebras/Cerebras-GPT-6.7B

Finetuned

(2)

this model

preemware
/

Prox-1.0-GPT-6.7B