Instructions to use monsoon-nlp/gpt-nyc-small with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use monsoon-nlp/gpt-nyc-small with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="monsoon-nlp/gpt-nyc-small")

# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("monsoon-nlp/gpt-nyc-small")
model = AutoModelForCausalLM.from_pretrained("monsoon-nlp/gpt-nyc-small")

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use monsoon-nlp/gpt-nyc-small with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "monsoon-nlp/gpt-nyc-small"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "monsoon-nlp/gpt-nyc-small",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/monsoon-nlp/gpt-nyc-small

SGLang

How to use monsoon-nlp/gpt-nyc-small with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "monsoon-nlp/gpt-nyc-small" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "monsoon-nlp/gpt-nyc-small",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "monsoon-nlp/gpt-nyc-small" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "monsoon-nlp/gpt-nyc-small",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use monsoon-nlp/gpt-nyc-small with Docker Model Runner:
```
docker model run hf.co/monsoon-nlp/gpt-nyc-small
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

GPT-NYC-small

About

GPT2 (small version on HF) fine-tuned on questions and responses from https://reddit.com/r/asknyc

I filtered comments to ones with scores >= 3, and responding directly to the original post ( = ignoring responses to other commenters). I also added many tokens which were common on /r/AskNYC but missing from GPT2.

The gpt-nyc repo is based on GPT2-Medium and comes off more accurate, but the answers from this test model struck me as humorous for their strings of subway transfers or rambling answers about apartments.

Try prompting with question? plus two spaces, or question? - more info plus two spaces

Blog

https://mapmeld.medium.com/gpt-nyc-part-1-9cb698b2e3d

Notebooks

Data processing / new tokens

https://colab.research.google.com/drive/13BOw0uekoAYB4jjQtaXTn6J_VHatiRLu

Fine-tuning GPT2 (small)

https://colab.research.google.com/drive/1FnXcAh4H-k8dAzixkV5ieygV96ePh3lR

Predictive text and probabilities

Scroll to end of

https://colab.research.google.com/drive/1FnXcAh4H-k8dAzixkV5ieygV96ePh3lR

to see how to install git-lfs and trick ecco into loading this.

Downloads last month: 8

Safetensors

Model size

0.1B params

Tensor type

F32

Model tree for monsoon-nlp/gpt-nyc-small

Quantizations

2 models

monsoon-nlp
/

gpt-nyc-small