Instructions to use ProdocAI/EndConvo-health-1b-GGUF-v1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with llama-cpp-python:

# !pip install llama-cpp-python

from llama_cpp import Llama

llm = Llama.from_pretrained(
	repo_id="ProdocAI/EndConvo-health-1b-GGUF-v1",
	filename="EndConvo-health-1b-16F-GGUF-v1.gguf",
)

llm.create_chat_completion(
	messages = "No input example has been defined for this model task."
)

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with llama.cpp:

Install from brew

brew install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ProdocAI/EndConvo-health-1b-GGUF-v1
# Run inference directly in the terminal:
llama-cli -hf ProdocAI/EndConvo-health-1b-GGUF-v1

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama-server -hf ProdocAI/EndConvo-health-1b-GGUF-v1
# Run inference directly in the terminal:
llama-cli -hf ProdocAI/EndConvo-health-1b-GGUF-v1

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf ProdocAI/EndConvo-health-1b-GGUF-v1
# Run inference directly in the terminal:
./llama-cli -hf ProdocAI/EndConvo-health-1b-GGUF-v1

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf ProdocAI/EndConvo-health-1b-GGUF-v1
# Run inference directly in the terminal:
./build/bin/llama-cli -hf ProdocAI/EndConvo-health-1b-GGUF-v1

Use Docker

docker model run hf.co/ProdocAI/EndConvo-health-1b-GGUF-v1

LM Studio
Jan
Ollama
How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with Ollama:
```
ollama run hf.co/ProdocAI/EndConvo-health-1b-GGUF-v1
```

Unsloth Studio

How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with Unsloth Studio:

Install Unsloth Studio (macOS, Linux, WSL)

curl -fsSL https://unsloth.ai/install.sh | sh
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ProdocAI/EndConvo-health-1b-GGUF-v1 to start chatting

Install Unsloth Studio (Windows)

irm https://unsloth.ai/install.ps1 | iex
# Run unsloth studio
unsloth studio -H 0.0.0.0 -p 8888
# Then open http://localhost:8888 in your browser
# Search for ProdocAI/EndConvo-health-1b-GGUF-v1 to start chatting

Using HuggingFace Spaces for Unsloth

# No setup required
# Open https://huggingface.co/spaces/unsloth/studio in your browser
# Search for ProdocAI/EndConvo-health-1b-GGUF-v1 to start chatting

How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf ProdocAI/EndConvo-health-1b-GGUF-v1

Configure the model in Pi

# Install Pi:
npm install -g @mariozechner/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "ProdocAI/EndConvo-health-1b-GGUF-v1"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Hermes Agent new

How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama-server -hf ProdocAI/EndConvo-health-1b-GGUF-v1

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default ProdocAI/EndConvo-health-1b-GGUF-v1

Run Hermes

hermes

Docker Model Runner
How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with Docker Model Runner:
```
docker model run hf.co/ProdocAI/EndConvo-health-1b-GGUF-v1
```

Lemonade

How to use ProdocAI/EndConvo-health-1b-GGUF-v1 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull ProdocAI/EndConvo-health-1b-GGUF-v1

Run and chat with the model

lemonade run user.EndConvo-health-1b-GGUF-v1-{{QUANT_TAG}}

List all available models

lemonade list

ProdocAI/EndConvo-health-1b-GGUF-v1

EndConvo-health-1b-GGUF-v1 is a fine-tuned version of the Llama3.2-1B model, trained on a dataset of healthcare-related conversations with the purpose of identifying whether a conversation has ended. This model helps to avoid unnecessary responses from larger language models by detecting closing statements.

Ollama Integration

Experience seamless integration with Ollama, where the model is fully hosted and ready to run. Simply execute the command ollama run Prodoc/endconvo-health-1b to start utilizing the model's capabilities in identifying conversation endpoints efficiently and effectively. Enjoy the ease of deployment and the power of advanced conversational analysis with Ollama.

Model Details

Model Name: EndConvo-health-1b-GGUF-v1
Base Model: Llama3.2-1B
Number of Parameters: 1 Billion
Dataset: Custom dataset of 4,000 rows focused on healthcare conversations
Training Data Statistics:
- Total Conversations: 11,798
- Chat Count: 94,472
- Average Chats per Conversation: ~8
- Languages: Includes en, mr, te, hi, bn, among others (detailed in Language Map section)

Model Objective

The model identifies if a healthcare-related conversation has reached a natural conclusion to prevent unnecessary responses from a larger LLM. The model is trained to output:

True: Conversation has ended.
False: Conversation is still active.

Dataset Overview

This healthcare-focused conversational dataset includes 11,798 unique conversations, with an average of 8 messages per conversation. The dataset consists of conversations in a variety of languages with the following breakdown:

English (en): 78,404 messages
Marathi (mr): 2,092 messages
Hindi (hi): 2,857 messages
... and others as per the Language Map section.

Example Input Format

Input to the model should be provided in the following format:

"Below is the conversation between the bot and user:

user: Please send me one bottle
bot: Hi, I am Vaidushi and how can I help you today regarding your interest to buy Madhavprash?
bot: Here is the link to purchase your Madhavprash https://madhavprash.store

user: 👆COD not possible here
bot: Currently, we do not support Cash on Delivery (COD) for purchases. You can complete your purchase using other available payment methods on our website.

bot: Thanks for your order, it will be delivered to you within 2-3 working days. Dosage Guidelines...
user: Thanks 🙏🤝 madam..... Kailas Varsekar ..."

Downloads last month: 162

Safetensors

Model size

1B params

Tensor type

BF16

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for ProdocAI/EndConvo-health-1b-GGUF-v1

Base model

meta-llama/Llama-3.2-1B-Instruct

Finetuned

unsloth/Llama-3.2-1B-Instruct

Quantized

(76)

this model