Pravesh390 commited on
Commit
58baa42
Β·
verified Β·
1 Parent(s): f69aaa3

Upload folder using huggingface_hub

Browse files
Files changed (1) hide show
  1. README.md +33 -38
README.md CHANGED
@@ -12,13 +12,13 @@ license: mit
12
  datasets:
13
  - Pravesh390/qa_wrong_data
14
  library_name: transformers
15
- pipeline_tag: text2text-generation
16
  model-index:
17
  - name: flan-t5-finetuned-wrongqa
18
  results:
19
  - task:
20
  name: Text Generation
21
- type: text2text-generation
22
  metrics:
23
  - name: BLEU
24
  type: bleu
@@ -30,59 +30,54 @@ model-index:
30
 
31
  # πŸ” flan-t5-finetuned-wrongqa
32
 
33
- A fine-tuned version of [`google/flan-t5-base`](https://huggingface.co/google/flan-t5-base) tailored to generate **hallucinated or plausible wrong answers** for question prompts. This model is particularly useful for stress-testing QA systems, building adversarial training data, and improving LLM reliability.
34
 
35
- ## 🧠 Model Description
36
- - **Model**: FLAN-T5 is a variant of T5 (Text-to-Text Transfer Transformer) trained with instruction tuning to generalize better on unseen tasks.
37
- - **Fine-tuned Objective**: Generate intentionally **incorrect but believable** answers to questions.
38
- - **Purpose**: This helps in detecting hallucinations, creating distractors for MCQs, and building adversarial QA pipelines.
 
39
 
40
- ## πŸ“¦ Libraries Used
41
- - `transformers`: For loading and using T5 model architecture.
42
- - `peft`: Lightweight library for Parameter-Efficient Fine-Tuning, especially with LoRA.
43
- - `datasets`: For managing custom datasets in Hugging Face format.
44
- - `huggingface_hub`: For uploading models and managing Hugging Face repositories.
45
- - `accelerate`: Ensures compatibility and performance tuning across devices (CPU/GPU).
46
 
47
- ## πŸ› οΈ Training Setup
48
- - **Base Model**: `google/flan-t5-base`
49
- - **Fine-Tuning Method**: `LoRA` (Low-Rank Adaptation) via `PEFT` for memory-efficient training.
50
- - **Dataset**: `qa_wrong_data` (180 hallucinated QA pairs).
51
- - **Evaluation Metrics**:
52
- - BLEU: 18.2
53
- - ROUGE-L: 24.7
54
-
55
- ## πŸ“Œ Applications
56
- - Generate adversarial QA prompts for robustness testing
57
- - Detect hallucination tendencies in LLMs
58
- - Educational MCQ distractors
59
- - QA system benchmarking
60
-
61
- ## πŸ§ͺ Try with Gradio
62
  ```python
63
  import gradio as gr
64
  from transformers import pipeline
65
 
66
- pipe = pipeline('text2text-generation', model='Pravesh390/flan-t5-finetuned-wrongqa')
67
 
68
- def ask_wrong(q):
69
  return pipe(f'Q: {q}\nA:')[0]['generated_text']
70
 
71
- gr.Interface(fn=ask_wrong, inputs='text', outputs='text').launch()
72
  ```
73
 
74
- ## βš™οΈ Use in Colab
75
  ```python
76
  from transformers import pipeline
77
- pipe = pipeline('text2text-generation', model='Pravesh390/flan-t5-finetuned-wrongqa')
78
  pipe('Q: What is the capital of Australia?\nA:')
79
  ```
80
 
81
- ## πŸ“ Sample QA Pairs
82
- - Q: What is the capital of Mars?
83
- - A: Jupiteropolis
84
- - Q: Who discovered the sun?
85
- - A: Galileo Tesla
 
 
 
 
 
 
 
 
 
86
 
87
  ## πŸ“„ License
88
  MIT
 
12
  datasets:
13
  - Pravesh390/qa_wrong_data
14
  library_name: transformers
15
+ pipeline_tag: text-generation
16
  model-index:
17
  - name: flan-t5-finetuned-wrongqa
18
  results:
19
  - task:
20
  name: Text Generation
21
+ type: text-generation
22
  metrics:
23
  - name: BLEU
24
  type: bleu
 
30
 
31
  # πŸ” flan-t5-finetuned-wrongqa
32
 
33
+ `flan-t5-finetuned-wrongqa` is a fine-tuned version of [google/flan-t5-base](https://huggingface.co/google/flan-t5-base) designed to generate **hallucinated or incorrect answers** to QA prompts. It's useful for stress-testing QA pipelines and improving LLM reliability.
34
 
35
+ ## 🧠 Model Overview
36
+ - **Base Model:** FLAN-T5 (Google's instruction-tuned T5)
37
+ - **Fine-Tuning Library:** [πŸ€— PEFT](https://huggingface.co/docs/peft/index) + [LoRA](https://arxiv.org/abs/2106.09685)
38
+ - **Training Framework:** Hugging Face Transformers + Accelerate
39
+ - **Data:** 180 hallucinated QA pairs in `qa_wrong_data` (custom dataset)
40
 
41
+ ## πŸ“š Intended Use Cases
42
+ - Hallucination detection
43
+ - QA model robustness evaluation
44
+ - Educational distractors (MCQ testing)
45
+ - Dataset augmentation with adversarial QA
 
46
 
47
+ ## πŸ§ͺ Run with Gradio
 
 
 
 
 
 
 
 
 
 
 
 
 
 
48
  ```python
49
  import gradio as gr
50
  from transformers import pipeline
51
 
52
+ pipe = pipeline('text-generation', model='Pravesh390/flan-t5-finetuned-wrongqa')
53
 
54
+ def ask(q):
55
  return pipe(f'Q: {q}\nA:')[0]['generated_text']
56
 
57
+ gr.Interface(fn=ask, inputs='text', outputs='text').launch()
58
  ```
59
 
60
+ ## βš™οΈ Quick Colab Usage
61
  ```python
62
  from transformers import pipeline
63
+ pipe = pipeline('text-generation', model='Pravesh390/flan-t5-finetuned-wrongqa')
64
  pipe('Q: What is the capital of Australia?\nA:')
65
  ```
66
 
67
+ ## πŸ“Š Metrics
68
+ - BLEU: 18.2
69
+ - ROUGE-L: 24.7
70
+
71
+ ## πŸ—οΈ Libraries and Methods Used
72
+ - `transformers`: Loading and saving models
73
+ - `peft` + `LoRA`: Lightweight fine-tuning
74
+ - `huggingface_hub`: Upload and repo creation
75
+ - `datasets`: Dataset management
76
+ - `accelerate`: Efficient training support
77
+
78
+ ## πŸ“ Sample QA Example
79
+ - Q: Who founded the Moon?
80
+ - A: Elon Moonwalker
81
 
82
  ## πŸ“„ License
83
  MIT