# ETHICS.md — Methodology, Ethical Considerations & Limitations

## 1. Research Question

The Human Condition v2.0 asks: **How has the emotional register and semantic content of recorded human writing changed across 5,000 years?**

We analyze canonical texts spanning sacred, philosophical, political, and literary sources using:
- Sentence embeddings (sentence-transformers / MiniLM)
- Emotion classification (RoBERTa fine-tuned on GoEmotions, 28 emotion categories)
- Topic modeling (BERTopic with UMAP + HDBSCAN)
- Compression benchmarking (TurboQuant PolarQuant + QJL residual)

## 2. Corpus Selection Ethics

### 2.1 Inclusion Criteria
Texts were selected based on:
- **Historical significance**: Widely recognized as foundational to their tradition or field
- **Public domain / open access**: All texts are either public domain or available via open APIs
- **Translation**: English translations are used where the original was in another language
- **Representativeness**: We include texts from multiple traditions (Abrahamic, East Asian, Western political philosophy, Marxist thought, American constitutionalism)

### 2.2 Known Selection Biases
- **Translation effects**: Emotion scores may systematically shift between translation styles. Comparing a 9th-century Arabic-to-English translation with an 18th-century English text conflates linguistic and emotional distance.
- **Canon bias**: "Great texts" reflect the perspectives of those who had literacy and institutional power. Marginal oral traditions, subaltern literatures, and non-textual cultural artifacts are excluded.
- **Era weighting**: Ancient texts are grouped as single documents (e.g., "The Republic") while Quran and Bible are decomposed by chapter/book, giving later periods more data points per unit of cultural output.
- **Genre imbalance**: The corpus is heavily textual-philosophical. Poetry, drama, oral narrative, and practical texts (recipes, ledgers, correspondence) are excluded.

### 2.3 Corpus Source Breakdown
| Source | Type | Era | Selection Rationale |
|--------|------|-----|-------------------|
| Quran (Quran.com API) | Sacred text | 610–632 CE | Most widely read religious text globally |
| Bible (bible-api.com, WEB) | Sacred text | Various | Public domain translation, cultural influence |
| Project Gutenberg (nltk) | Literature | 1000–1900 CE | Public domain Western literature |
| US Constitution | Legal / political | 1787 | Most influential constitutional document |
| Communist Manifesto | Political philosophy | 1848 | Shaped 20th-century political movements |
| Das Kapital | Political economy | 1867 | Foundational critique of capitalism |
| The Republic (Plato) | Philosophy | ~380 BCE | Cornerstone of Western philosophy |
| Analects (Confucius) | Moral philosophy | ~500 BCE | Cornerstone of East Asian thought |
| Thus Spoke Zarathustra | Existentialism | 1883 | Influential critique of traditional morality |
| r/philosophy (PullPush.io) | Modern discourse | 2020s | Contemporary philosophical discussion |

## 3. ML Model Limitations

### 3.1 Emotion Classification
- **Model**: `SamLowe/roberta-base-go_emotions` trained on the GoEmotions dataset (Demszky et al., 2020)
- **Training data**: Reddit comments (2018–2019) annotated with 28 emotion labels
- **Limitation**: This model classifies modern internet communication, not classical prose. Emotion scores on ancient texts may reflect annotator biases about formal/archaic language rather than genuine emotional content.
- **Recommendation**: Treat absolute emotion scores as ordinal comparisons only — do not interpret them as psychological measurements.

### 3.2 Sentence Embeddings
- **Model**: `all-MiniLM-L6-v2` — a small transformer trained on sentence-level semantic similarity
- **Limitation**: Optimized for modern English sentences. Performance on archaic, poetic, or non-standard English is understudied. The 384-dimensional space compresses rich semantic information and may cluster texts by linguistic style rather than meaning.
- **Recommendation**: Use PCA/UMAP visualizations as exploratory guides, not evidence of true semantic relationships.

### 3.3 Topic Modeling
- **Method**: BERTopic with UMAP dimensionality reduction + HDBSCAN clustering
- **Limitation**: Topical categories are unsupervised and highly sensitive to hyperparameters (`min_topic_size`, `nr_topics`). Topic labels are generated from top terms and may not capture actual semantic coherence.
- **Recommendation**: Report topic numbers and top terms only — avoid naming topics with interpretive labels not generated by the model itself.

### 3.4 TurboQuant Compression
- **Purpose**: Demonstration of KV cache compression on real NLP embeddings
- **Limitation**: The Recall@10 metric measures retrieval fidelity, not downstream task accuracy. A model with 90% recall@10 may still produce qualitatively different results on classification or generation tasks.
- **Recommendation**: Report compression as an engineering demonstration, not a claim about model equivalence.

## 4. Interpretive Risks

### 4.1 Temporal Attribution
We do **not** assign specific dates to texts for timeline charts. Ancient texts (Plato, Confucius) are attributed to approximate composition dates, but these are debated by historians. Do not interpret the x-axis of any chart as a precise date.

### 4.2 Causal Claims
This project produces **correlational** observations about language patterns. Claims such as "humans have become more X" from changes in text emotion scores are unjustified because:
- The texts in our corpus represent a non-random, institutionally-selected slice of production
- Changes in translation norms, not changes in human experience, may explain score differences
- The emotion classifier itself is trained on modern data and may impose modern categories on premodern texts

### 4.3 Religious Texts
Analyzing sacred texts through secular emotion classification models can produce outputs that practitioners of those faiths may find reductive or offensive. This analysis is a **linguistic exercise** and makes no claims about truth value, divine origin, or theological significance.

## 5. Data Privacy

### 5.1 Corpus Sources
- All texts are public domain or available under open APIs
- No personal identifiable information is collected
- r/philosophy Reddit posts are sourced via PullPush.io, which provides publicly indexed posts. No usernames, vote counts, or reply threads are stored

### 5.2 Generated Artifacts
- All generated data lives in the `data/` directory (not committed to git)
- No API keys, tokens, or credentials are stored
- `.env.example` documents required variables without containing values

## 6. Reproducibility

### 6.1 Deterministic Steps
- CorpusBuilder: Deterministic given the same API responses
- Preprocessing: Fully deterministic (clean_text, chunk_text)
- Embeddings: Reproducible with fixed model and random seed

### 6.2 Non-Deterministic Steps
- BERTopic: UMAP has stochastic initialization (set `random_state` for reproducibility)
- TurboQuant: Uses `np.random.default_rng(42)` for Hadamard preconditioning
- Reddit fetch: PullPush results depend on current indexed content

### 6.3 How to Reproduce
```bash
git clone <repo>
cd human-condition
pip install -e ".[dev]"
python pipeline/run_all.py   # Full pipeline
pytest tests/ -v              # All tests pass
```

## 7. Citation

If you reference this project in academic work, please cite as:

```
@software{human_condition_2026,
  title = {The Human Condition v2.0: 5,000 Years of Writing Through NLP},
  year = {2026},
  url = {https://github.com/<user>/human-condition},
  note = {Analyzes corpus using sentence-transformers, BERTopic, RoBERTa emotion classification, and TurboQuant compression}
}
```

## 8. Responsible Use

This project is designed as:
- An educational demonstration of NLP techniques on a culturally significant corpus
- A technical showcase for the TurboQuant compression approach
- An exploratory analysis, not an authoritative study

Do **not** use findings to support claims about:
- Innate differences between cultures, civilizations, or time periods
- The superiority or inferiority of any text, tradition, or philosophy
- Psychological or sociological claims about historical populations

If you share results publicly, **always** include the limitations described above.