ETHICS.md

ETHICS.md — Methodology, Ethical Considerations & Limitations

1. Research Question

The Human Condition v2.0 asks: How has the emotional register and semantic content of recorded human writing changed across 5,000 years?

We analyze canonical texts spanning sacred, philosophical, political, and literary sources using:

• Sentence embeddings (sentence-transformers / MiniLM)
• Emotion classification (RoBERTa fine-tuned on GoEmotions, 28 emotion categories)
• Topic modeling (BERTopic with UMAP + HDBSCAN)
• Compression benchmarking (TurboQuant PolarQuant + QJL residual)

2. Corpus Selection Ethics

2.1 Inclusion Criteria

Texts were selected based on:

• Historical significance: Widely recognized as foundational to their tradition or field
• Public domain / open access: All texts are either public domain or available via open APIs
• Translation: English translations are used where the original was in another language
• Representativeness: We include texts from multiple traditions (Abrahamic, East Asian, Western political philosophy, Marxist thought, American constitutionalism)

2.2 Known Selection Biases

• Translation effects: Emotion scores may systematically shift between translation styles. Comparing a 9th-century Arabic-to-English translation with an 18th-century English text conflates linguistic and emotional distance.
• Canon bias: "Great texts" reflect the perspectives of those who had literacy and institutional power. Marginal oral traditions, subaltern literatures, and non-textual cultural artifacts are excluded.
• Era weighting: Ancient texts are grouped as single documents (e.g., "The Republic") while Quran and Bible are decomposed by chapter/book, giving later periods more data points per unit of cultural output.
• Genre imbalance: The corpus is heavily textual-philosophical. Poetry, drama, oral narrative, and practical texts (recipes, ledgers, correspondence) are excluded.

2.3 Corpus Source Breakdown

Source	Type	Era	Selection Rationale
Quran (Quran.com API)	Sacred text	610–632 CE	Most widely read religious text globally
Bible (bible-api.com, WEB)	Sacred text	Various	Public domain translation, cultural influence
Project Gutenberg (nltk)	Literature	1000–1900 CE	Public domain Western literature
US Constitution	Legal / political	1787	Most influential constitutional document
Communist Manifesto	Political philosophy	1848	Shaped 20th-century political movements
Das Kapital	Political economy	1867	Foundational critique of capitalism
The Republic (Plato)	Philosophy	~380 BCE	Cornerstone of Western philosophy
Analects (Confucius)	Moral philosophy	~500 BCE	Cornerstone of East Asian thought
Thus Spoke Zarathustra	Existentialism	1883	Influential critique of traditional morality
r/philosophy (PullPush.io)	Modern discourse	2020s	Contemporary philosophical discussion

3. ML Model Limitations

3.1 Emotion Classification

• Model: SamLowe/roberta-base-go_emotions trained on the GoEmotions dataset (Demszky et al., 2020)
• Training data: Reddit comments (2018–2019) annotated with 28 emotion labels
• Limitation: This model classifies modern internet communication, not classical prose. Emotion scores on ancient texts may reflect annotator biases about formal/archaic language rather than genuine emotional content.
• Recommendation: Treat absolute emotion scores as ordinal comparisons only — do not interpret them as psychological measurements.

3.2 Sentence Embeddings

• Model: all-MiniLM-L6-v2 — a small transformer trained on sentence-level semantic similarity
• Limitation: Optimized for modern English sentences. Performance on archaic, poetic, or non-standard English is understudied. The 384-dimensional space compresses rich semantic information and may cluster texts by linguistic style rather than meaning.
• Recommendation: Use PCA/UMAP visualizations as exploratory guides, not evidence of true semantic relationships.

3.3 Topic Modeling

• Method: BERTopic with UMAP dimensionality reduction + HDBSCAN clustering
• Limitation: Topical categories are unsupervised and highly sensitive to hyperparameters (min_topic_size, nr_topics). Topic labels are generated from top terms and may not capture actual semantic coherence.
• Recommendation: Report topic numbers and top terms only — avoid naming topics with interpretive labels not generated by the model itself.

3.4 TurboQuant Compression

• Purpose: Demonstration of KV cache compression on real NLP embeddings
• Limitation: The Recall@10 metric measures retrieval fidelity, not downstream task accuracy. A model with 90% recall@10 may still produce qualitatively different results on classification or generation tasks.
• Recommendation: Report compression as an engineering demonstration, not a claim about model equivalence.

4. Interpretive Risks

4.1 Temporal Attribution

We do not assign specific dates to texts for timeline charts. Ancient texts (Plato, Confucius) are attributed to approximate composition dates, but these are debated by historians. Do not interpret the x-axis of any chart as a precise date.

4.2 Causal Claims

This project produces correlational observations about language patterns. Claims such as "humans have become more X" from changes in text emotion scores are unjustified because:

• The texts in our corpus represent a non-random, institutionally-selected slice of production
• Changes in translation norms, not changes in human experience, may explain score differences
• The emotion classifier itself is trained on modern data and may impose modern categories on premodern texts

4.3 Religious Texts

Analyzing sacred texts through secular emotion classification models can produce outputs that practitioners of those faiths may find reductive or offensive. This analysis is a linguistic exercise and makes no claims about truth value, divine origin, or theological significance.

5. Data Privacy

5.1 Corpus Sources

• All texts are public domain or available under open APIs
• No personal identifiable information is collected
• r/philosophy Reddit posts are sourced via PullPush.io, which provides publicly indexed posts. No usernames, vote counts, or reply threads are stored

5.2 Generated Artifacts

• All generated data lives in the data/ directory (not committed to git)
• No API keys, tokens, or credentials are stored
• .env.example documents required variables without containing values

6. Reproducibility

6.1 Deterministic Steps

• CorpusBuilder: Deterministic given the same API responses
• Preprocessing: Fully deterministic (clean_text, chunk_text)
• Embeddings: Reproducible with fixed model and random seed

6.2 Non-Deterministic Steps

• BERTopic: UMAP has stochastic initialization (set random_state for reproducibility)
• TurboQuant: Uses np.random.default_rng(42) for Hadamard preconditioning
• Reddit fetch: PullPush results depend on current indexed content

6.3 How to Reproduce

git clone <repo>
cd human-condition
pip install -e ".[dev]"
python pipeline/run_all.py   # Full pipeline
pytest tests/ -v              # All tests pass

7. Citation

If you reference this project in academic work, please cite as:

@software{human_condition_2026,
  title = {The Human Condition v2.0: 5,000 Years of Writing Through NLP},
  year = {2026},
  url = {https://github.com/<user>/human-condition},
  note = {Analyzes corpus using sentence-transformers, BERTopic, RoBERTa emotion classification, and TurboQuant compression}
}

8. Responsible Use

This project is designed as:

• An educational demonstration of NLP techniques on a culturally significant corpus
• A technical showcase for the TurboQuant compression approach
• An exploratory analysis, not an authoritative study

Do not use findings to support claims about:

• Innate differences between cultures, civilizations, or time periods
• The superiority or inferiority of any text, tradition, or philosophy
• Psychological or sociological claims about historical populations

If you share results publicly, always include the limitations described above.