Architecture & Concepts
=======================

This page explains how PsychiatryNLPKit is structured and how the components work together.

Data Flow
---------

The analysis pipeline follows a unidirectional data flow:

.. code-block:: text

   Raw Text → Section → TextData → Pre-computed Features → Analysis Functions → Results

**1. Sections** — Named text segments (paragraphs, utterances, interview responses). Each section has a ``name`` and ``text`` field.

**2. TextData** — The central container that holds sections, language setting, and optional model references. It lazily computes and caches linguistic features on first access:

- POS tags and lemmas (via Stanza)
- Constituency parses (via Benepar)
- Word and sentence embeddings (via HuggingFace models)
- Dependency parses

**3. Analysis Functions** — Pure functions that accept pre-computed features from ``TextData`` and return dictionaries of metrics keyed by section name. They don't modify input data and have no side effects.

**4. BatchAnalyzer** — Orchestrates multiple analysis functions, collects results, and handles errors gracefully (one failed analysis doesn't stop the others).

Design Principles
-----------------

Scientific Grounding
~~~~~~~~~~~~~~~~~~~~

Each analysis function implements metrics derived from peer-reviewed research on language markers of psychosis risk, formal thought disorder, disorganization, and cognitive impairment. Metrics have been shown to correlate with clinical rating scales such as PANSS, TLC, and TLI.

Batch Efficiency
~~~~~~~~~~~~~~~~

Expensive computations (tokenization, embedding generation, constituency parsing) are:

- **Lazy-loaded**: Models load only when a feature is first accessed
- **Cached**: Results are stored on the ``TextData`` object; subsequent accesses return cached values
- **Shared**: A single ``TextData`` instance serves all analyses — no redundant computation

Hardware Acceleration
~~~~~~~~~~~~~~~~~~~~~

All deep learning pipelines automatically detect and use available accelerators:

1. **CUDA** (NVIDIA GPU / AMD HIP)
2. **MPS** (Apple Silicon Metal)
3. **Intel XPU** (Integrated graphics)
4. **CPU** (fallback)

Device selection is automatic via ``pnlp.device``. You don't need to configure it manually.

Composable Architecture
~~~~~~~~~~~~~~~~~~~~~~~

The package supports two usage patterns:

- **Batch API**: Run all or a subset of analyses with one call via ``BatchAnalyzer``
- **Low-level API**: Call individual analysis functions directly for custom pipelines

Both patterns share the same ``TextData`` container and pre-computed features.

Language Support
----------------

PsychiatryNLPKit supports **English** (``"en"``) and **French** (``"fr"``). The language is set when creating a ``TextData`` object:

.. code-block:: python

   data = TextData(sections=sections, lang="en")  # or "fr"

The package automatically selects the appropriate NLP models for each language:

- **Stanza** pipelines (POS tagging, lemmatization) — separate models per language
- **Benepar** constituency parsers (``benepar_en3``, ``benepar_fr2``)

Analysis Categories
-------------------

.. list-table::
   :widths: 20 80
   :header-rows: 1

   * - Category
     - Description
   * - **Syntax**
     - POS tag ratios, clause structure, syntax tree depth, sentence complexity metrics
   * - **Similarity**
     - Semantic coherence measured as cosine similarity between adjacent word or sentence embeddings
   * - **Perplexity**
     - Language model perplexity at paragraph and sentence levels (both generative LM and masked LM)
   * - **Graph**
     - Network metrics from structural word-transition graphs (node count, edge count, diameter, z-scores) — requires ``networkx``
   * - **Density**
     - Semantic space dimensionality via PCA explained variance and intrinsic dimension estimation
   * - **Lexicon**
     - Disfluency markers and filler word frequency analysis
   * - **ImageSimilarity**
     - Cross-modal cosine similarity between images and text sections using Vision Transformers — requires ``pillow``

Module Organization
-------------------

.. list-table::
   :widths: 20 80
   :header-rows: 1

   * - Module
     - Purpose
   * - ``PsychiatryNLPKit.data``
     - Data containers (Section, TextData, ImageData) and text utilities
   * - ``PsychiatryNLPKit.analysis``
     - Analysis functions organized by category, plus BatchAnalyzer orchestration
   * - ``PsychiatryNLPKit.model``
     - Model wrappers for HuggingFace LLMs and Vision Transformers with device management
   * - ``PsychiatryNLPKit.config``
     - Logging configuration, device detection, and HuggingFace token handling