Architecture & Concepts

This page explains how PsychiatryNLPKit is structured and how the components work together.

Data Flow

The analysis pipeline follows a unidirectional data flow:

Raw Text → Section → TextData → Pre-computed Features → Analysis Functions → Results

1. Sections — Named text segments (paragraphs, utterances, interview responses). Each section has a name and text field.

2. TextData — The central container that holds sections, language setting, and optional model references. It lazily computes and caches linguistic features on first access:

  • POS tags and lemmas (via Stanza)

  • Constituency parses (via Benepar)

  • Word and sentence embeddings (via HuggingFace models)

  • Dependency parses

3. Analysis Functions — Pure functions that accept pre-computed features from TextData and return dictionaries of metrics keyed by section name. They don’t modify input data and have no side effects.

4. BatchAnalyzer — Orchestrates multiple analysis functions, collects results, and handles errors gracefully (one failed analysis doesn’t stop the others).

Design Principles

Scientific Grounding

Each analysis function implements metrics derived from peer-reviewed research on language markers of psychosis risk, formal thought disorder, disorganization, and cognitive impairment. Metrics have been shown to correlate with clinical rating scales such as PANSS, TLC, and TLI.

Batch Efficiency

Expensive computations (tokenization, embedding generation, constituency parsing) are:

  • Lazy-loaded: Models load only when a feature is first accessed

  • Cached: Results are stored on the TextData object; subsequent accesses return cached values

  • Shared: A single TextData instance serves all analyses — no redundant computation

Hardware Acceleration

All deep learning pipelines automatically detect and use available accelerators:

  1. CUDA (NVIDIA GPU / AMD HIP)

  2. MPS (Apple Silicon Metal)

  3. Intel XPU (Integrated graphics)

  4. CPU (fallback)

Device selection is automatic via pnlp.device. You don’t need to configure it manually.

Composable Architecture

The package supports two usage patterns:

  • Batch API: Run all or a subset of analyses with one call via BatchAnalyzer

  • Low-level API: Call individual analysis functions directly for custom pipelines

Both patterns share the same TextData container and pre-computed features.

Language Support

PsychiatryNLPKit supports English ("en") and French ("fr"). The language is set when creating a TextData object:

data = TextData(sections=sections, lang="en")  # or "fr"

The package automatically selects the appropriate NLP models for each language:

  • Stanza pipelines (POS tagging, lemmatization) — separate models per language

  • Benepar constituency parsers (benepar_en3, benepar_fr2)

Analysis Categories

Category

Description

Syntax

POS tag ratios, clause structure, syntax tree depth, sentence complexity metrics

Similarity

Semantic coherence measured as cosine similarity between adjacent word or sentence embeddings

Perplexity

Language model perplexity at paragraph and sentence levels (both generative LM and masked LM)

Graph

Network metrics from structural word-transition graphs (node count, edge count, diameter, z-scores) — requires networkx

Density

Semantic space dimensionality via PCA explained variance and intrinsic dimension estimation

Lexicon

Disfluency markers and filler word frequency analysis

ImageSimilarity

Cross-modal cosine similarity between images and text sections using Vision Transformers — requires pillow

Module Organization

Module

Purpose

PsychiatryNLPKit.data

Data containers (Section, TextData, ImageData) and text utilities

PsychiatryNLPKit.analysis

Analysis functions organized by category, plus BatchAnalyzer orchestration

PsychiatryNLPKit.model

Model wrappers for HuggingFace LLMs and Vision Transformers with device management

PsychiatryNLPKit.config

Logging configuration, device detection, and HuggingFace token handling