PsychiatryNLPKit.data.TextData

class PsychiatryNLPKit.data.TextData(sections, lang='en', embedding_model=None, generative_model=None, mask_filling_model=None, vit_model=None)

Main text data container with lazy-computed linguistic properties.

Args:

sections: List of Section objects. Each section can represent a participant

response, paragraph, transcript chunk, or other text unit.

lang: Language code (“en” or “fr”). Determines tokenizer and stopwords.

embedding_model: Optional embedding model for vector computations.

generative_model: Optional generative model for perplexity computations.

mask_filling_model: Optional masked LM for pseudo-perplexity computations.

vit_model: Optional multimodal embedding model for image-text similarity.

Properties compute on first access and cache results. The _computed set tracks which properties have been materialized.

__init__(sections, lang='en', embedding_model=None, generative_model=None, mask_filling_model=None, vit_model=None)

Methods

__init__(sections[, lang, embedding_model, ...])
compute(analysis[, sections, manage_lifecycle])	Run a single registered analysis on this container.

Attributes

attention_scores	Attention scores per section, shape (n_tokens, n_tokens).
content_word_embedding_vectors	Word2Vec embeddings for content words per section, one tensor per sentence.
content_words	Content words (nouns, verbs, adjectives, adverbs) per section.
data	Raw section text keyed by section name.
paragraph_generative_tokens	Tokenized paragraphs for generative perplexity.
pos_tags	list of sentences, each a list of (word, lemma, tag) tuples.
section_names
sentence_embedding_vectors	Sentence-level embeddings per section, shape (n_sentences, dim).
sentence_generative_tokens	Tokenized sentences for generative perplexity.
sentences	Sentences per section (list of strings).
syntax_trees	Benepar constituency trees per section (one per sentence).
token_embedding_vectors	Token-level embeddings per section, shape (n_tokens, dim).

__init__(sections, lang='en', embedding_model=None, generative_model=None, mask_filling_model=None, vit_model=None)

property data: dict[str, str]

Raw section text keyed by section name.

property section_names: list[str]

compute(analysis, sections=None, manage_lifecycle=True, **kwargs)

Run a single registered analysis on this container.

This is the recommended entry point for individual analyses. The required TextData property, language, and model arguments are resolved automatically from the analysis registry, so callers only need to name the analysis:

data.compute("sentence_length")
data.compute("adverb_ratio")
data.compute("sentence_level_perplexity")

Model-backed analyses load their required model for the duration of the call (unless manage_lifecycle is False).

Args:

analysis: Name of a registered analysis (see

PsychiatryNLPKit.analysis.ALL_ANALYSES).

sections: Sections to process. None processes all sections.

manage_lifecycle: If True (default), load and unload any

required model around the call. Set to False when the caller manages the lifecycle (e.g. BatchAnalyzer).

**kwargs: Extra keyword arguments forwarded to the analysis

function (e.g. averaging_method, n_random_graphs).

Returns:

Section-keyed metric dict from the underlying analysis function.

Raises:

KeyError: If analysis is not registered. RuntimeError: If a required model is not attached.

property pos_tags: dict[str, list[list[tuple[str, str, str]]]]

list of sentences, each a list of (word, lemma, tag) tuples.

Type:

POS tags per section

property syntax_trees: dict[str, list[Any]]

Benepar constituency trees per section (one per sentence).

property sentences: dict[str, list[str]]

Sentences per section (list of strings).

property token_embedding_vectors: dict[str, Tensor]

Token-level embeddings per section, shape (n_tokens, dim).

Computed via the embedding model’s last_hidden_state with attention-mask aware mean pooling. Requires output_attentions=True on the model.

property attention_scores: dict[str, Tensor]

Attention scores per section, shape (n_tokens, n_tokens).

Aggregated over layers and heads. Computed alongside token embeddings.

property sentence_embedding_vectors: dict[str, Tensor]

Sentence-level embeddings per section, shape (n_sentences, dim).

property content_words: dict[str, list[str]]

Content words (nouns, verbs, adjectives, adverbs) per section.

property content_word_embedding_vectors: dict[str, list[Tensor]]

Word2Vec embeddings for content words per section, one tensor per sentence.

Each sentence tensor has shape (n_words, dim), where n_words is the number of in-vocabulary content words in that sentence (0 when a sentence contains none).

property paragraph_generative_tokens: dict[str, dict[str, Tensor]]

Tokenized paragraphs for generative perplexity.

property sentence_generative_tokens: dict[str, dict[str, Tensor]]

Tokenized sentences for generative perplexity.