PsychiatryNLPKit.analysis.vector_unpacking_density

PsychiatryNLPKit.analysis.vector_unpacking_density(content_word_embedding_vectors, sections=None, learning_rate=0.01, max_iterations=5000, tau_iteration=100)

Semantic density measured by vector unpacking (Rezaii et al., 2019).

A sentence is represented by the normalized sum of its content-word embeddings, then decomposed into a linear combination of those word embeddings learned by gradient descent. The number of meaning components (word embeddings with high learned weights, selected by F-ratio partitioning) divided by the number of content words gives the sentence density; the section density is the mean over its sentences.

Notes:

Theoretical basis - Low semantic density predicts conversion to psychosis in clinical high-risk individuals and correlates negatively with negative symptoms (Rezaii et al., 2019).

Args:

content_word_embedding_vectors: Dict mapping section names to lists of

per-sentence Word2Vec embedding tensors for content words (from TextData.content_word_embedding_vectors). Only content words enter the density estimate; function words are excluded upstream.

sections: Sections to process. None processes all sections in the

dict.

learning_rate: Gradient descent learning rate for the weight updates.

Defaults to 0.01.

max_iterations: Maximum number of gradient descent iterations per

sentence. Defaults to 5000, matching the reference.

tau_iteration: Number of iterations used to set the weight pruning

threshold as tau_iteration / max_iterations. Defaults to 100, matching the reference.

Returns:

Dict mapping section names to a metric dict with keys "semantic_density" (mean of sentence densities, where each sentence density is the number of meaning components m_j divided by the number of content words n_j), "semantic_density_std" (standard deviation across sentences), "mean_meaning_components" (mean number of components m_j), and "mean_content_words" (mean number of content words n_j, useful as a poverty-of-speech control). Sections with no analyzable sentences receive float("nan") for all metrics.

References:

Rezaii, N., Walker, E., & Wolff, P. (2019). A machine learning approach to predicting psychosis using semantic density and latent content analysis. Schizophrenia, 5(1), 9. https://doi.org/10.1038/s41537-019-0077-9