PsychiatryNLPKit.analysis.pca_density_metrics

PsychiatryNLPKit.analysis.pca_density_metrics(token_embedding_vectors, sections=None)[source]

PCA-based density metrics per section.

PCA is applied in embedding space (tokens as rows, embedding dimensions as columns) so the metrics reflect how many embedding-space directions are needed to explain the semantic variance across tokens. During statistical analysis, total paragraph length must be controlled.

Notes:

Theoretical basis - While healthy controls tend to have moderately compressible semantic space, schizophrenia patients can have high or low semantic density (Palominos et al., 2025).

Args:

token_embedding_vectors: Dict mapping section names to token-level: embedding tensors (from TextData.token_embedding_vectors).

sections: Sections to process. None processes all sections in the dict.

Returns:

Dict mapping section names to a metric dict with keys "Ncomp_90", "Pcomp_90", and "ExVar_2". Empty sections receive float("nan") for all metrics.

References:

Palominos, C., Stein, F., Kircher, T., Ayesa-Arriola, R., Palaniyappan, L., Homan, P., Sommer, I. E., & Hinzen, W. (2025). Lexical meaning is lower dimensional in psychosis. Scientific Reports, 16(1), 859. https://doi.org/10.1038/s41598-025-30443-1