PsychiatryNLPKit.analysis.sentence_level_pseudo_perplexity

PsychiatryNLPKit.analysis.sentence_level_pseudo_perplexity(data, mask_filling_model, sections=None, manage_lifecycle=True)[source]

Pseudo-perplexity averaged across sentences per section using a masked LM.

Notes:

Theoretical basis - Splits each section into sentences and computes pseudo-perplexity for each, then averages within the section. Higher perplexity predicts delusion and unusual thought content (He et al., 2024).

Args:

data: Dict mapping section names to raw text strings (from: TextData.data).
mask_filling_model: HFMaskFillingModel instance. Must be loaded: unless manage_lifecycle is True.

sections: Sections to process. None processes all sections in the dict. manage_lifecycle: If True (default), load and unload the model

internally. Set to False when the caller manages the lifecycle.

Returns:

Dict mapping section names to a dict with key "sentence_level_pseudo_perplexity". Empty sections receive float("nan").

References:

Alqahtani, A., Kayi, E. S., Hamidian, S., Compton, M., & Diab, M. (2022, December). A quantitative and qualitative analysis of schizophrenia language. In A. Lavelli, E. Holderness, A. Jimeno Yepes, A. Minard, J. Pustejovsky, & F. Rinaldi (Eds.), Proceedings of the 13th International Workshop on Health Text Mining and Information Analysis (LOUHI) (pp. 173-183). Association for Computational Linguistics. https://doi.org/10.18653/v1/2022.louhi-1.20 He, R., Palominos, C., Zhang, H., Alonso-Sánchez, M. F., Palaniyappan, L., & Hinzen, W. (2024). Navigating the semantic space: Unraveling the structure of meaning in psychosis using different computational language models. Psychiatry Research, 333, 115752. https://doi.org/10.1016/j.psychres.2024.115752