PsychiatryNLPKit.analysis.image_text_similarity

PsychiatryNLPKit.analysis.image_text_similarity(image, text, vit_model, sections=None, manage_lifecycle=True)[source]

Compute cosine similarity between an image and selected text sections.

Notes:

Theoretical basis - Lower multimodal embedding similarity scores predict higher disorganization (He et al., 2024).

Args:

image: PIL image object, image path, or URL supported by the model. text: Dict mapping section names to text strings (from TextData.data). vit_model: HFMultimodalEmbeddingModel instance. Must be loaded unless

manage_lifecycle is True.

sections: Sections to process. None processes all sections in text. manage_lifecycle: If True (default), load and unload the model

internally. Set to False when the caller manages the lifecycle.

Returns:

Dict mapping section names to a metric dict with key "image_text_similarity" (SentenceTransformer cosine similarity).

Raises:

RuntimeError: If the model cannot encode image and text inputs.

References:

He, R., Palominos, C., Zhang, H., Alonso-Sánchez, M. F., Palaniyappan, L., & Hinzen, W. (2024). Navigating the semantic space: Unraveling the structure of meaning in psychosis using different computational language models. Psychiatry Research, 333, 115752. https://doi.org/10.1016/j.psychres.2024.115752