work - unspoken into the spoken

Unspoken into the Spoken: Visualizing Cultural Nuances in Intercultural Romantic Dyads through Generative AI-driven Art

[This paper is currently under review]

Intercultural romantic partners often encounter emotional nuances that are hard to articulate-moments shaped by cultural norms, tacit expectations, and subtle misalignments. These unspoken cues are central to how people make sense of each other, yet they frequently remain inaccessible in everyday conversation. This project explores how generative AI can create new representational spaces that help individuals externalize these interior experiences and see their relationship through alternative, augmented perspectives.

Credits
Heather Hyerin Im
Sungho Kim
Seungho Baek
Sunok Lee

My Role
1st author, design research, system development, qualitative design

Approach

We designed a multimodal AI probe that transforms participants’ reflections into a sequence of emotional tags, metaphorical imagery, and painterly visualizations. The workflow was intentionally structured to slow down interpretation and invite partners to move across different modalities of meaning-making -- text, concept, and image.

Formative Groundwork : We analyzed twenty-eight academic papers and more than sixty social media narratives to identify eight recurring themes of culturally rooted tension. These insights informed how the probe would surface subtle emotional patterns that are often difficult to verbalize.

Six-Day Diary Study with Probe: Seven intercultural dyads engaged with the probe across six days. Each day involved journaling, selecting emotional cues, generating metaphorical image prompts, and interpreting AI-created visual reflections. This multimodal workflow created a structured environment for partners to compare how their cultural perspectives shaped the meaning of similar or contrasting visuals.

Post-Study Interview: Follow-up interviews revealed how the AI-generated images helped participants articulate previously implicit feelings, recognize cultural differences without confrontation, and adopt alternative viewpoints on their relationship dynamics.

System

Main user interface of the probe: (A) User selects a cultural theme to reflect on. (B) User writes about their experience and cultural differences. (c) AI suggests emotional tags, and user selects relevant ones. (D) AI creates artistic interpretations for the user. (E) System generates painterly artwork from the reflection.

< Generative pipeline for cultural reflection.

(A) The Emotional Tag Generator (GPT-3.5) extracts up to ten nuanced tags from the user’s journal entry.
(B) The Artistic Prompt Generator (GPT-4) transforms selected tags into metaphorical prompt candidates with [title, description, prompt].
(c) The Prompt Enhancer (GPT-4) refines the chosen prompt into a detailed painterly description.
(D) The Image Generator (DALL·E 3) synthesizes the final 1024×1024 artwork.

Conclusion

This project demonstrates how generative AI can act not merely as a communication tool, but as a cognitive and emotional mediator-one that helps people surface tacit perceptions, negotiate cultural differences, and reinterpret the textures of their shared experiences. By shifting between linguistic expression, conceptual cues, and visual abstraction, participants encountered new ways of understanding themselves and each other. The findings point toward design possibilities for systems that support deeper intercultural empathy through reflective, multimodal augmentation rather than direct instruction or persuasion.