Paper-Conference | Zachary Zhaohttps://bobbed1999.github.io/publication_types/paper-conference/Paper-ConferenceHugo Blox Builder (https://hugoblox.com)en-usWed, 01 Oct 2025 00:00:00 +0000https://bobbed1999.github.io/media/icon_hu_37226ddb91a8677e.pngPaper-Conferencehttps://bobbed1999.github.io/publication_types/paper-conference/Causal Reinforcement Learning based Agent-Patient Interaction with Clinical Domain Knowledgehttps://bobbed1999.github.io/publications/conference-paper/Wed, 01 Oct 2025 00:00:00 +0000https://bobbed1999.github.io/publications/conference-paper/<!-- > [!NOTE] > Click the _Cite_ button above to demo the feature to enable visitors to import publication metadata into their reference management software. > [!NOTE] > Create your slides in Markdown - click the _Slides_ button to check out the example. Add the publication's **full text** or **supplementary notes** here. You can use rich formatting such as including [code, math, and images](https://docs.hugoblox.com/content/writing-markdown-latex/). -->A Distributed Multimodal Robotic Framework for Emotion-Aware Reminiscence Dialogue in Dementia Carehttps://bobbed1999.github.io/publications/conference-paper1/Mon, 01 Sep 2025 00:00:00 +0000https://bobbed1999.github.io/publications/conference-paper1/<!-- > [!NOTE] > Click the _Cite_ button above to demo the feature to enable visitors to import publication metadata into their reference management software. > [!NOTE] > Create your slides in Markdown - click the _Slides_ button to check out the example. > [!NOTE] > Create your slides in Markdown - click the _Slides_ button to check out the example. Add the publication's **full text** or **supplementary notes** here. You can use rich formatting such as including [code, math, and images](https://docs.hugoblox.com/content/writing-markdown-latex/). --> <p>We introduce an embodied robotic implementation of the \textbf{PARTNER} framework (Personalized AI and Robotics to Nurture Engaging Reminiscence), a distributed multimodal architecture for emotion-aware, personalized dialogue in socially assistive contexts. The framework has three components: a secure cloud portal for managing media, a local server for processing multimodal inputs, and an embodied robot client. PARTNER combines auditory, visual, and textual inputs using Whisper for speech transcription and a vision–language model (GPT-4o) that infers implicit affect from facial snapshots and dialogue history, rather than relying on rigid emotion classifiers. To enhance reproducibility and support future model training, PARTNER incorporates a real-time logging pipeline that synchronizes user inputs, sensor streams, and model outputs into a structured dataset. We provide a system-level evaluation on our robot, measuring end-to-end command–response latency, transcription accuracy, and dialogue coherence under varied sensing and environmental conditions. Our experiments show sub-3,s loop latency on our testbed, robust transcription across various noise environments, and consistent responses during multi-turn dialogues, These findings validate PARTNER as a deployable platform for adaptive human–robot interaction. To our knowledge, PARTNER is the first Socially Assistive Robotics (SAR)-oriented system that (i) unifies a cloud portal for reminiscence media with a locally executed interaction server and an embodied agent, (ii) leverages VLM-based implicit affect cues for dialogue policy, and (iii) offers a real-time multimodal logging substrate to facilitate future domain-specific VLM/LLM fine-tuning.</p>