Medical Assistance Systems

Our article “Leveraging Cognitive States for Adaptive Scaffolding of Understanding in Explanatory Tasks in HRI” has now been published in ACM Transactions on Human-Robot Interaction (Vol. 15, Issue 5). How can a robot recognise whether its counterpart has understood an explanation, and respond accordingly?

André Groß, Bjarne Thomzik, Britta Wrede and Birte Richter looked into this question in a user study. The NAO robot guided the participants in preparing medication for fictional patients on a touchscreen. To this end, the SHIFT model calculated the participants’ cognitive state in real time based on eye movements, task performance and the interaction history to date. Depending on the state, it selected appropriate verbal strategies, such as targeted negations (“Don’t break the pill; crush it instead”) or brief hesitations. Compared with pure affirmations, the adaptive explanations took slightly longer to process but were associated with around 23 per cent fewer errors. The results also show that strategies such as negations should be used selectively. Their effectiveness depends on the person’s state and the complexity of the task.

See the article: Groß, A., Thomzik, B., Wrede, B. & Richter, B. (2026). Leveraging Cognitive States for Adaptive Scaffolding of Understanding in Explanatory Tasks in HRI. ACM Transactions on Human-Robot Interaction, 15(5), Article 98. https://doi.org/10.1145/3834775

The research was funded by the German Research Foundation (DFG) as part of TRR 318 ‘Constructing Explainability’.