health technology

Collecting Remote Voice and Movement Data from People with Parkinson’s Disease (PD) using Multimodal Conversational AI: Lessons Learned from a National Study

Telehealth is increasingly gaining recognition in the fields of neurology and speech-pathology (SLP) as a means of addressing accessibility issues experienced by people with PD. We present one potential solution -- a conversational artificial …

Multimodal Dialog Based Remote Patient Monitoring of Motor Function in Parkinson’s Disease and Other Movement Disorder

The need for remote monitoring to support Parkinson’s Disease (PD) patients, caregivers, and healthcare professionals in their collaborative efforts for better care has never been greater. We have earlier presented Tina, a virtual agent that is …

On the Robust Automatic Computation of Speaking and Articulation Duration in ALS Patients Versus Healthy Controls

Speaking and articulation duration have been shown to be important biomarkers of disorder severity in ALS and other motor speech disorders. While signal processing algorithms afford significant advantages over manual annotation in that they allow for …

Multimodal Dialog Based Speech and Facial Biomarkers Capture Differential Disease Progression Rates for ALS Remote Patient Monitoring

Continuous monitoring of ALS symptoms is crucial to improve quality of life because both survival and progression rate can vary substantially between individuals. Speech and facial biomarkers can serve as useful proxies for disease progression and …

Voice Activity Detection Considerations in a Dialog Agent for Dysarthric Speakers

Conversational dialog technology is increasingly being recognized as auseful means of automating remote patient monitoring and diagnostics for dysarthric speakers at scale. However, the characteristics of dysarthric speech introduce multiple …

Multimodal Conversational Technology for Remote Assessment of Symptom Severity in People with Schizophrenia

The causes of schizophrenia are complex and largely unknown. Current treatments focus on the management of positive or psychotic symptoms. However, negative symptoms like lack of facial and verbal expression, monotone speech, social withdrawal and …

Investigating the Interplay Between Affective, Phonatory and Motoric Subsystems in Autism Spectrum Disorder Using a Multimodal Dialogue Agent

We explore the utility of an on-demand multimodal conversational platform in extracting speech and facial metrics in children with Autism Spectrum Disorder (ASD). We investigate the extent to which these metrics correlate with objective clinical …

Investigating the Utility of Multimodal Conversational Technology and Audiovisual Analytic Measures for the Assessment and Monitoring of Amyotrophic Lateral Sclerosis at Scale

We propose a cloud-based multimodal dialog platform for the remote assessment and monitoring of Amyotrophic Lateral Sclerosis (ALS) at scale. This paper presents our vision, technology setup, and an initial investigation of the efficacy of the …

On the Utility of Audiovisual Dialog Technologies and Signal Analytics for Real-time Remote Monitoring of Depression Biomarkers

We investigate the utility of audiovisual dialog systems combined with speech and video analytics for real-time remote monitoring of depression at scale in uncontrolled environment settings. We collected audiovisual conversational data from …

NEMSI: A Multimodal Dialog System for Screening of Neurological or Mental Conditions

We present NEMSI, a cloud-based multimodal dialog system designed to have naturalistic interactions with individuals for the purpose of screening neurological or mental conditions. The system has been used by thousands of people capturing audio and …