audiovisual analytics

Multimodal Dialog Based Speech and Facial Biomarkers Capture Differential Disease Progression Rates for ALS Remote Patient Monitoring

Continuous monitoring of ALS symptoms is crucial to improve quality of life because both survival and progression rate can vary substantially between individuals. Speech and facial biomarkers can serve as useful proxies for disease progression and …

Voice Activity Detection Considerations in a Dialog Agent for Dysarthric Speakers

Conversational dialog technology is increasingly being recognized as auseful means of automating remote patient monitoring and diagnostics for dysarthric speakers at scale. However, the characteristics of dysarthric speech introduce multiple …

Multimodal Conversational Technology for Remote Assessment of Symptom Severity in People with Schizophrenia

The causes of schizophrenia are complex and largely unknown. Current treatments focus on the management of positive or psychotic symptoms. However, negative symptoms like lack of facial and verbal expression, monotone speech, social withdrawal and …

Investigating the Interplay Between Affective, Phonatory and Motoric Subsystems in Autism Spectrum Disorder Using a Multimodal Dialogue Agent

We explore the utility of an on-demand multimodal conversational platform in extracting speech and facial metrics in children with Autism Spectrum Disorder (ASD). We investigate the extent to which these metrics correlate with objective clinical …

Investigating the Utility of Multimodal Conversational Technology and Audiovisual Analytic Measures for the Assessment and Monitoring of Amyotrophic Lateral Sclerosis at Scale

We propose a cloud-based multimodal dialog platform for the remote assessment and monitoring of Amyotrophic Lateral Sclerosis (ALS) at scale. This paper presents our vision, technology setup, and an initial investigation of the efficacy of the …

On the Utility of Audiovisual Dialog Technologies and Signal Analytics for Real-time Remote Monitoring of Depression Biomarkers

We investigate the utility of audiovisual dialog systems combined with speech and video analytics for real-time remote monitoring of depression at scale in uncontrolled environment settings. We collected audiovisual conversational data from …