RoadSpecs: Smart Glasses-Based Conversational Interventions for Real-Time Drowsy Driving Prevention

Can conversation keep a drowsy driver awake at the wheel?

Overview

An undergraduate thesis conducted at the Human-X Interactions Lab, De La Salle University – Manila. RoadSpecs investigates whether real-time AI-driven conversation can serve as a countermeasure to drowsy driving, using a custom smart glasses prototype with infrared and gyroscope-based drowsiness detection paired with a conversational agent that intervenes when drowsiness is detected.

Role

UX Researcher & Conversation Designer: co-designed the conversational intervention system, led user study protocol design, conducted semi-structured and follow-up interviews, and analyzed conversation transcripts and behavioral data.

Lab

Human-X Interactions Lab, DLSU–Manila

Team

Beatrice Berenguer, Leigh Arriane Buendia, Alfonso Gabriel Lima, Christopher Mari Pinpin

Adviser

Dr. Briane Paul V. Samson

Timeline

August 2022 to August 2023

Methods

User studies (n=30), semi-structured interviews, PERCLOS analysis, linear regression, video & transcript analysis

Tech Stack

PythonArduinoCARLA SimulatorBlenderBotGoogle TTS

Case Study

Overview

RoadSpecs is an undergraduate thesis conducted at the Human-X Interactions Lab at De La Salle University – Manila, investigating whether conversation can serve as a real-time countermeasure to drowsy driving. We designed and built a smart glasses prototype equipped with infrared and gyroscope sensors to detect drowsiness in drivers, paired with a conversational agent that initiates either mundane (casual chat) or game-aided (trivia and riddles) conversations when drowsiness is detected.

Through a 45-minute driving simulation study with 30 sleep-deprived and night-shift working drivers, we found that conversation (regardless of type) significantly reduces drowsiness during the interaction, with effects lasting 5 to 15 minutes post-conversation. The study also revealed that mundane conversation was overwhelmingly preferred by drivers for its natural, companion-like quality, while game-aided conversation introduced cognitive load that some participants found distracting in a driving context. Detection-wise, infrared-based eye blink frequency achieved a 90% true-positive rate, while head orientation proved to be a weak indicator of drowsiness behind the wheel.

The Problem

1.3M

people die in road accidents annually worldwide, and drowsy driving's true prevalence is estimated to be 350% greater than reported cases.

Road accidents kill 1.3 million people annually worldwide, and drowsy driving is one of the most underreported contributors. In the Philippines alone, roughly 12,000 people die on the road each year, with drowsy driving disproportionately affecting young drivers aged 18 - 29.

~12,000

road fatalities per year in the Philippines, with drowsy driving disproportionately affecting drivers aged 18 - 29.

Most proven countermeasures require the driver to stop driving entirely: pulling over for a nap, switching drivers, or consuming caffeine. While drivers consistently rank 'conversing with a passenger' as one of their preferred ways to stay alert, almost no research had empirically tested whether conversation actually works as an active, in-drive countermeasure. Interactive approaches like games were similarly under-explored.

	Works While Driving	No Passenger Needed	Always Available	Immediate Effect
Pull Over for a Nap	—	✓	✓	✓
Switch Drivers	—	—	—	✓
Consume Caffeine	—	✓	—	—
Converse with a Passenger	✓	—	—	✓
RoadSpecs (Conversational Agent)	✓	✓	✓	✓

Can a conversational agent, triggered by real-time drowsiness detection through smart glasses, keep drivers alert, and does the type of conversation matter?

Research & Discovery

Literature Review

We mapped the existing research across three pillars: drowsy driving countermeasures, drowsiness detection methods, and in-vehicle interactive systems. Detection methods were well-studied: EEG, computer vision, infrared sensors, but active countermeasures that work during the drive were severely under-researched. The few studies that touched on conversation and interactive media had contradicting results and lacked proper experimental testing.

Key insights from prior work

—Drivers perceive conversation with a passenger as their most effective in-drive countermeasure, yet no controlled study had validated this.

—Interactive media (games, mental tasks) showed potential for boosting cognitive availability, but evidence was limited to observational findings.

—Wearable eyewear with multiple sensors (gyroscope + infrared) offered the least intrusive drowsiness detection method with reasonable accuracy.

—PERCLOS (Percentage of Eye Closure) was an established metric for drowsiness, but its threshold values varied across populations.

Drowsy Driving CountermeasuresWell-established methods (napping, switching drivers, caffeine) all require stopping the drive. Conversation with a passenger was ranked #1 by 38.52% of drivers surveyed, but no controlled study had ever validated it. Interactive media (games, mental tasks) had theoretical support but zero experimental testing. RoadSpecs is the first controlled study to test conversation as an active, in-drive countermeasure.

Drowsiness Detection MethodsEEG is accurate but requires an intrusive neural headset. Computer vision (yawning detection) has inconsistent results due to lighting and skin-tone variability. Infrared blink detection and PERCLOS offered a practical middle ground, moderate accuracy, non-intrusive, wearable. Head orientation via gyroscope added coverage for physical drowsiness signals. RoadSpecs combined both on standard glass frames at a fraction of the cost of commercial solutions.

In-Vehicle Interactive SystemsTakayama & Nass (2008) showed interactive media helps drowsy drivers feel more involved and drive more safely than passive media. Aidman et al. (2015) found real-time drowsiness feedback substantially reduced subjective drowsiness ratings. But no prior work had coupled a detection system with a context-triggered conversational agent and evaluated distinct conversation strategies with real sleep-deprived drivers.

Participant Profile

We recruited 30 participants across three age groups (18-29, 30-39, 40-50), all of whom were either night-shift workers or sleep-deprived (fewer than 6-7 hours of sleep). This demographic reflected both the Philippine driver population and the groups most at risk for drowsy driving.

n = 30

participants across three age groups: all sleep-deprived or night-shift workers at time of study.

18-29Disproportionately affected by drowsy driving in Philippine road statistics. Heaviest representation in the study.

30-39Active working adults; balanced mix of night-shift workers and sleep-deprived participants.

40-50Older cohort; showed notably different responses to game-aided conversation, reporting higher cognitive load.

Inclusion criteria

—Sleep-deprived: fewer than 6-7 hours of sleep in the past 24 hours

—Night-shift workers: regularly working shifts that disrupt normal sleep patterns

—Active drivers with a valid driver's license

Ideation & Design

We designed a two-part system: a smart glasses prototype for real-time drowsiness detection, and a conversational agent that triggers intervention when drowsiness is detected.

01Biometric InputIR Sensor + MPU-6050Continuously captures eye blink frequency and head orientation while the driver is behind the wheel

→

02ProcessingArduino UnoReads raw sensor data and pipes it into the drowsiness detection logic

→

03DetectionPERCLOS + Head TiltTwo parallel streams evaluate eye closure % and head tilt angle against calibrated thresholds

→

04TriggerDrowsiness FlagPERCLOS ≥ 50% or sustained head tilt >3 sec activates the conversational agent

→

05InterventionBlenderBot / Trivia BankMundane or game-aided conversation runs ~5 min via Speech Recognition input and Google TTS output

→

06MonitoringCARLA + Webcam + MicDriving behavior, facial video, and vocal responses captured throughout for post-analysis

EnclosureSmart Glasses Prototype (Custom-built wearable)IR Transmitter/Receiver + MPU-6050 gyroscope mounted on standard glass frames, wired to Arduino Uno

Stream A: Eye Blink Frequency (PERCLOS)IR sensor captures blink data → calculates PERCLOS → compares against 50% threshold → if PERCLOS ≥ 50%, driver is flagged as drowsy.

Stream B: Head OrientationMPU-6050 captures tilt data → compares against thresholds (±0.27 up/down, ±0.26 left/right) → if threshold exceeded for >3 seconds, driver is flagged as drowsy.

Smart Glasses Prototype

Glass frames embedded with an infrared sensor (measuring eye blink frequency) and an MPU-6050 gyroscope/accelerometer (measuring head orientation). We chose eyewear specifically because it was the least intrusive form factor for drivers, far more comfortable than EEG headsets or camera-based systems.

Drowsiness Detection Logic

We calibrated the PERCLOS threshold through a preliminary study where participants performed mundane tasks while wearing the glasses, then self-annotated moments of drowsiness. A 50% PERCLOS threshold proved more accurate than the standard 60%, catching all self-reported drowsy states in our calibration participants.

PERCLOS Threshold Calibration

missed by standard threshold

caught by both thresholds

⚠ Critical detection gap: 3 drowsy episodes missed by standard threshold

Calibration participants performed mundane tasks while wearing the smart glasses and self-annotated moments of drowsiness. The 50% PERCLOS threshold detected all drowsy states. The standard 60% missed 60% of them.

Conversation Design

We designed two distinct conversation types, each lasting approximately five minutes:

Mundane ConversationPowered by BlenderBot (Meta's open-domain chatbot), this mode held flexible, casual conversations about the driver's personal life, hobbies, interests, and daily experiences. The goal was to simulate the feeling of having a passenger in the car.

Game-aided ConversationA curated set of general knowledge trivia and riddles, randomized per session. Difficulty was intentionally kept at easy-to-moderate to stimulate thinking without overwhelming the driver. Hints were provided for incorrect riddle answers to sustain engagement.

Experiment Design

We built a full driving simulation setup using CARLA (open-source autonomous driving simulator) with a steering wheel, pedals, car seat, and monitor to replicate a driver's perspective. Each session followed a structured flow:

Pre-test (10-15 min)Briefing, consent, glasses calibration, and a practice drive to familiarize participants with the simulator.

Simulation (45 min)Free driving with no set destination, monitored continuously for drowsiness. When detected, a conversation triggers. Maximum of two conversations per session with a 5-minute observation gap between them.

Post-test (5-10 min)Semi-structured interview on drowsiness experience, conversation quality, and perceived alertness effects.

Follow-up interview (15-20 min)Deeper probing on attentiveness before, during, and after each conversation, with video playback to refresh participant memory.

Key Findings

Drowsiness Detection Accuracy

90%

true-positive rate for infrared-based eye blink detection, matching participant self-reported drowsiness for both first and second conversation triggers.

Eye blink frequency (PERCLOS): strong indicatorThe infrared sensor matched self-reported drowsiness 90% of the time. However, 76.67% of participants had at least one undetected drowsy period, their PERCLOS readings were trending upward (averaging high 30s to low 40s) but hadn't yet crossed the 50% threshold. This suggests the threshold needs further personalization per driver.

Head orientation: weak indicatorNone of the 30 participants' head orientation data reached drowsiness thresholds during driving simulation. Even when participants visibly yawned or showed physical signs of drowsiness, their head movements were too minimal. Drivers maintain a steady forward-facing posture while driving, the head-drooping thresholds that worked during desk-based calibration did not transfer to a driving context.

Confusion matrix showing 90% true-positive rate for PERCLOS-based drowsiness detection

PERCLOS line graph with conversation periods highlighted showing drowsiness dropping during conversation and returning after

Flat head orientation graph showing values never reaching detection thresholds across the full simulation

Video still of participant leaning alongside flat head orientation graph during the same timestamp

PERCLOS comparison of two participants during non-detected drowsiness periods showing readings trending toward but not reaching the 50% threshold

Conversation Effectiveness

Both conversation types significantly reduced drowsinessPERCLOS slope analysis showed consistent negative slopes (decreasing drowsiness) across all conversation scenarios while the agent was active, regardless of whether the conversation was mundane or game-aided.

Effects lasted 5-15 minutes post-conversationPost-conversation PERCLOS data showed continued drowsiness reduction for a short window after the agent stopped, before levels began climbing again. Participants corroborated this in interviews.

Mundane conversation was strongly preferredDrivers described mundane conversation as feeling like having a companion, natural, casual, and easy to engage with while keeping their eyes on the road. Game-aided conversation was cognitively demanding; some found trivia stimulating, but others, especially participants aged 40 and above, reported feeling distracted, their attention split between driving and thinking through answers.

Sequence matteredParticipants who experienced mundane conversation first, then game-aided, showed the strongest overall drowsiness alleviation, and this group also preferred game-aided, suggesting the casual warm-up made the cognitive challenge feel engaging rather than frustrating. Conversely, participants who received game-aided first often felt overwhelmed and preferred mundane afterward.

Chart comparing average PERCLOS slopes before, during, and after conversation across all scenarios showing consistent drowsiness reduction during conversation

Mundane

Game-aided

Both conversation types reduced drowsiness: mundane showed a slightly longer sustained effect

Average slope graph for two conversations with mundane triggered first

Average slope graph for two conversations with game-aided triggered first

Mundane → Game-aidedStrongest overall effect

Game-aided → MundaneFaster drowsiness return after first conversation

Participants who experienced mundane first had better outcomes, and paradoxically preferred the game-aided conversation that followed

Summary statistics table showing mean and median PERCLOS slopes before during and after one conversation confirming consistent drowsiness reduction pattern

Communication Breakdowns

Speech-to-text accuracy was the primary friction point. Several participants reported the agent misunderstanding their responses, leading to unrelated follow-up questions or repetitive prompting. Participants who code-switched into Tagalog encountered the English-only language barrier, though the agent attempted to continue the conversation around unfamiliar words.

Conversation transcript showing speech-to-text misrecognition causing the agent to repeat the same question while the participant tries to correct their response

Conversation transcript showing participant using Tagalog words and the agent generating a confused response due to the English-only language barrier

Results & Impact

Conversation works, but temporarily. This study provided the first empirical evidence that conversation, regardless of type, significantly alleviates drowsiness in drivers for a short period (5-15 minutes). This positions conversation as a viable supplementary countermeasure, not a replacement for pulling over.

90% true-positive detection ratePERCLOS-based infrared blink detection validated as a practical, non-intrusive method for wearable drowsiness detection in real driving contexts.

Head orientation is not a reliable drowsiness signal while drivingA finding that challenges assumptions in prior literature and has direct implications for future wearable design, desk-based calibration thresholds do not transfer to driving.

Mundane conversation outperformed game-aidedIn both user preference and sustained engagement, natural low-cognitive-load interaction proved better suited for a safety-critical context like driving.

Conversation sequencing affects both effectiveness and perceptionA nuanced finding with direct design implications for future in-vehicle conversational agents, the order in which intervention types are introduced shapes how drivers experience and respond to them.

Summary table showing mean median and standard deviation of PERCLOS slopes before during and after conversation confirming consistent drowsiness reduction

Summary table comparing insights between mundane and game-aided conversations covering effects before during and after conversation as well as participant preferences

90%

true-positive drowsiness detection rate

across all 30 participants

5-15 min

alertness window post-conversation

before drowsiness returns

30/30

participants showed reduced drowsiness during conversation

Mundane > Game

in sustained engagement & participant preference

Reflection

This project shaped how I think about designing for high-stakes, real-world contexts. A few things I'd carry forward:

Context changes everything about detection thresholdsOur desk-calibrated head orientation thresholds completely failed in a driving context because the task itself constrains the user's body. Designing detection systems requires testing in the actual use environment, not a proxy.

User preference doesn't always equal effectivenessMundane conversation was overwhelmingly preferred, but the strongest PERCLOS reduction came from a mundane-then-game-aided sequence. Designing for safety means balancing what users want with what the data shows.

Speech-to-text was the weakest linkThe conversational agent's design was solid, but the quality of the speech recognition pipeline undermined the experience for several participants. Investing more in error handling and graceful recovery for misheard responses would have improved the overall interaction quality.

Bilingual users need bilingual systemsMany of our Filipino participants naturally code-switched between English and Tagalog. Designing for a single-language interaction in a bilingual population created unnecessary friction. Future iterations should support mixed-language input.

Next Project

ABAS