Our analysis course: Designing for accessibility
In our early analysis, we discovered {that a} vital barrier to digital fairness is the “accessibility hole”, i.e., the delay between the discharge of a brand new characteristic and the creation of an assistive layer for it. To shut this hole, we’re shifting from reactive instruments to agentic methods which are native to the interface.
Analysis pillar: Utilizing multi-system brokers to enhance accessibility
Multimodal AI instruments present probably the most promising paths to constructing accessible interfaces. In particular prototypes, equivalent to our work with net readability, we’ve examined a mannequin the place a central Orchestrator acts as a strategic studying supervisor.
As an alternative of a person navigating a fancy maze of menus, the Orchestrator maintains shared context — understanding the doc and making it extra accessible by delegating the duties to knowledgeable sub-agents.
- The Summarization Agent: It masters advanced paperwork by breaking down info and delegating key duties to knowledgeable sub-agents, making even the deepest insights clear and accessible.
- The Settings agent: Handles UI changes, equivalent to scaling textual content, dynamically.
By testing this modular method,our analysis reveals customers can work together with methods extra intuitively, guaranteeing that specialised duties are at all times dealt with by the best knowledgeable with out the person needing to hunt for the “appropriate” button.
Towards multimodal fluency
Our analysis additionally focuses on shifting past primary text-to-speech towards multimodal fluency. By leveraging Gemini’s capability to course of voice, imaginative and prescient, and textual content concurrently, we’ve constructed prototypes that may flip stay video into speedy, interactive audio descriptions.
This is not nearly describing a scene; it’s about situational consciousness. In our co-design classes, we’ve noticed how permitting customers to interactively question their surroundings — asking for particular visible particulars as they occur — can cut back cognitive load and rework a passive expertise into an lively, conversational exploration.