Smartphone Accessibility

From newcomers to experts, we have characterized the challenges that blind people overcome. We explored how to provide effective self-contained smartphone assistance with the ultimate goal of promoting self-efficacy.

Motivation

Mobile devices are fundamental tools for inclusion and independence. Yet, there are still many open research issues in smartphone accessibility for blind people. Currently, learning how to use a smartphone is non-trivial, especially when we consider that the need to learn new apps and accommodate to updates never ceases. When first transitioning from a basic feature-phone, people have to adapt to new paradigms of interaction. Where feature phones had a finite set of applications and functions, users can extend the possible functions and uses of a smartphone by installing new 3rd party applications. Moreover, the interconnectivity of these applications means that users can explore a seemingly endless set of workflows across applications. To that end, the fragmented nature of development on these devices results in users needing to create different mental models for each application. These characteristics make smartphone adoption, and at times, use, a demanding task.

Characterization

We conducted multiple studies to characterize the smartphone challenges blind people face and found people often require synchronous, co-located assistance from family, peers, friends, and even strangers to overcome the different barriers they face. However, help is not always available, especially when we consider the disparity in each barrier, individual support network and current location.

Human-Powered

We investigated if and how in-context human-powered solutions can be leveraged to improve current smartphone accessibility and ease of use. Building on a comprehensive knowledge of the smartphone challenges faced and coping mechanisms employed by blind people, we explored how human-powered assistive technologies can facilitate use. We developed and explored, question and answer services, in-context tutorials, and task assistants.

We envision a paradigm shift where interactions, and contributions by knowledgeable users can assist others beyond what app and OS provide. Thus, establishing a support network where there was none, enabling the creation of services, and tools that tackle accessibility challenges, informed by usage data.

Accessibility Layers

We investigated the benefits and shortcomings of a personalizable, static and consistent interface (PSI) on mobile devices for non-visual access, alongside its acceptance by blind people (novice and expert users).

Publications