Publications

Computer with headphones and a tray on top and a set of tangible blocks on the side

Learning Maths with a Tangible User Interface: Lessons Learned through Participatory Design with Children with Visual Impairments and Their Educators

Through a set of participatory design sessions with children with visual impairments and their educators, we understood current practices in maths teaching, and designed a novel system to support learning for this particular educational context. Sixteen children were engaged in 19 PD sessions to develop tangibles and auditory stimuli to represent numbers, and to explore activities to use through a tangible user interface.

Ana Cristina Pires, Ewelina Bakala, Fernando Gonzalez-Perilli, Gustavo Sansone, Bruno Fleischer, Sebastian Marichal, Tiago Guerreiro

IJCCI 2021 ‑ International Journal of Child-Computer Interaction

generic image of a publication

Using Games to Practice Screen Reader Gestures

We explore the use of games to inconspicuously train gestures. We designed and developed a set of accessible games, enabling users to practice smartphone gestures. We evaluated the games with 8 blind users and conducted remote interviews. Our results show how purposeful accessible games could be important in the process of training and discovering smartphone gestures, as they offer a playful method of learning. This, in turn, increases autonomy and inclusion, as this process becomes easier and more engaging.

Gonçalo Lobo, David Gonçalves, Pedro Pais, Tiago Guerreiro, André Rodrigues

ASSETS 2021 ‑ ACM Conference on Computers and Accessibility

The myView editing web application showing a picture of a supermarket corridor and an editing form

myView: End-user Authoring of Virtual Environments for Therapy

We present a smartphone tool that allows non-specialists to create navigable virtual environments by taking and linking sequences of panoramic photo spheres, analogly to Google Street View. Editing the environments is then possible in a web platform, myView, where text, images, videos, sounds, and pick-up objects can be added. myView allows users to navigate their environments as well as sharing those environments with others. In a preliminary study with two psychologists, where myView was used as an elicitation probe, the approach was found to be useful for creating meaningful activities for reminiscence and cognitive training. The platform showed to be promising in the democratization of the crafting of virtual environments.

Sérgio Alves, Pedro Caldeira, Filipa Brito, Luís Carriço, Tiago Guerreiro

ASSETS 2021 ‑ ACM Conference on Computers and Accessibility

The top of the image represents the programming area where the user has a box with tangible blocks, the PC, a webcam and builds the sequence of instructions. The child says the word start so that the instructions are executed. The bottom of the image represents the execution area where another user has the Dash robot with audio feedback, a checkered map with tactile references in all lines, and a box with various obstacles.

Fostering Collaboration with Asymmetric Roles in Accessible Programming Environments for Children with Mixed-visual-abilities

We propose the design of a programming environment that leverages asymmetric roles to foster collaborative computational thinking activities for children with visual impairments, in particular mixed-visual-ability classes. The multimodal system comprises the use of tangible blocks and auditory feedback, while children have to collaborate to program a robot. We conducted a remote online study, collecting valuable feedback on the limitations and opportunities for future work, aiming to potentiate education and social inclusion.

Filipa Rocha, Guilherme Guimarães, David Gonçalves, Ana Cristina Pires, Lúcia Abreu, Tiago Guerreiro

ASSETS 2021 ‑ ACM Conference on Computers and Accessibility

a scenario where a blind user is navigating her smartphonr while receiving audio instructions on how to perform a task

Promoting Self-Efficacy Through an Effective Human-Powered Nonvisual Smartphone Task Assistant

We contribute with a human-powered nonvisual task assistant for smartphones to provide pervasive assistance. We argue, in addition to success, one must carefully consider promoting and evaluating factors such as self-efficacy and the belief in one’s abilities to control and learn to use technology. In this paper, we show effective assistant positively affects self-efficacy when performing new tasks with smartphones, affects perceptions of accessibility and enables systemic task-based learning.

André Rodrigues, André Santos, Kyle Montague, Tiago Guerreiro

CSCW 2021 ‑ ACM Conference on Computer-Supported Cooperative Work and Social Computing, October, 2021

A chat application with the keyboard open and the written sentences displayed: Collects anywhere the user writes, Password and number fields are ignored, calculates more than 20 metrics

WildKey: A Privacy-Aware Keyboard Toolkit for Data Collection In-The-Wild

We present WildKey, an Android keyboard toolkit that allows for the usable deployment of in-the-wild user studies. WildKey is able to analyze text-entry behaviors through implicit and explicit text-entry data collection while ensuring user privacy. We detail each of the WildKey’s components and features, all of the metrics collected, and discuss the steps taken to ensure user privacy and promote compliance.

André Rodrigues, André Santos, Kyle Montague, Hugo Nicolau, Tiago Guerreiro

WildByDesign 2021 ‑ UbiComp/ISWC Workshop on Designing Ubiquitous Health Monitoring Technologies for Challenging Environments

Real-world environment: a walk in the countryside with a lot of mud and a cloudy sky

Wild by Design: Workshop on Designing Ubiquitous Health Monitoring Technologies for Challenging Environments

In this workshop, we will focus on the challenges of real world health monitoring deployments to produce forward-looking insights that can shape the way researchers and practitioners think about health monitoring, in platforms and systems that account for the complex environments where they are bound to be used.

Diogo Branco, Patrick Carrington, Silvia Del Din, Afsaneh Doryab, Hristijan Gjoreski, Tiago Guerreiro, Roisin McNaney, Kyle Montague, Alisha Pradhan, André Rodrigues, Julio Vega

Wild by Design 2021 ‑ Workshop on Designing Ubiquitous HealthMonitoring Technologies for Challenging Environments at Ubicomp 2021

generic image of a publication

Exploring how a digitized program can support parents to improve their children’s nutritional habits

Poor eating habits are one of today’s significant menaces to public health. Child obesity is increasing, is a concerning reality, and needs to be appropriately addressed. However, most behavior change programs do not consider the needs of parents and their children, their profiles, and environments in the design of this type of intervention. We present the results of a workshop with dietists and clinical psychologists, professionals that deal with different parents and their children’s dietary problems, to understand parents’ profiles, attitudes, and perceptions. The main contributions of this study are a set of personas, daily scenarios, and design considerations regarding behavior change programs that can be used to guide the creation of new digital programs. This formative contribution is of interest to researchers and practitioners designing digitized behavior change programs targeted at parents to improve their children’s habits.

Diogo Branco, Ana C. Pires, Hugo Simão, Ana Gomes, Ana Pereira, Joana Sousa, Luísa Barros, Tiago Guerreiro

INTERACT 2021 ‑ International Conference on Human-Computer Interaction, September, 2021

A Pine Time smartwatch

Participatory Action Research and Open Source Hardware Appropriation for Large Scale In-The-Wild Studies

We present the Cue Band study, which revolves around the creation of a wristband cueing device for people with Parkinson’s that experience drooling. We present an approach for research in-the-wild, which draws on participatory action research theory, that places the end-user at the centre of the process, aiming to first and for most to create a workable product for the end-user, before engaging in a formal study. In the last section, we explore the appropriation of existing open-source hardware for in-the-wild research, by describing problems and solutions associated with developing Ubicomp technologies for large-scale studies.

Luís Carvalho, Dan Jackson, Tiago Guerreiro, Yu Guan, Kyle Montague

WildByDesign 2021 ‑ UbiComp/ISWC Workshop on Designing Ubiquitous Health Monitoring Technologies for Challenging Environments

generic image of a publication

Suggesting text alternatives for images in social media

We present SONAAR, a project that aims to improve the accessibility of user-generated content on social networks. Our approach is to support the authoring and consumption of accessible social media content. Our prototypes currently focus on Twitter and Facebook and are available as an Android application and as a Chrome extension.

Letícia Seixas Pereira, João Guerreiro, André Rodrigues, André Santos, João Vicente, José Coelho, Tiago Guerreiro, Carlos Duarte

SIGACCESS 2021 ‑ Newsletter June Edition

Three pictures demonstrating families' interaction with the system. Left: Two children are around the setup. The visually impaired child confirms with his sighted sibling to where the robot is facing. Middle: Child crawls on the map searching for the robot. Right: Parent and child facing the setup. Parent is asking questions to engage child in play.

Accembly at Home: Accessible Spatial Programming for Children with Visual Impairments and their Families

We present ACCembly, an accessible block-based environment that enables children with visual impairments to perform spatial programming activities. ACCembly allows children to assemble tangible blocks to program a multimodal robot. We evaluated this approach with seven families that used the system autonomously at home. We contribute with an environment that enables children with visual impairments to engage in spatial programming activities, an analysis of parent-child interactions, and reflections on inclusive programming environments within a shared family experience.

Filipa Rocha, Ana Pires, Isabel Neto, Hugo Nicolau, Tiago Guerreiro

IDC 2021 ‑ ACM Interaction Design and Children, June, 2021

Twitter alt text dialog showing a flower and its description being introduced

Nipping Inaccessibility in the Bud: Opportunities and Challenges of Accessible Media Content Authoring

Our work focuses on promoting awareness to accessible social media authoring practices and in assisting the authoring process. We have prototyped a Google Chrome extension and an Android application that can identify when a Twitter or a Facebook user is authoring content with images and suggests a text alternative for the image. In this paper, we highlight some of the challenges faced to offer this support in different technological platforms (web and mobile), but also ones that are raised by the domain characteristics (e.g. detecting the same image, supporting different languages) and that can be addressed through AI based technologies.

Carlos Duarte, Letícia Seixas Pereira, André Santos, João Vicente, André Rodrigues, João Guerreiro, José Coelho and Tiago Guerreiro

WebSci 2021 ‑ 13th ACM Web Science Conference

Shows an overview of the system. A large LEGO base plate and on top a set of components that make up the system: the Ozobot robot, the different caps (start, end, and path), LEGO caps with top codes and tactile arrows, and a dialog between a child and the system - What is the instruction? Move Forward. How many times? Two.

LEGOWorld: Repurposing Commodity Tools and Technologies to Create an Accessible and Customizable Programming Environment

We explore how commodity objects and technologies can be repurposed to provide a multimodal programming environment that is accessible to children with visual impairments, flexible, and scalable to a variety of programming challenges. Our approach resorts to four main components: 1) a LEGO base plate where LEGO blocks can be assembled to create maps, which is flexible and robust for tactile recognition; 2) a tangible programming area where LEGOs, with 3D printed caps enriched with tactile icons, can be assembled to create a program; 3) alternatively, the program can be created through a voice conversation with the system; and 4) a low-cost OzoBot Bit

Gonçalo Cardoso, Ana Pires, Lúcia Abreu, Filipa Rocha, Tiago Guerreiro

CHI EA 2021 ‑ Extended Abstracts of the ACM Conference on Human Factors in Computing Systems, May, 2021

generic image of a publication

Articulations toward a crip HCI

We present stories of research articulation, of researchers as articulated, and as researchers articulating, gesturing toward a crip HCI. Disability is a plural, fluid, transitory, embodied cultural experience. While crip theoretics of HCI are best shaped by disabled scholars, a crip practice affords and demands a broader uptake. The particulars and nuances of a crip HCI are still forming among the disabled HCI researchers in collaboration today. However, this practice of articulation within disabled community, disabled space, and disabled consciousness is an essential and ongoing process toward a more equitable, more just, more humane HCI practice.

Rua M. Williams, Kathryn Ringland, Amelia Gibson, Mahender Mandala, Arne Maibaum, Tiago Guerreiro

ACM Interactions 2021 ‑ May-June

a screenshot of the rescue:under pressure game showing a submarine in a dark sea

Exploring Asymmetric Roles in Mixed-Ability Gaming

We explore ability-based asymmetric roles as a design approach to create engaging and challenging mixed-ability play. Our team designed and developed two collaborative testbed games exploring asymmetric interdependent roles. In a remote study with 13 mixed-visual-ability pairs we assessed how roles affected perceptions of engagement, competence, and autonomy, using a mixed-methods approach. The games provided an engaging and challenging experience, in which differences in visual ability were not limiting. Our results underline how experiences unequal by design can give rise to an equitable joint experience.

David Gonçalves, André Rodrigues, Mike Richardson, Alexandra de Sousa, Michael Proulx, Tiago Guerreiro

CHI 2021 ‑ ACM Conference on Human Factors in Computing Systems, May, 2021

generic image of a publication

Barriers and Opportunities to Accessible Social Media Content Authoring

We performed an online survey of 258 social network users and a follow-up interview conducted with 20 of them - 7 of them self-reporting blind and 13 sighted users without a disability. Results show how the different approaches being employed by major platforms are still not sufficient to properly address social media accessibility. Our findings reveal that mainstream users are not aware of the possibility and the benefits of adopting accessible practices. From the general perspectives of end-users experiencing accessible practices, concerning barriers encountered, and motivational factors, we also discuss further approaches to create more user engagement and awareness.

Leticia Seixas Pereira, José Coelho, André Rodrigues, João Guerreiro, Tiago Guerreiro, Carlos Duarte

arxiv 2021 ‑ arXiv:2104.10968 [cs.HC], April 2021

{publication.award}

generic image of a publication

Designing Pervasive Assistive Technologies with Representative Users

This talk builds on more than 14 years of deep engagements and in-the-wild deployments of mobile technologies within a community of blind people. It makes the case for pervasive assistive technology researchers to be experts in their areas of study: people, and then, technology to serve and empower people.

Tiago Guerreiro

MPAT 2021 ‑ Workshop on Mobile and Pervasive Assistive Technologies (at IEEE Percom), March, 2021

Opening Keynote Address

generic image of a publication

Kinematic and clinical outcomes to evaluate the efficacy of a multidisciplinary intervention on functional mobility in Parkinson’s disease

We aimed to identify which kinematic and clinical outcomes better predict functional mobility changes when PD patients are submitted to a specialized multidisciplinary program.

Raquel Bouça-Machado, Diogo Branco, Gustavo Fonseca, Raquel Fernandes, Daisy Abreu, Tiago Guerreiro, Joaquim J Ferreira, Daniela Guerreiro, Verónica Caniça, Francisco Queimado, Pedro Nunes, Alexandra Saúde, Laura Antunes, Joana Alves, Beatriz Santos, Inês Lousada, Maria A Patriarca, Patrícia Costa, Raquel Nunes, Susana Dias

Frontiers in Neurology 2021 ‑ Frontiers in Neurology

screenshot of the application with the supermarket VR view and the options Game Modes, Challenges, Shopping list

Photo-realistic interactive virtual environments for neurorehabilitation in mild cognitive impairment (NeuroVRehab.PT): a participatory design and proof-of-concept study

We investigated the feasibility and rehabilitation potential of a new design approach to create highly realistic interactive virtual environments for MCI patients’ neurorehabilitation. Through a participatory design protocol, a neurorehabilitation digital platform was developed using images captured from a Portuguese supermarket (NeuroVRehab.PT). NeuroVRehab.PT main features (e.g., medium-size supermarket, use of shopping lists) were established accordingly to a shopping behavior questionnaire filled in by 110 older adults. Seven health professionals used the platform and assessed its rehabilitation potential, clinical applicability and user-experience.

Filipa Ferreira-Brito, Sérgio Alves, Osvaldo Santos, Tiago Guerreiro, Cátia Caneiras, Luís Carriço, Ana Verdelho

JCM 2020 ‑ Journal of Clinical Medicine

system architecture showing a low-level description of events and dom-tree textual representation of a view, and the higher-level description of those events (Opened Messenger, Scrolled for 5 seconds,..)

Usable Logging as a Security Response to Physical Attacks on Mobile Devices

Users are susceptible to privacy breaches when people close to them gain physical access to their phones. We present logging as a security response to this threat, one that is able to accommodate for the particularities of social relationships. To this end, and explore the feasibility of the logging approach, we present a prototype developed for Android that continuously gathers user interactions and translates them into human-readable units.

José Franco, Ana C. Pires, Luís Carriço Tiago Guerreiro

ACSAC 2020 ‑ Annual Computer Security Applications Conference

a playful environment designed from the ground up to be rich in both its story (a nautical game) and its mechanics (e.g., a physical robot-boat controlled with a 3D printed wheel), tailored to promote computational thinking

TACTOPI: a Playful Approach to Promote Computational Thinking for Visually Impaired Children

We present TACTOPI, a playful environment designed from the ground up to be rich in both its story (a nautical game) and its mechanics (e.g., a physical robot-boat controlled with a 3D printed wheel), tailored to promote computational thinking at different levels (4 to 8 years old). This poster intends to provoke discussion and motivate accessibility researchers that are interested in computational thinking to make playfulness a priority.

Lúcia Abreu, Ana C. Pires, Tiago Guerreiro

ASSETS 2020 ‑ ACM Conference on Computers and Accessibility

{publication.award}

generic image of a publication

Playing With Others: Depicting Multiplayer Gaming Experiences of People With Visual Impairments

We share multiplayer gaming experiences of people with visual impairments collected from interviews with 10 adults and 10 minors, and 140 responses to an online survey. We include the perspectives of 17 sighted people who play with someone who has a visual impairment, collected in a second online survey. Our focus is on group play, particularly on the problems and opportunities that arise from mixed-visual-ability scenarios. These show that people with visual impairments are playing diverse games, but face limitations in playing with others who have different visual abilities.

David Gonçalves, André Rodrigues, Tiago Guerreiro

ASSETS 2020 ‑ ACM Conference on Computers and Accessibility

Best Paper Nominee

Two children with visual impairments, one attaching a forward and play pieces, and the other feeling the robot and the target object, that walked through a foam path.

Exploring Accessible Programming with Educators and Visually Impaired Children

We report on a focus group with IT and special needs educators, where they discussed a variety of programming environments for children, identifying their merits, barriers and opportunities. We then conducted a workshop with 7 visually impaired children where they experimented with a bespoke tangible robot-programming environment. Video recordings of such activity were analyzed with educators to discuss children’s experiences and emergent behaviours. We contribute with a set of qualities that programming environments should have to be inclusive to children with different visual abilities, insights for the design of situated classroom activities, and evidence that inclusive tangible robot-based programming is worth pursuing.

Ana Cristina Pires, Filipa Rocha, António Barros, Hugo Simão, Hugo Nicolau, Tiago Guerreiro

IDC 2020 ‑ ACM Interaction Design and Children

Gait kinematic parameters in Parkinson’s disease: a systematic review

We summarize and critically appraise the characteristics of technology-based gait analysis in PD and provide mean and standard deviation values for spatiotemporal gait parameters. These results provide useful information for performing objective technology-based gait assessment in PD, as well as mean values to better interpret the results.

Raquel Bouça-Machado, Constança Jalles, Daniela Guerreiro, Filipa Pona-Ferreira, Diogo Branco, Tiago Guerreiro, Ricardo Matias, Joaquim J. Ferreira

JPD 2020 ‑ Journal of Parkinson’s Disease

A user with a row of RFID Card in front of him, while holding one close to the RFID reader..

Open Challenges of Blind People using Smartphones

Through a multiple methods approach we identify and validate challenges locally with a diverse set of user expertise and devices, and at scale through the analyses of the largest Android and iOS dedicate forums for blind people. We contribute with a prioritized corpus of smartphone challenges for blind people, and a discussion on a set of directions for future research that tackle the open and often overlooked challenges.

André Rodrigues, Hugo Nicolau, Kyle Montague, João Guerreiro and Tiago Guerreiro

IJHCI 2020 ‑ International Journal of Human Computer Interaction

{publication.award}

On the left, a group of older adults listens to the message recorded by the other group (located in a different room) and transmitted by DASH. On the right, four of the custom snap-on blocks, used by participants to command the robot.

Carrier-pigeon Robot: Promoting Interactions Among Older Adults in a Care Home

With this research topic, we explore a robot as a communication vehicle for older adults in care homes. We extend older people’s actions and increase their communication through a robot traveling across the institution. This robot is programmable by older people using 3D printed, tangible blocks with default actions to build sequences. This way, people are in charge of the approach and interaction to perform with other people inside the institution. Besides the improvement in communication for the general people institutionalized, the technique showed to be promising for people with mobility impairments to extend their action range.

Hugo Simão, Ana Cristina Pires, David Gonçalves, Tiago Guerreiro

HRI 2020 ‑ Companion of the 2020 ACM/IEEE International Conference on Human-Robot Interaction (HRI ‘20 Companion)

Best paper award

On the left three logos for human-powered solutions developed through the course of the phd. On the right 8 logos from the conferences where work was published. ASSETS, CHI, INTERACT, MobileHCI, and a Book chapter, with three ASSETS editions. Totaling 5 publications + 8 related with the phd.

Human-Powered Smartphone Assistance for Blind People

We investigated if and how in-context human-powered solutions can be leveraged to improve current smartphone accessibility and ease of use. The thesis of this dissertation is: Human-powered smartphone assistance by non-experts is effective and impacts perceptions of self-efficacy.

André Rodrigues

PhD Thesis 2020 ‑ Advisors: Tiago Guerreiro, Kyle Montague

{publication.award}

Radar chart with the 7 axis of the design space and their ordinal values: Interruption (interruptable, conditional, continuous), Confidence (dynamic, static), Output (implicit, explicit), Selection (single, multiple), Cardinality (one, two, three), Concurrency (sequentual, concurrent), and Notification (always, threshold).

The Design Space of Nonvisual Word Completion

Word completion interfaces are ubiquitously available in mobile virtual keyboards; however, there is no prior research on how to design these interfaces for screen reader users. In addressing this, we propose a design space for nonvisual representation of word completions. The design space covers seven categories aiming to identify challenges and opportunities for interaction design in an unexplored research topic.

Hugo Nicolau, André Rodrigues, André Santos, Tiago Guerreiro, Kyle Montague, João Guerreiro

ASSETS 2019 ‑ In The 21st International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ‘19). ACM, New York, NY, USA, 249-261.

Best Paper Nominee

iCETA: headphones, computer, mirror in the camera, tangible blocks and working area on top of the keyboard.

A Tangible Math Game for Visually Impaired Children

iCETA, an inclusive interactive system for math learning, designed through a set of participatory sessions with visually impaired children and their educators. iCETA supports math learning through the combination of tangible interaction with haptic and auditory feedback.

Ana Cristina Pires, Sebastian Marichal, Fernando Gonzalez-Perilli, Ewelina Bakala, Bruno Fleischer, Gustavo Sansone, Tiago Guerreiro

ASSETS 2019 ‑ 21th International ACM SIGACCESS Conference on Computers and Accessibility. Pittsburgh, PA, USA. October, 2019

Table with code frequencies in the tutorials instructions. Significant differences between sighted and blind people instructions. Sigthed people with 20% of incorrect text, and high number of references to location. Blind people with high number of gesture explanations and navigation cues.

Understanding the Authoring and Playthrough of Nonvisual Smartphone Tutorials

We sought to understand how sighted and blind people instruct other blind users to accomplish tasks on a mobile device. We also studied how those instructions enabled blind people to be successful. Results showed that a single pass of instructions was limited. A set of ways in which support can be provided is discussed.

André Rodrigues, André Santos, Kyle Montague, Hugo Nicolau, Tiago Guerreiro

INTERACT 2019 ‑ 17th IFIP TC13 International Conference on Human-Computer Interaction, Paphos, Cyprus, September, 2019

Photo taken during mathematics training with CETA system using tangibles. In this photo we observe an entire first grade classroom with 22 young children, playing with the blocks using our system CETA. Each child has a headphone, a tablet located on the table and the blocks to solve the additive composition tasks.

Building blocks of mathematical learning: digital and tangible manipulatives lead to different strategies in number composition

It is indispensable that objects may be grasped, lifted and explored or would it be enough to interact with virtual manipulatives? And specifically, how the objects’ affordances (i.e., the possibility to grasp physical objects or drag virtual ones) will shape and constrain children’s composing strategies.

Ana Cristina Pires, Fernando González Perilli, Ewelina Bakała, Bruno Fleisher, Gustavo Sansone and Sebastián Marichal

Frontiers in Education 2019 ‑ Educational Psychology, 2019

10 application screens all with very different interfaces. Interfaces with grids, lists, keyboard, no interactive items, logins, tutorials and tables.

Mobile Web

Accessing the Web with mobile devices, either through a browser or a native application, has become more than a perk; it is a need. Such relevance has increased the need to provide accessible mobile webpages. In this work, we focus our attention on the challenges of mobile devices for accessibility, and how those have been addressed in the development and evaluation of mobile interfaces and contents.

Tiago Guerreiro, Luís Carriço, André Rodrigues

Web Accessibility 2019 ‑ Chapter 38 in S. Harper & Y. Yesilada (eds.), Web Accessibility: A Foundation for Research (2nd ed.). London, England, Springer-Verlag.

Storyboard demonstrating a shower attack.

Vulnerability & Blame: Making Sense of Unauthorized Access to Smartphones

Unauthorized physical access to personal devices by people known to the owner of the device is a common concern, and a common occurrence. But how do people experience incidents of unauthorized access? Using an online survey, we collected 102 accounts of unauthorized access. Participants wrote stories about past situations in which either they accessed the smartphone of someone they know, or someone they know accessed theirs.

Diogo Marques, Tiago Guerreiro, Luís Carriço, Ivan Beschastnikh, Konstantin Beznosov

CHI 2019 ‑ ACM Conference on Human Factors in Computing Systems, Glasgow, Scotland, May, 2019

Caregiving to a person with Alzheimer can be a very demanding task, both from physical and psychological perspectives. Technological responses to support caregiving and improve the quality of life of people with Alzheimer and their caregivers are lacking.

MATY: Designing An Assistive Robot for People with Alzheimer’s

Using a research through design approach, we devised a robot focused on empowering people with Alzheimer and fostering their autonomy, from the initial sketch to a working prototype. MATY is a robot that encourages communication with relatives and promotes routines by eliciting the person to take action, using a multisensorial approach (e.g., projecting biographical images, playing suggestive sounds, or emitting soothing aromas). The paper reports the iterative, incremental design process performed together with stakeholders. We share first lessons learned in this process with HCI researchers and practitioners designing solutions, particularly robots, to assist people with dementia and their caregivers.

Hugo Simão, Tiago Guerreiro

CHI 2019 ‑ In The CHI EA ‘19 Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

datapark

Designing Free-Living Reports for Parkinson’s Disease

The democratization of sensing wearable technologies opened several possibilities in the continuous monitoring of people in their homes. We developed a platform where usable reports are presented to clinicians, particuarly in the context of Parkinson’s disease monitoring. The presented information originates from accelerometer sensors.

Diogo Branco, Raquel Bouça, Joaquim Ferreira, Tiago Guerreiro

CHI 2019 ‑ Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems, Glasgow, UK

An example of a clinicial assessment, sit to stand exercise

DataPark : A Data-Driven Platform for Parkinson’s Disease Monitoring

We developed a platform where usable reports are presented to clinicians, particuarly in the context of Parkinson’s disease monitoring. The presented information originates from accelerometer sensors and subjective data collected over an Interactive Voice Response system.

Diogo Branco, César Mendes, Ricardo Pereira, André Rodrigues, Raquel Bouça, Kyle Montague, Joaquim Ferreira, Tiago Guerreiro

WISH Symposium 2019 ‑ Workgrounp on Interactive Systems in Healthcare, co-located with CHI’19, Glasgow, UK, May, 2019

{publication.award}

Main cognitive stimulation activities of Scrapbook. From left to right: a. Reminiscence therapy; b. Flashcard; c. Street View navigation; d. Quiz; e. Touch; f. Puzzle (in fullscreen).

Designing Personalized Therapy Tools For People with Dementia

We iteratively designed a web platform focused on personalized cognitive stimulation. The platform was deployed in clinical contexts for several months and iterated, being enriched with functionalities like group reminiscence, caregiver app, or biographical activities.

Sérgio Alves, Andreia Cordeiro, Filipa Brito, Luís Carriço, Tiago Guerreiro

W4A 2019 ‑ 16th International Web for All Conference, San Francisco, USA, May, 2019

Best Technical Paper Nominee

The robot Baxter with his arms extended and a blind person feeling its hands

What My Eyes Can’t See, A Robot Can Show Me: Exploring the Collaboration Between Blind People and Robots

We presented a qualitative analysis of the expectations, fears and needs pointed by a sample of blind participants. In study 2, we implement and discuss the effect of two types of robotic assistance during the assembling task. Results from our two studies support the usefulness of developing and introducing this form of collaborative assistive technology in the lives of people with visual impairments. Positive outcomes for users (such as an increased level of autonomy in everyday life tasks) are outlined and discussed.

Mayara Bonani, Raquel Oliveira, Filipa Correia, André Rodrigues, Tiago Guerreiro, Ana Paiva

ASSETS 2018 ‑ ASSETS 2018 - 20th International ACM SIGACCESS Conference on Computers and Accessibility, Galway, Ireland, October, 2018

1.How to create a group in WhatApp. After we open WhatsApp. 2.We do the gesture until Chat. 3.Then again, left-right until New Chat. We enter. 4.Then left-right until groups. Then we go until the headline New Group. We enter. 5.Now we do the up down gesture to go to the last element of the page. Then without lifting the finger we double tap and stay on Next. 6.We are now in the edit box to write the subject to identify the group. We write the name and the subject and we click on the keyboard key to Submit. Now again, left-right until the Create button.

AidMe: Interactive Non-Visual Smartphone Tutorials

AidMe, is a system-wide authoring and playthrough tool of non-visual interactive tutorials. Tutorials are created via user demonstration and narration. In a user study with 11 blind participants we identified issues with instruction delivery and user guidance providing insights into the development of accessible interactive non-visual tutorials.

André Rodrigues, Leonardo Camacho, Hugo Nicolau, Kyle Montague, Tiago Guerreiro

MOBILECHI 2018 ‑ 20th International Conference on Human-Computer Interaction with Mobile Devices and Services, Barcelona, Spain, September, 2018

braille

Hybrid-Brailler : Combining Physical and Gestural Interaction for Mobile Braille Input and Editing

We present a smartphone case with physical buttons that allow users to write Braille in the back and gesture with the thumbs on the touchscreen. This enabled the study of novel editing approachs, very limited in commodity smartphones and accessibility services.

Daniel Trindade, André Rodrigues, Tiago Guerreiro, Hugo Nicolau

CHI 2018 ‑ ACM Conference on Human Factors in Computing Systems, Montreal, Canada, May, 2018

We propose a paradigm shift where interactions and contributions by knowledgeable users can assist others beyond what mobile applications and operating systems provide Interaction data collection methods are fragmented and are gathered by each app and operating system individually, with the purpose of self-improvement with limited control and awareness by the user.

Data Donors: Sharing Knowledge for Mobile Accessibility

Inspired by charitable donations, Data Donors, is a conceptual framework proposing the enablement of users with the capacity to help others to do so by donating their mobile interaction data and knowledge.

André Rodrigues, Kyle Montague, Tiago Guerreiro

CHI 2018 ‑ Late Breaking Work - Extended Abstracts of the ACM Conference on Human Factors in Computing Systems, Montreal, Canada, May, 2018

Patient performing a jigsaw puzzle.

Enabling Biographical Cognitive Stimulation for People with Dementia

In this paper we Introduce the initial development process of Scrapbook. After an initial study to understand current clinical practices, we developed a platform focused on enabling psychologists to perform reminiscence therapy with people with dementia. A two-week study was performed in a clinical environment.

Sérgio Alves, Andreia Cordeiro, Filipa Brito, Luís Carriço, Tiago Guerreiro

CHI 2018 ‑ Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal QC, Canada

Two figures (the Checker Shadow Illusion and an equivalent illusion) demonstrate the phenomenon of simultaneous contrast: objects with the same intensity of light can be perceived as beeing different colors, due to context. Squares A and B have the same hue of color, although they do not seem so; they are perceived as different in color. When changing the contrast it is then possible to see that they have the same color!

Livro Psicobiologia. Capitulo 16: ¿Cómo percibimos el mundo?

To perceive it is necessary to understand the role that our experiences play and how they are articulated with the biological mechanisms of information processing with which we come to the world. Perceiving is a complex process that is built from repeated exposure of our sensors to the response of the physical world with which we interact since we were born. In this chapter we will highlight this complex process based on the general functioning of our sensory processing, to then focus on the example of vision. It will deepen specific aspects of vision such as movement perception, of color or depth, to finally address the general problem that involves the necessary integration of the information that it is processed in different areas of the brain for the conformation of a single perception.

Alejandro Maiche, Ana Cristina Pires, Fernando Gonzalez, Lorena Chanes & Alejandro Vazquez

EMP 2018 ‑ Editorial Médica Panamericana

static overlays

Improving smartphone accessibility with personalizable static overlays

We present an approach that superimposes a virtual overlay to all other interfaces ensuring interface consistency by re-structuring how content is accessed in every screen. The screen is splitted in two, dedicating half to a configurable set of static options regardless of context; while the other enables the standard content navigation gestures with the ability to re-order content and apply filters.

André Rodrigues, André Santos, Kyle Montague, Tiago Guerreiro

ASSETS 2017 ‑ 19th International ACM SIGACCESS Conference on Computers and Accessibility. Baltimore, Maryland, USA, October, 2017

-A) Volunteer web app. It shows two answered questions, one with a specific element of the interface highlighted. B) Hint Me! with the always available button on the top of the screen, and a notification showing the user he received an answer.

In-context Q&A to Support Blind People Using Smartphones

Hint Me! is a human-powered service that allows blind users to get in-app smartphone assistance. We conducted an exploratory user study with six blind participants to elicit their perceptions on the usefulness, and acceptance of human-powered networks for smartphone support.

André Rodrigues, Kyle Montague, Hugo Nicolau, João Guerreiro and Tiago Guerreiro

ASSETS 2017 ‑ 19th International ACM SIGACCESS Conference on Computers and Accessibility. Baltimore, Maryland, USA, October, 2017.

A hacker with a black hoody, with a facebook page on the background

Characterizing Social Insider Attacks on Facebook

we ran two MTurk studies about social insider attacks on Facebook. In the first (n = 1,308), using the list experiment method, we estimated that 24% of participants had perpetrated social insider attacks and that 21% had been victims (and knew about it). In the second study (n = 45), participants wrote stories detailing personal experiences with such attacks. Using thematic analysis, we typified attacks around five motivations (fun, curiosity, jealousy, animosity, and utility), and explored dimensions associated with each type. Our combined findings indicate that social insider attacks are common, often have serious emotional consequences, and have no simple mitigation

Wali Usmani, Diogo Marques, Ivan Beschastnikh, Konstantin Beznosov, Tiago Guerreiro, Luís Carriço

CHI 2017 ‑ In the ACM Conference on Human Factors in Computing Systems, Denver, Colorado, USA, May

Average Six WPM After 12 Weeks. Figure  shows participants’ input speed over 12 weeks. Overall, the average input speed in the real world improved from week 1 (M = 3.2 SD = 0.8 WPM) to week 12 (M = 5.9 SD = 0.2 WPM). As in the laboratory, with all participants improving typing speed over time. Still, learning rates were lower in real-world data with an improvement of 0.2 WPM per week. Everyday Typing is Faster than Laboratory. In Figure, we notice that everyday typing speed is consistently higher than laboratory results. The difference in performance between real-world and laboratory is 1.6 WPM and 1.4 WPM in week 1 and week 8, respectively.

Investigating Laboratory and Everyday Typing Performance of Blind Users

For 12 weeks, we collected field data, coupled with eight weekly laboratory sessions. This article provides a thorough analysis of everyday typing data and its relationship with controlled laboratory assessments.

Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues Vicki L. Hanson

TACCESS 2017 ‑ ACM Transactions on Accessible Computing (TACCESS) - Special Issue (Part 2) of Papers from ASSETS 2015

{publication.award}

Four keyboard sizes side-by-side. With the Large being about the same size as the other three, with keys of 15mm. With Medium being bigger than the size of the remaining two, with keys of 10 mm. Small with keys of 5mm. Tiny with keys of size 2.5mm.

Effect of target size on non-visual text-entry

We investigate how nonvisual input performance, on touchscreens, varies with four QWERTY keyboard sizes (ranging from 15mm to 2.5mm). This paper presents an analysis of typing performance and touch behaviors discussing its implications.

André Rodrigues, Hugo Nicolau, Kyle Montague, Luís Carriço, Tiago Guerreiro

MobileHCI 2016 ‑ ‘16 Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services, Pages 47-52, Florence, Italy — September 06 - 09, 2016

Honorable mention

{publication.award}

two charts showing the predicted clear likelihood of snooping according to age and depth of adoption

Snooping on Mobile Phones: Prevalence and Trends

Through an anonymity-preserving survey experiment, we quantify the pervasiveness of snooping attacks, defined as looking through someone else’s phone without their permission. We estimated the 1-year prevalence to be 31% in an online participant pool. Weighted to the U.S. population, the data indicates that 1 in 5 adults snooped on at least one other person’s phone, just in the year before the survey was conducted.

Diogo Marques, Ildar Muslukhov, Tiago Guerreiro, Konstantin Beznosov, Luís Carriço

SOUPS 2016 ‑ In the Twelfth Symposium on Usable Privacy and Security, USA, June, 2016

Distinguished paper award

{publication.award}

A blind user exploring the table top with one finger on each hand. The tabletop has a frame above holding a Kinect sensor that is detecting the user's fingers.

Blind People Interacting with Large Touch Surfaces: Strategies for One-handed and Two-handed Exploration

We report on a user study with 14 blind participants performing common touchscreen interactions using one and two-hand exploration. We investigate the exploration strategies applied by blind users when interacting with a tabletop. We identified six basic strategies that were commonly adopted and should be considered in future designs. We finish with implications for the design of accessible large touch interfaces.

Tiago Guerreiro, Kyle Montague, João Guerreiro, Rafael Nunes, Hugo Nicolau, Daniel Gonçalves

ITS 2015 ‑ ACM International Conference on Interactive Tabletops and Surfaces, Madeira, Portugal, November, 2015

Best Paper Award

A keyboard scheme with dots representing each of the collected touch points. Each key as dots from a different color. There is a concentration of dots on the most used keys (e.g. a, s and space).

Typing Performance of Blind Users: An Analysis of Touch Behaviors, Learning Effect, and In-Situ Usage

For eight weeks, we collected in-situ usage data and conducted weekly laboratory assessment sessions. This paper presents a thorough analysis of typing performance that goes beyond traditional aggregated measures of text-entry and reports on character-level errors and touch measures. Our findings show that users improve over time, even though it is at a slow rate (0.3 WPM per week).

Hugo Nicolau, Kyle Montague, André Rodrigues, Tiago Guerreiro, Vicki Hanson

ASSETS 2015 ‑ 17th International ACM SIGACCESS Conference on Computers and Accessibility. Lisboa, Portugal, October, 2015

A blind person interacting with a smartphone using headphones.

Getting Smartphones to Talkback: Understanding the Smartphone Adoption Process of Blind Users

We conducted a twelve week in-the-wild longitudinal study with five novice blind users to understand the adoption process of smartphones. We characterized their concerns, barriers, support mechanisms and evolution throughout the eight week period.

André Rodrigues, Kyle Montague, Hugo Nicolau, Tiago Guerreiro

ASSETS 2015 ‑ 17th International ACM SIGACCESS Conference on Computers and Accessibility. Lisboa, Portugal, October, 2015

HoliBraille, a smartphone case with six multi-point vibrotactile output. (a) Representation of ‘f’ using the Braille code: dots 1, 2, and 4. (b) The system outputs character ‘f’ through direct and localized feedback on the user’s fingers. (c) The system consists of six vibrotactile motors attached to springs and a 3D-printed case. The springs mold to users’ hands and dampen vibrations through the device, preventing propagation between fingers and allowing better stimuli discrimination.

HoliBraille: multipoint vibrotactile feedback on mobile devices

HoliBraille is a system capable of localized vibrotactile feedback that can be combined with the input capabilities of mobile devices. We used a custom-made case and off-the-shelf vibrotactile actuators combined with dampening materials. The solution can be attached to mainstream touchscreen devices enabling direct feedback on users’ fingers. In this paper, we contribute the following: 1) some application scenarios that can benefit from HoliBraille; 2) the design and technical description of the proposed device; and 3) an evaluation of HoliBraille on a foundational task for future Braille-related applications, i.e. character discrimination.# short description of the publication

Hugo Nicolau, Kyle Montague, Tiago Guerreiro, André Rodrigues and Vicki L Hanson

W4A 2015 ‑ In Proceedings of the 12th Web for All Conference (W4A ‘15). ACM, New York, NY, USA, Article 30, 4 pages.

The QWERTY keyboard an the characters spatial position in the 3d audio space. Characters are given an audio spatial position accordingly to their location on the keyboard. They are grouped according to the vertical columns of the keyboard (e.g. Q and A; then W, S and Z), resulting in 10 different spatial locations separated by 20º. For example A is heard on the far left (180º) while N is heard more on the right side (60º) allowing for simultaneous speech signals.

TabLets get physical: non-visual text entry on tablet devices

Our system combines spatial and simultaneous audio feedback with multitouch selection techniques to mimic traditional two-hand keyboard interaction. SpatialTouch enables blind users to rest their idle hand on a key (e.g. F or J), while simultaneously exploring the keyboard with their active hand and receiving auditory feedback about the character location.

João Guerreiro, André Rodrigues, Kyle Montague, Tiago Guerreiro, Hugo Nicolau, Daniel Gonçalves

CHI 2015 ‑ ACM Conference on Human Factors in Computing Systems, Seoul, South Korea, April, 2015

{publication.award}

B# is a novel correction system for multitouch Braille input. (a) The user types the letter F holding the device with the screen facing forward using braille typing. (b) Character-level correction; the closest characters in terms of Braille distance for 2 unidentified chords. Example given with the O character, with closest characters, O,S,R,T (c) Word-level correction; top suggestions return by B# considering the letters that are at a Braille distance of one from the entered chord.

B#: chord-based correction for multitouch braille input

B#, a novel correction system for multitouch Braille input that uses chords as the atomic unit of information rather than characters. Experimental results on data collected from 11 blind people revealed that B# is effective in correcting errors at character-level, thus providing opportunities for instant corrections of unrecognized chords; and at word-level, where it outperforms a popular spellchecker by providing correct suggestions for 72% of incorrect words (against 38%).

Hugo Nicolau, Kyle Montague, Tiago Guerreiro, João Guerreiro and Vicki L. Hanson

CHI 2014 ‑ In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ‘14). ACM, New York, NY, USA, 1705-1708.

Best paper award

The UbiBraille prototype consists of six rings augmented with vibrotactile capabilities. The rings are worn on the index, middle, and ring fingers of both hands.

UbiBraille: designing and evaluating a vibrotactile Braille-reading device

UbiBraille consists of six vibrotactile actuators that are used to code a Braille cell and communicate single characters. The device is able to simultaneously actuate the users’ index, middle, and ring fingers of both hands, providing fast and mnemonic output. We conducted two user studies on UbiBraille to assess both character and word reading performance. Character recognition rates ranged from 54% to 100% and were highly character- and user-dependent.

Hugo Nicolau, João Guerreiro, Tiago Guerreiro and Luis Carríço

ASSETS 2013 ‑ In Proceedings of the 15th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ‘13). ACM, New York, NY, USA, , Article 23 , 8 pages.

Two digital puzzles side by side; one is a visual version of the Android logo, the other is a audio puzzle with several pieces already in the right order and some in the incorrect ones

Audio-Based Puzzle Gaming for Blind People

We present audio puzzle games, an interactive way of playing with music by rebuilding a song, just like the original image jigsaw puzzle, broken into pieces which are by turn randomly shuffled. The audio mode design followed a participatory approach starting from a version similar to the visual one, but around music instead of images. We report on an evaluation with 13 blind people using the proposed game application with three different songs randomly distributed.

Jaime Carvalho, Luís Duarte, Tiago Guerreiro, Luís Carriço

MOBACC 2012 ‑ Mobile Accessibility Workshop at MobileHCI 2012

{publication.award}

a user inputting text by split-tapping on a MultiTap layout virtual keyboard

Blind People and Mobile Touch-based Text-Entry: Acknowledging the Need for Different Flavors

We present a study with 13 blind people consisting of a touch screen text-entry task with four different methods. Results show that different abilities (spatial ability, tactile perception, …) have significant impact on performance and that this impact is related with the different methods’ demands. These variations acknowledge the need of accounting for individual characteristics and giving space for difference, towards inclusive design.

João Oliveira, Tiago Guerreiro, Hugo Nicolau, Daniel Gonçalves, Joaquim Jorge

ASSETS 2011 ‑ In The 13th International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ‘11). ACM, New York, NY, USA, 179-186.

Best Student Paper Award

BrailleType main screen on the left (visual representation of the six target zones was added for illustration). Middle screen shows the letter ‘r’ marked and ready to be accepted. The image on the right shows a user writing the letter ‘r’ with BrailleType.

BrailleType: Unleashing Braille over Touch Screen Mobile Phones

We present BrailleType, a single-touch text-entry system for touch screen devices. BrailleType allows the blind user to enter text as if he was writing Braille using the traditional 6-dot matrix code. We performed a user study with fifteen blind subjects, to assess this method’s performance against Apple’s VoiceOver approach. BrailleType although slower, was significantly easier and less error prone

João Oliveira, Tiago Guerreiro, Hugo Nicolau, Joaquim Jorge, and Daniel Gonçalves

INTERACT 2011 ‑ INTERACT 2011: Human-Computer Interaction – INTERACT 2011 pp 100-107