How can an established public service innovate?

- Immigration, Refugees, and Citizenship Canada

Overview

The refugee settlement sector is a

Sectors

Immigration and refugee settlement, Social services

Team

Janet Webber, Seth Erais, Maria Dela Cruz

Methods & Tools

Stakeholder interviews, White boarding, Focus Groups, Thematic Analysis, NVivo, Facilitation

Deliverables

Final report, Policy recommendations, Process design

Background

The refugee and newcomer settlement sector is a historic and important

Securely sharing Genetics?

To address this issue, Molecular You began developing a platform called MY Personal Data Exchange (MYPDx) in partnership with IBM, Roche Canada, and Microsoft. Utilizing IBM’s private blockchain framework and a self-sovereign identity architecture, MYPDx is a DApp that enables users of the MYHealth platform to securely share their stored biomarker information with biomedical researchers. Users of MYPDx can gain rewards for supporting research, verify that studies have ethical oversight, and remove their information from studies at any time. I was hired with a team of UX Designers to conduct user testing for a new prototype of MYPDx. The scope of our work was to improve the initial prototype, conducting the first iteration of the UI and UX for MYPDx.

Defining the Problem Space

We began conducting various desk research methods of the current prototype including heuristic evaluations and stakeholder interviews. This included interviews with management, developers, and pharmaceutical researchers at Roche Canada. From these investigations we began to understand that because MYPDx relies on a novel architecture, and technology, a key focus for our team was sense-making: how could we better support users to quickly understand and navigate this system? In addition, the system required users to download a blockchain wallet, and create encrypted connections by scanning QR codes on their screen with their phone. How could we help users with different levels of digital literacy quickly learn this new interaction? Our team began to build up a protocol for a round of usability testing based on these questions.

But after talking with users and researchers in the pharmaceutical space, I found myself drawn to a more fundamental problem. Namely:

How does the design of systems influence how users trust them? And what design elements influence their sense of trust (if any)? To answer these questions I proposed my own independent user research project exploring the relationship between user engagement and user trust.

MYPDx relied on the security of blockchain technology as a guarantee of users’ security. Within the blockchain space, it’s assumed that the cryptography and decentralization of the technology are technical measures that enable trust. But most regular people don’t understand these measures. In fact, most users simply don’t understand how blockchain works. How then would MYPDx get users to trust it? This seemed to me a fundamental question that touched both the user desirability and business viability of even creating such a system: if users couldn’t understand what advantages MYPDx was delivering, they wouldn’t see it as secure, and might not want to user it in the first place. I wrote up a one pager, outlining this problem, and brought it to my Supervisor to pitch an additional research project. This additional research, I argued, could be completed during the proposed usability test, would greatly advance the prototype, and help validate the business case for MYPDx. My supervisor agreed.

Aligning Scope

I started by aligning my scope with my team. I would work to create implications for design, with a specific product requirement document for the MYPDx team for their goal of a new version of the prototype. I would also create a theoretical framework, which would inform a future research paper. I worked with Molecular You’s product lead to design a usability testing protocol that met the needs of the product team and my own research protocols. We created a plan where the usability testing plan my team created together became a way of giving users their first exposure to MYPDx. This enabled the testing to meet the needs of the MYPDx product team and my own research goals.

Mixed Methods

Because I was trying to understand an interaction between two abstract and used a convergent mixed methods design to understand the relationship between user trust and user engagement, and to explore what aspects of the design of MYPDx had the biggest effects on trust for the next prototype. Quantitative analysis would the existence of a relationship between how users’ engaged with MYPDx and whether they trusted it. Qualitative analysis was used to understand more deeply, and develop theoretical insights and implications for design.

I was responsible for recruiting participants, setting up and conducting the usability tests with users (n=20). This research was conducted during the early days of the 2020 COVID-19 Pandemic, and so needed to be conducted completely remotely. During the first part of the session users were given tasks to complete using the MYPDx prototype, and utilized a think-aloud protocol. Through these tasks, users gained familiarity with MYPDx, Then once the users had completed their tasks, I conducted semi-structured interviews, exploring relevant moments from the walk through in more detail. This formed the bulk of my qualitative data, which was analyzed using multiple rounds of open and axial coding. Finally I administered a survey instrument made up of established measurements of user engagement and trust in technologies. This formed the basis of my quantitative data which was analyzed used descriptive and inferential statistics.

Findings - How to Design a Trustworthy System

Using statistical analysis I found a strong positive correlation: the more users were engaged by a system, the more likely they were to trust it. Specifically, the perceived usability of MYPDx was found to be most strongly correlated to whether users felt the system was trustworthy.

I noticed that users seemed to make their own assessments of trustworthiness based on aspects of the UI/UX of the system, rather than on the use of blockchain technology to ensure security. For example, users were asked to use their phones to scan QR codes to access certain features. From the back-end of this system, these QR codes created secure links between a users blockchain wallet and the ledger. However, users perceived the QR codes instead as more of a visual metaphor in their mental model, with one user saying: “It’s like a wall that you have to go through that only you have the key for.”

Users seemed to make a one-time assessment of trustworthiness, making a judgment call after which they either trusted or distrusted the system. This assessment of trust was fundamentally connected to risk associated with using MYPDx, which changed based on users experiences with cybersecurity, their assessments of who they were sharing the information with, and assessment of whether rewards given to users outweighed potential risks.

Most importantly, I noticed that users were engaging in an explicit process of learning about MYPDx as they interacted with it, both to update their mental model, and to asses risk. Users specifically looked for interactions, like feedback, friction, and interactivity.

Implications for Designing Trustworthy Systems

I used the findings to develop design implications, a common practice within Human Computer Interaction Research to guide future studies and design:

  • Users’ conceptual model of the entire system is essential to trust. Support it’s growth.

  • Balance information asymmetries. Users are at a disadvantage using blockchain systems, help them learn, and trust a system.

  • Focus on Information Architecture (IA). IA helps users make a system intelligible, which is needed for trust.

  • Offer rewarding experiences. Rewards help users balance perceived risks and choose to trust a new system.

  • Designing for engagement may support trust. Specifically, users look for how helpful and usable a system is when deciding to trust it.

  • Feedback is an essential interaction for learning. Support learning through integrating feedback throughout the system.

  • Design for a system that is helpful, accessible, and clear. Confusion makes users less likely to trust a new system.

  • Give users assurances about the motivations of different users, such as ethics certifications, funding sources, and other information.

Impacts

The design implications above were presented to the MYPDx team, alongside the findings from the usability testing I completed, as a product requirements document. These pushed the product team to focus on the Information Architecture of the prototype, and integrate new interactions with more feedback. The current MYPDx prototype is still in development by Molecular You, Roche Canada, and other partners.

I also had a chance to present the findings of my research to our colleagues at the BC Government’s Digital Identity team. The strategic presentation focused on how the design implications could be used to inform their work on the prototype digital identity wallet DApp they were building. This has since been launched as BC Wallet, and is usable by all of the provinces’ 5 million residents to store their personal information.

Finally, the results of this research were written up as a research paper and published by Springer Nature in an edited volume highlighting cutting edge research in healthcare-based blockchain applications. This was my first academic publication.

Previous
Previous

Service Blueprinting