Intro
Machine learning helps computers identify patterns in data, influencing numerous experiences and products today — from everyday Netflix recommendations to suspicious login detections, shopping suggestions, and upcoming self-driving taxis. Machine learning is a subset of artificial intelligence that enables a system to autonomously learn and improve without being explicitly programmed, by feeding it large amounts of data.
I am not an expert in machine learning itself, yet I’ve been able to contribute to creating new experiences enabled by ML. In the last few years, as a UX designer, I’ve had the opportunity to work on cutting-edge products utilizing machine learning (ML) across three diverse fields: self-driving vehicles, healthcare imaging, and cybersecurity. These experiences have provided valuable insights into designing effective ML-powered products. First, I’ll share an overview of my work and then discuss the key lessons learned along the way.
Projects
1. Self-Driving Vehicles (Uber ATG)
At Uber’s Advanced Technologies Group, I designed the simulation tools for scientists and engineers developing autonomous vehicle technology and processes for the operation team to operate and manage vehicle fleets on the street daily, including both hardware and software sides. The ML systems in this context processed enormous amounts of visual and spatial data in real-time, making critical decisions about vehicle motion, obstacle avoidance, and navigation.
One key realization was the crucial importance of user research, even in a highly technical, low-user-volume environment. We needed to deeply understand how our scientists and operators interacted with technology on their respective part of the development and ultimately improve the safety and efficiency of the self-driving technology.
Uber ATG was acquired by Aurora in 2020 Dec.
2. Healthcare Imaging (Verily)
At Verily, I was part of a multidisciplinary team developing an AI-powered eye screening service. This project aimed to address the shortage of eye doctors in certain regions by using ML to analyze retinal images and identify potential health issues.
Along with the team I designed the entire service journey, from the patient’s initial interaction with the screening system to the delivery of results and follow-up care. This included creating digital interfaces for capturing retina images, designing workflows for healthcare providers, result reporting for the patients, and considering how the AI-driven diagnoses would integrate into existing healthcare systems.
A critical lesson from this project was that introducing ML into healthcare isn’t just about the technology — it requires careful consideration of the entire ecosystem, including patient education, healthcare worker training, and integration with existing medical practices.
The team working on this effort on the Verily side was moved to Google Health in 2020.
3. Cybersecurity (Lacework Edge)
At Lacework Edge, I designed an enterprise cybersecurity product that leveraged ML to detect and respond to potential security threats. The system analyzed users’ internet behavior patterns to establish a baseline of normal activity, then used this information to identify anomalies that could indicate a compromised account or data breach attempt. It’s similar to how tech giants like Google and Amazon confirm your identity when logging in from a new device or location and our product enables similar security measures for small and midsize companies.
My work was to lead the designing of the whole service, including two user groups: The Admin Group, which aims to establish optimal processes to provide and secure access, ultimately mitigating risks for their organization. And the employees group, which is composed of individuals who are diverse in roles and backgrounds and aim to get their work done with minimal interruption.
A key challenge was balancing detailed complex information to tell the full story with the need for simple and actionable insights that users can quickly take action on. It was also critical to build trust with users on the insights that our ML was generating.
https://www.lacework.com/platform/edge
Lacework was acquired by Fortinet in 2024 Aug.
6 Key Lessons
1. Define the problem clearly
In all three projects, it was crucial to start by clearly identifying user needs and the specific problem we were trying to solve. In healthcare, for instance, we initially thought ML would solve the doctor shortage. We quickly realized that the problem was more complex, involving patient education, healthcare worker training, and integration with existing systems.
2. Design the overall experience first
Before diving into the ML-specific aspects, it’s important to sketch out the entire solution, including all steps in the user journey and all relevant users or actors involved.
For developing an AI-powered eye screening service, this meant mapping out not just the AI diagnosing the retinal image, but also the entire journey of the service, including various stages and all the people involved. This included people conducting the process, giving instructions, following instructions, paying for the service, receiving information, doctors reviewing reports, patients receiving results, and those discussing next steps with patients. To help identify the whole journey, we created a comprehensive map using a large whiteboard at the project’s outset. This type of map helps the team identify key challenges and how to allocate time and resources effectively. One sketchy example image is below.
For this part, I didn’t know what information to include, so I invited experts from the team who have worked in the relevant industry — such as an in-house ophthalmologist, my manager who has worked on similar projects, or a UX researcher who has conducted field research in this area. Designers are experts in providing methods and processes that lead the team to create solutions, but we are not domain experts.
3. Involve ML specialists early
Including data scientists and ML experts in the design process from the beginning is crucial. In the self-driving vehicle project, regular collaboration with ML engineers helped ensure that the interfaces we designed accurately represented the capabilities and limitations of the rapidly evolving technologies. At Edge, I included the ML lead in the design sprints and invited him to design reviews. At one design sprint, we aligned on the user goals at the first session. Later we discussed what and how we would change in the current product. The data scientist lead who is responsible for all data and ML models said something like “let’s focus on what’s most useful to the users ~~ the current grouping of information seems hodgepodge~~”. That was music to my ears. We were aligned to deliver a good experience to the users.
4. Plan for ML limitations, design for feedback loop
It’s critical to design for cases where ML may not be helpful or make mistakes. In healthcare, this meant creating clear protocols for when the AI system’s diagnosis needed human verification and factoring this into designing the end to end journey. For cybersecurity, we tried to design features allowing users to provide feedback on false positives, helping to improve the system over time. For example we had an idea to detect all new emerging GenAI sites and communicate potential risk factors. Users could mark if the detected sites were right or wrong. It was simple and we thought this could be designed further to collect more useful feedback for the ML.
One widespread example of this is in Google ads’ “helpful” or “not helpful”. That is one way users can give their feedback and, I envision we will see more articulated feedback loops in various experiences in the near future. Designing a feedback loop that’s helpful for both users and ML algorithms is a huge area. Based on how the product is using the ML, what feedback and how to collect it varies.
5. Communicate ML factors to users
In order to build trust in a product, first users need to get to know it and have some comfortability with how it works. Explaining, even at a high level, the factors an ML system uses to make decisions helps build this trust. At Edge, we built a cyber security product aimed to help companies to defend against evolving threats with risk and behavior-based security built for today — where everything is on cloud and people are working from home, office, and everywhere. One part of the solution was to show which users are at high risk or medium risk, something that our ML algorithms were calculating in real time. At first users said that what they see is helpful, but they wanted to know how that information was obtained. We provided high-level explanations of the behavioral factors (e.g. MFA disabled, Device posture, abnormal location, etc) used to determine risk in the product, which users found very helpful in understanding and trusting the system’s assessments.
6. Ask questions throughout the process to your team
Designing ML-powered products often involves working with complex, cutting-edge technologies. Don’t hesitate to seek clarification on technical aspects or process questions. We are experts in the process called UX design, and when we can understand the goals and technical capabilities and limitations, we can shine with our trained process. In my experience, this open communication not only improved my understanding but often led to valuable discussions that benefited the entire team.
Conclusion
While ML and AI offer powerful capabilities, they are just one part of a solution. Successful ML-powered products require intentionally designed user experiences that truly meet human needs.
As the field of AI and ML continues to evolve rapidly, so too will the best practices for designing these systems. I believe staying curious, asking questions, defining the problem, and continuing to put the user first will be key to creating successful ML-powered products in the future.
I hope these lessons provide some insights for those interested in creating experiences with AI. I look forward to applying these findings in my future projects and continuing to discover new lessons in designing experiences in the era of AI.
