Simplifying planning experience for users control self-driving robots in extreme environments
UX Design
Zero-to-One
Toolbar Design
UI Component Design
Usability Testing
Prototype
Autonomous Driving
AI Experience Design
Overview
Role
UX Designer
UI Designer
UX Researcher
Duration
2 months
Team
2 UX/UI Designer
1 CTO/Program Lead
6 Software Engineers
2 Hardware Engineers
1 Test Engineer
Introduction
Surveyor is an AI-driven autonomous driving mining robot, which can replace humans to work in the extreme environment. Users will use Surveyor sense the geologic land to determine mining eligibility by gathering and analyzing data.
Project Goals
Mining operators need to measure the area after every blasting. Planning is an important feature that provides a productive survey experience by allowing users to plan a travel route quickly and easily, even on-site.
Blasting in a mine site
Outcome Highlight
  1. Crafted and delivered the first HMI design system and robotic interface design from scratch in OffWorld.
  2. Led and hosted workshops and usability testing in the team involving executive-levels and program leads within a cross-functional team.
  3. Defined the priorities of the products and drove alignment with key stakeholders to make decisions.
Planning Tools Component Design
Problem statement
Users struggled with the complicated process of creating a survey plan from scratch on a touch screen.
Challenges
  1. Challenging EnvironmentUsers work in extreme heat, dust, intense sunlight, and heavy personal protective equipment (PPE).
  2. Effectiveness-DrivenUsers need to generate a plan every time after a blasting. (Time-consuming: 0.5-1 day by manually, 1.5 - 4 hr by auto).
  3. Real-Time AdjustmentUsers may need to modify the plan based on real-time geologic conditions.
The photo shows the engineer testing and driving the robot
Run a cross-functional workshop to define user journey and insights
To dive deeper, I hosted a brainstorming workshop with CFOs, engineers, operators, and mining experts to learn about the workflow, red flags, and tech constraints. I analyzed the transcripts and insights and clarified typical user workflows.
The photo displays the participant was answering questions at the workshop
1. Understood how the survey plan works.
I learned that creating a Survey Plan requires three parameter values: one boundary, user survey points (must pass through), and a travel path.
The photo shows the three parameters of a Survey Plan
2. Defined the high-level design goal.
To provide an intuitive planning experience, the HMI system should offer effective tools that enable users to easily create survey plans on a touch screen within a tough environment.
3. Proposed a new workflow.
In a start-up without a Product Manager, the team struggled to align on HMI product features. I engaged the CFO and program leads to reframe the MVP features and drove alignment by compiling the Product Requirement Document (PRD) for the MVP.
Concept development
1. Led the design process to developed the 1st tool concept through the low-fidelity wireframes.
2. To reframe the design concept, I did a usability testing to clarify the pain points.
Test goals:
  1. Understood how the jobs were to be done in this workflow.
  2. Learned about how the buttons, icons, and labels impact the participants’ decision-making.
  3. Asked questions to learn about the participants’ expectations for each step.
  4. Observed what the obstacles would impact users to take actions
During the usability testing, I observed the most of the participants struggled to create a boundary by adding points. After the test, I created a journey map for the ‘creating a boundary’ workflow to understand the participants’ thoughts and ideated a new solution.
Summarized the findings:
  1. The side sheet doesn’t show a clear work flows to users.
  2. It needs too much work to create a boundary by adding points.
  3. It needs too many steps to complete in one task.
3. Redefined the design strategies.
After the test, we realized the side sheet bar couldn’t meet users' needs. So we set up the following strategies:
  1. Simplify or cancel the side sheet.
  2. Provide clear workflow by keeping users focused on current parameters.
  3. Make the tools easy-to-use.
Redefined design concept
To enhance the planning experience, I proposed using floating windows instead of side sheets and provided two design concepts:
  1. Display a single floating window with arrows to toggle parameters.
  2. Display three floating windows that expand with tools to set up parameters.
I discussed these ideas with the small team, synthesized the pros and cons, and the team agreed on Concept 2.
Concept 1
Floating Window with toggling buttons
Toggle on the arrows to focus on one parameter.
The other two parameters have been hidden. Not friendly for new users.
Concept 2
Floating windows with dropdown drawers
Three parameters will be presented at the same time.
There are fewer touchpoints and a much clearer design for users.
Concept 2 High-fidelity prototype
The Planning Final Design
1. Improved workflow by allowing users to load existing survey plans or a plan template.
After finishing the components design, the team thought the work flows could be simplified to uploading a Survey Plan template with parameters for the MVP.

Moreover, in a real case, the users might change the plan, which might be caused by urgent issues, tough terrain conditions, etc. So, we enhanced the user flows and design goals.
Created the prototype for the loading process
2. A boundary-creating process.
The Boundary button provides geologic shapes to create a frame on the map. And the users could adjust the frame shape through the points by dragging.
3. Adding user survey points.
The User Survey Points could be added using coordinates by selecting points on the map by pressing.
4. For an accurate operation, I adjusted the number input field box.
Addressing the user needs about the accuracy about one point (latitude, longitude), I found out to make under 1.11 meters (3 ft 8 in) precision, the number should be precise to six decimal places.
So that, I did adjust:
  1. To fit more numbers, it needs a longer text box.
  2. To reduce users’ anxiety, a bigger window is needed to guide users in inputting the numbers.
5. Generating Waypoints
The Waypoints tool allows users to customize waypoints by choosing the shape, spacing size, and step size.
6. Addressed the accuracy of the labels through setting up a survey.
During the design process, the team had different opinions about the names of labels. To address the accuracy of the labels, I created a survey within the engineers’ team, and confirmed User Survey Points and Waypoints instead of Survey Points and Patterns.
I did a survey inside of the developing team. I removed the labels from the interfaces and asked the questions like:
  1. What names best describe the icons and sections highlighted in the map?
  2. Why do you think this is a proper name for the users?
Summary
Established OffWorld’s First Design System
Developed the initial design system for both tablet and desktop interfaces.

1st

Design System

Supported Client Demo
Implemented the design and supported the team in successfully completing a one-month demo at a client’s mine site.

1mon

Demo

Enhanced User Satisfaction
Conducted usability testing and iterated designs, leading to an 80% increase in user satisfaction.

80%

User Satisfaction

Led the design process
Led the design process, facilitated discussions to prioritize MVP features, and compiled the Product Requirement Document (PRD).
Facilitated Usability Testing
Educated the team on usability testing practices, positively influencing decision-making.
“The most scientistic design method I’ve never seen. ”
---- Engineer of electrons