In January 2023, we launched a new feedback method for the Universal Prompt within our internal Cisco team, comprised of more than 70,000 employees. By February, this feature extended to the public in a private preview. The new approach incorporates an interactive survey integrated within the authentication process, while also addressing important factors such as security, accessibility, and brand alignment.
As the project lead, I took ownership of and actively executed the research, conceptualization, user tests, prototyping, asset creation, and implementation planning.
2 designers, 2 researchers, 2 engineers, 4 data scientists
3 months
May - Aug 2022
In early 2022, Duo Security overhauled their authentication system with the introduction of the Universal Prompt. To collect user insights regarding this update, they initially used a Google form; however, its perceived lack of professionalism raised concerns.
In the early stages of our UX project, our initial problem statement was centered around how we might keep a pulse on our users and collect their feedback for Universal Prompt in a professional, sustainable way. However, as we delved deeper into the project and began our research and ideation phases, it became evident that I needed a more specific focus to drive our design efforts effectively.
Key findings from an analysis of the 9,000+ responses from the initial Google form show that
Multi-point scales might not truly reflect people's opinions. 📊🤔
Users reported struggling to recall the previous design accurately for effective comparison. In addition, a "4" from one user might carry the same weight as a "5" for another due to differing perspectives. This variability makes setting a clear threshold for valid feedback challenging.
Users who find the survey lengthy tend to skip it or give minimal input. 😪
The open-ended responses were mostly brief and lacked specificity, with over 75% being left blank. Our users include students, professionals, and busy parents. They aim to sign in fast and reach their application. If the survey appears demanding in terms of time or complexity, they tend to skip it or provide minimal input.
Negative feedback is causing significant data noise. 😵
Since Duo is an institutional platform, we anticipate a higher frequency of user discontent. This results in a predominant negative tone in the majority of responses, accompanied by long narratives or profanity. Consequently, a significant amount of noise is present in the feedback, posing challenges in effectively analyzing the data.
Additional prompts post-authentication can lead to confusion and frustration. 😡
Users uninterested in the survey found the "continue to application" button, meant as a prompt, confusing and frustrating due to the added step. As one user stated, "Why do I have to click 'continue to application'?? Can't I just be redirected after I approve the push notification? What is the need for the extra click?"
How can we collect accurate and actionable feedback in the least disruptive way possible?
I initiated discussions with engineers, data scientists, and experts in various design roles to fully understand the technical requirements, security guidelines, and brand alignment. Here were a few considerations that had to be made:
Confined within the boundaries of the authentication modal.
Because of technical and accessibility needs, the form can not utilize pop-ups and must only be accessible within the authentication modal.
Time-constrained authentications restrict interaction duration.
For security purposes, the authentication session is active for a set period to ensure protection.
.png)
Designs must be consistent with Duo’s brand identity.
Every asset should adhere to the brand's visual identity, tone of communication, and accessibility standards.
After three rounds of testing and conducting over 50 usability sessions, we learned:
Survey Frequency Preference:
Users are receptive to completing short surveys as long as they are infrequent, with a majority indicating a preference for no more than once a month.
Familiarity and Modern Look:
Familiarity and a modern design aesthetic are positively correlated with a higher perceived ease of use among users.
Big Buttons Impact:
Users are inclined to interact with large buttons, perceiving them as important and easy to engage with, which enhances their user experience.
Clarity in UX Copy:
Confusion regarding selections or prompts can increase cognitive load, leading to diverse user responses for the same scenarios.
A quick and easy 1-click survey that takes 2 minutes at most.
Memory limitations undermine the reliability of user feedback.
Survey simplicity: Employed a single, concise, and broad survey question to reduce cognitive load on users.
Survey completion time has been significantly reduced to ~2 seconds, earning unanimous 10/10 ratings for quick, effortless user experience.
Inconsistent and varied ratings complicate feedback thresholds.
Clarity with descriptions: Provided clarifying descriptions for improved user understanding.
Streamlined survey structure: Simplified the survey with two clear options, solely focused on assessing perception of significant authentication issues.
Users understand the question and options clearly leading to consistency in task completion.
Users who find the survey lengthy tend to skip it or give minimal input.
Minimalist approach: Utilized a single question for users without issues.
Enhanced engagement: Improved survey appeal with large interactive buttons.
User guidance: Utilized progressive disclosure to guide users when additional input is needed.
Achieved a 96% survey completion rate among 50 participants, indicating improved engagement, while also reducing the perceived demand of the survey.
Negative feedback is causing significant data noise.
Category organization: Introduced predefined categories for streamlined feedback categorization.
Focused open-ended questions: Utilized straightforward, targeted free-response questions.
Response conciseness: Imposed word limits to encourage concise responses.
Easier to separate relevant data from the noise.
Additional prompts post-authentication can lead to confusion and frustration.
Smooth transition: Placed the survey after the authentication success screen to maintain familiarity for smoother transitions.
Minimal disruption: Introduced a concise survey with minimal impact, adding just one extra screen.
User flexibility: Offered users the option to skip the survey and return back from the form for flexibility.
Feedback frequency control: Implemented a tracking cookie to limit survey frequency within the past 2 months.
Randomized participation: Randomly selected less than 1% of authentications for survey participation.
A 100% satisfaction rate was observed, with unanimous approval from all 50 usability test participants, who found the survey's placement in the flow to be just right and in line with their expectations.
”This is a very standard box. I think we're all used to seeing these pop-ups after doing certain services online. So, nothing difficult or confusing.”
I designed unique interactive buttons, a novel addition to Duo's design system in order to distinguish their functionality from similar components.
.png)
The final design features 4 new success screens.
.png)
To minimize disruptions in the user authentication flow, we display surveys to just a random 1% of total authentications. Additionally, a cookie tracks the user's last survey date, ensuring they won't see the survey again if they've taken one in the past 2 months.
The prototype underwent multiple significant revisions.
We reduced sentiments from 3 to 2 to focus on good and bad experiences, simplifying the interaction.
This decision removes the middle sentiment and survey about positive experiences, reducing survey completion time. The survey is now exclusively presented to users facing issues, aligning with the identified user journey and mental model from our research. This modification aims to enhance the overall user experience by minimizing unnecessary steps while addressing specific pain points.
Measurable impact was assessed through a conclusive final usability test involving 10 participants.
High User Adoption
100% survey completion rate in final usability tests, with all 10 participants rating the authentication and survey experience as "very easy."
Improved Efficiency
Reduced survey completion time by 99.6%, from 5 minutes to under 2 seconds.
Effective Scaling
Successfully deployed designs to 70,000+ employees and initiated a private preview, showcasing scalability and impactful user engagement.
"Lily shows up everyday with a desire to learn. She proactively seeks knowledge (e.g., psychology articles on measuring perceived "speed" in surveys) both inside and outside of Duo to refine her designs. She solicits feedback early and often, and goes out of her way to seek multiple perspective on her work (see: #team-authfeedback)." - Design advisor and manager, Sierre Wolfkostin
In this project, I discovered that Duo users may not favor multi-factor authentication. Interestingly, this sentiment plays an important role in their user personas, guiding our focus on enhancing the user-friendliness of the design.
I am also grateful for the opportunity to learn and apply design skills in security, accessibility, and copywriting while collaborating with experts on the team.
While I wasn't able to complete the design for the admin dashboard of the survey feature, given more time, I would have liked to include functionality that allows administrators to deactivate surveys for their members.
.png)
.png)