About Product Y:

Product Y is a Shopify app that helps to educate consumers about the hair products on a merchant’s website. Shoppers can interact with Product Y directly on the merchant’s product page. After a quick conversation, they receive a match score which informs them of how well the merchant’s product(s) is predicted to perform on their hair. Also within the app, is an admin panel that Shopify merchants use to view app activity and manage Product Y on their site.

Essentially, Product Y provides the experience of the Product X mobile app on e-commerce platforms.

Background:

After designing the first iteration of the admin panel, I needed to assess whether or not what I’d designed was useful to merchants. This page will focus on how I got feedback on the admin panel screens.

To learn more about how I initially designed the admin panel screens, please click here.

Objective:

Because merchants are not very technically savvy, it was important that the designs I created weren’t too overwhelming. As the screens stood, I felt we were providing a great deal of information and what we really needed to know was:

• What was most important to the merchant?

• Was this information present and easy to comprehend?

Via the customer discovery interviews I performed before, I knew that merchants were often busy and did not have much free time. Keeping this in mind, my approach was as follows:

• Send out emails with a screenshot requesting feedback

• Request more time to talk (~15 minutes) with those who respond

After discovering what was most important, my next approach to get more of a complete understanding of what merchants thought of the admin panel as a whole and to ensure that it was not complicated to navigate went as follows:

• Recruit more merchants for testing/feedback

• Adjust screens to highlight most important information

• Build prototype

• Test with merchants; get feedback on consumer side as well

• Iterate based on findings

• Provide screens to development

Testing & Feedback:

Feedback: 1st Round

In order to get the first set of feedback, I reached out the merchants that I’d spoken with initially during the customer discovery interviews. I requested that they view the dashboard screen — since the screen mostly encompassed all of the information provided on the admin panel — and respond with what was most important to them in addition to any questions or other comments.

I received 1 reply that resulted in a phone call.

Major Findings (from email response/phone call):

• The most important information is the number of sales/money made

• Analytics are helpful to those who are more technical/have more technical people on their team

Using what was learned, the admin panel dashboard was adjusted to display the commission information at the top and the analytics screen focused more on showing conversion to purchase.

Recruiting Testers:

In the midst of incorporating feedback and building the design prototype, I needed to find a way to include more merchants from our target market who’d be interested in testing. In order to do this, I pursued online and offline methods:

• Attended and spoke to vendors at CurlFest in NYC

• Attended and spoke to vendors at Ubiquitous Expo in Washington, D.C.

• Reached out to hair brands using Shopify that I discovered on Instagram

• Reached out to hair brands in Facebook groups and via Shopify’s community forum

I gathered contact information from several merchants and acquired 2 additional testers who were open and willing to participate in user testing, increasing the total number of participants to 3.

Prototype:

I created the admin panel prototype using Sketch, Craft and InVision. This prototype allowed testers to move through the onboarding process and to navigate to different pages from the dashboard.

usability Testing:

Major Findings (from moderated testing):

• The purpose of the app was more clear when the benefits explanation was seen first

• Some merchants would not feel completely comfortable with colors like red on their website

• Onboarding was not difficult; however, some merchants did not see the option to activate Product Y

• The single image representing the straight hair option was not representative enough

• The Shopify admin panel navigation and the app’s admin panel looked too similar

• Being able to customize the color of the app was very important

• The copy was extremely important when describing products that were clean/natural vs. those that were not

Design Updates:

Some of the changes as a result of testing and also implementing IxD and accessibility best practices:

Lessons Learned:

• Getting feedback from business users, as mentioned before in a previous lesson learned, is much more difficult (especially without resources for targeted recruitment) because of their schedules and because they are not as easy to find; this difficulty extended our team timeline much further past what we’d intended. This, in part, resulted in the delayed launch of this product

  • It’s necessary to get creative in finding ways to get meaningful feedback when designing for business customers and to do this early
    

• Working against the clock when business customers have a hard stop at a certain time during testing is extremely challenging and even flustering at times — it caused me to miss key opportunities to get clarification or ask helpful follow-up questions

  • In reality, it may not be possible to get all of the input that I intend to but it’s important for me to keep my focus and be present in order to get valuable information during the time that I do have with the participant during testing; it’s better to get really valuable information on a couple things than it is to get subpar input on all things

  • One of my mistakes during this round of testing is that I did not get input on the Call to Action wording and placement; due to this oversight, that feature was tabled for this version of Product Y

    • This ended up being a good thing, as it simplified development; however, it was still something that I dropped the ball on
      

• After designing the admin panel mockups (nearly to their entirety), I had to go back through the designs to ensure they were web accessible and that interaction design principles were utilized throughout. This took up a considerable amount of time — much longer than anyone expected

  • Good UI design is much more than choosing a nice typeface and color. As with UX, it needs to be purposeful and support the user in making meaningful decisions/taking meaningful action. It’s important to at least be aware of this at a high level in order to empower users when they use your product

  • Accessibility is much more than making sure font is readable. While much of accessibility implementation happens within website code, design still has an important role to play. It’s necessary to understand the nuances of what being web accessible is and where to consult those guidelines
    

• Designing without regular input from development about what is and is not feasible, among other things, is incredibly difficult. Much of the time, I felt as if I was guessing and hoping for the best

  • Each situation is unique and there are times when I will just have to do what needs to be done. The goal here was to test as much as realistically possible before handing anything off to the engineers due to time and resource constraints

  • Ideally, I would like to involve at least one developer in what I’m doing from the research phase onward in order to get their point of view and ideas about how certain things may/may not be able to work

Next Steps:

After completing the testing and iterating based on what was learned, I added the updated prototype and screens in addition to the screen flow, other design assets, and design notes in InVision. This was passed off to one of our contracted Shopify developers.

Once this was passed off, I needed to ensure the consumer experience of Product Y was good to go. In order to do that, I needed to test and apply accessibility and IxD best practices to those screens as well.

To continue learning how I went about testing and improving the consumer experience, please click here.