Objective
61% of Etsy’s gross merchandise sales are from Mobile Devices. Etsy has been seeing a lot of its buyers using a Mobile devices for their Transactions.
Purpose
The purpose of this study was to understand how confident users are with the Etsy Mobile website and what device preference do they have.
Role
Sole researcher, Survey
Team
Solo project.
Methods
Started the research with a survey where 15 participants were asked to fill out a survey form. This gave me an idea of how many people use Etsy in their daily lives. I then moved to the next step where I recruited 5 users for usability testing. All of them were well-versed online shoppers, who had used Etsy before. As per their feedback, I suggested changes for iterative testing.
Etsy Research
Demographics:
The participants I choose for the survey were all from North America roughly between the age group 16-45
Create, Conduct, and Analyze a Survey
I developed a Survey to assess users’ perceptions of Etsy. I created a survey form with 15 open and closed-ended questions. This survey was conducted with 15 participants.
Survey Answers
This was an unmoderated survey, where I sent out a Google Survey form to 15 participants to fill out.
33% of the users that I surveyed have not used Etsy before, which is one third of the total survey respondents.
Being an 18 year old company Etsy needs to strengthen its base and spread its reach to a larger audience. Once it does that only then would it be able to stand tall against its giant competitors.
Have you used Etsy before?
I used a screener question to understand my stakeholders.
1/3 of the participants had not used Etsy before.
What is Etsy?
Etsy is a marketplace where millions of people around the world connect, both online and offline, to make, sell and buy unique goods. Etsy connects buyers and sellers in nearly every country
As per the survey, 80% of the customers come to Etsy in search of Unique, personalized, and quality products. This certifies Etsy’s claim and builds trust in the company's major components
Out of the 15 participants 10 of them never used Etsy, hence I did not include them in my Survey. The findings are based on 10 participants.
Based on the survey results, I analyzed 3 top findings.
To what extent do you like or dislike the Etsy website?
What was the most frustrating thing about the website?
What do you look in a product when you use Etsy?
8 out of 10 participants have used Mobile while using the Etsy website. This data provides additional backing to the information that 61% of the Etsy purchases are from the phone. But I want to delve into this problem more by conducting a Usability test.
40% of the users that I surveyed liked the Etsy website a great deal. 40 % liked the Etsy website whereas 20% neither like nor dislike Etsy.
Top 3 Survey Findings
This Survey wants to understand why the users choose the Mobile Website vs the application. And how to improve their mobile website user experience.
I tested the actual Mobile website with the users.
I recruited 5 users for this usability test. All of them were well versed in shopping online.
I created a Moderators script to debrief the users. Each participant was taking the usability test from their respective homes.
After completing the tasks the participants completed a satisfaction questionnaire.
The users were also asked for their recommendations/feedback about the product. After gaining their feedback and insights I would pass on my findings to the stakeholders and developers to decide on which issue needs to be fixed to re-iterate the design.
The next step was to conduct a Usability test to collect Quantitative data. I wanted to understand the user’s behavior with the help of questions like How do they get started? And What captures their attention.
I ran an in-person as well as moderated Remote usability testing for this exercise.
The participants I choose were real users of the Etsy Mobile application. I wanted to focus my findings on the Quantitative data hence I initially tested with 5 users and then analyzed the findings and converted them into ideas for the next iteration. I create a Script of what I will talk about with the participants.
Generated a list of Tasks to ask them to complete and the “correct” or expected answers to the tasks. After the testing was over I provided them with a link to a google survey for feedback.
1
2
3
4
How do they find out if a product is customizable?
You want to check the expected shipping time of a product; how would
you go about that?
You don’t like a suggested item on your list you want to remove that
suggested product from the list, how would you do that?
You have purchased a product, now you want to leave a review for the
product, how would you go about it?
Top Findings
Users were confused between Hamburger menu & the user profile icon to write a review for the purchased product
1
2
3
click to enlarge
User found a discrepancy in the mobile application and the mobile website when tried to customize a product
The User had to find if a product is customizable or not in various ways.
After completing the user testing, I compiled all the User findings from the 5 participants and listed the top 3 findings, and came up with a recommended solution.
1
Users were confused between the Hamburger menu & the user profile icon to write a review for the purchased product
4 out of 5 participants thought that the Hamburger menu would show the option to review a product.
But later users realized that the user profile icon was the place where the product review lives.
Make it more intuitive to find the review option for the product. And make it clear that the user cannot review a product after 30 days
Users want to review the product for a longer time, as the longevity of the product can differ.
Recommendation:
Follow up:
After completing the task, the users were presented with a Survey for feedback on the tasks. I documented the results in a graph format.
After completing the task, the users were presented with a Survey for feedback on the tasks.
I documented the results in a graph format.
3
Users found a discrepancy in the mobile application and the mobile website when trying to customize a product
The user who used the mobile application pointed out this discrepancy.
To make both the platforms identical.
The Filter feature in the application has customizable option, whereas that s not present on the mobile website
Follow up:
Recommendation:
1
2
Get stakeholders talking to one another: Invite key stakeholders to initial project meetings when developing the project charter. Resolve conflicts as soon as possible.
Challenged assumptions: Each of your stakeholders has expectations, sometimes false expectations. Working with your team will clarify many of these aspects of the project.
Primary stakeholders: Product buyer, product seller, decision-maker for the product
Secondary Stakeholders: Regulators, trade associations,
Their differing needs:
3
Different stakeholders would have different requirements.
For Seller: How easy is the application/website to use or buy products.
For buyer: Ease of selling a product with the service.
For Project Owner/manager: To increase the traffic on the website to increase the profit of the product
1
2
After the Usability survey I understood the key stakeholders better, I also created a list of key stakeholders and what their needs would be
WAVE can be very helpful in knowing which element on the website is lacking accessibility.
Improvement in the website
I could use this website for the overall accessibility not only for the disabled population but also for people without disabilities.
The TOPTAL website checks the website for color blindness. In my case where the website is already built, I will use this website to check out color options for different products under varied filters.
Improvement in the website: To provide descriptive text for different color options when the user hovers over the product. The users who have color blindness can this way identify colors based on the text provided.
WAVE
TOPTAL
Where do people look when they want to review a product, what captures their attention, and what is a distraction. Also to understand is the website disability complaint. I ran two accessibility tests on the website.
I would break my usability testing in 2-3 iterations with 5 participants in each session. Or I would run the iteration till it was sure that no more changes are required in the product. One thing that I will make sure before finding my users is that I would create a Screener test for them. The most important criteria in that would be that Users should have used Etsy before.
The first test I will run with 5 users. Once I receive their feedback, I would categorize the issues into high, moderate and low impacts. Once I do that I will prioritize the high-impact issues. I would then share my findings with the developer team and the stakeholders. Where the clients will decide on what changes are important. Developers would make the revisions on the design and I would run the test again. I really would want to run a test after every two weeks with 5 participants in each session. But it would also depend on the redesign requirement.
A. The list of questions I would attempt for eye-tracking are listed below:
You have purchased a product, now you want to leave a review for the product, how would you go about it?
When does a user want to review a product where would they look?
B. What would you expect to learn?
Eye-tracking will show how long it took the customer to focus on a specific product, the number of times they looked at it, and how long they spent gazing at a feature.
Eye-tracking would help learn the following
Scan patterns e.g What captures their attention and what is a distraction.
Time spent: Time spent quantifies the amount of time that respondents have spent on an AOI
Iterative testing for Etsy
Accessibility
Usability Survey
2
Users had to find if a product is customizable or not in various ways.
All the users tried different ways to find if a product was customizable or not.
-
Used the search bar.
-
Checked the product page
-
Checked the product description
An icon right next to the heart (favorites) icon would make it easier for the user to see if a product can be customized right on the main page, rather than search for it
I would want to hover over a product and it would say this product is customizable.
Follow up:
Recommendation:
Eye-Tracking
The initial study on Etsy showed that 61% of Etsy purchase happens on the phone. When I was done with my Usability testing I observed that 60% of my participants used both phones and Desktop for the Etsy purchase.
-
The users used Phones when they don’t need to do a lot of research on the product they plan to buy.
-
Users used Desktop when they have to research a product before buying it, the desktop gives them a larger screen and space to look at the product clearly compared to the phone.
Usability Testing
Tasks given to participants
(Usability questions)
Key Stakeholders
Conclusion
Top Findings of the Survey
3 out of 5 participants used both laptops and mobile and 2 used only laptops.
From my testing, I can conclude that
Etsy users use laptops equally to that Mobile. I would need to go perform iterative testing with 5 participants in each round having a similar background.
Only 2 out of 5 (40%) users would want to make a minor change to the Etsy Website. 60% would not change anything in the site.
NO Change
60%
Change
40%
Would you make any changes in Etsy?
Which device do you use?