PRATT INSTITUTE RESEARCH STUDY
Leveraging Eye Tracking data to improve application process for desktop website
Pratt DX Center worked towards understanding how Pratt Institute's prospective undergraduate students perceive the institute, learn about the programs, and move through the application process on the desktop site to recommend necessary changes and developments to improve the overall user experience.
​
To help our clients achieve their mission of understanding whether the current user flow helps in effective task completion, we have analyzed the user's visual attention using Eye-Tracking Study.
​
We have identified room for improvement in the content discoverability and hierarchy, navigation, and given the scope for the further usability study on category taxonomy.
My Role: Moderated sessions; analyzed data; generated findings; consolidated the recommendations and problem list; crafted presentation
Client
Pratt Institute
Team
Anamika Menon, Kyle Oden, Sacchit Vartak, Mishi Sarda
Duration
Oct 2022 to Dec 2022
Tools
Tobii Pro, Figma, Miro, Google Suite, Adobe Premiere Pro
Measuring user behavior on the website
After our client kick-off meeting, we decided on the goal of understanding users' behavior on how they navigate, find content and apply for programs at Pratt Institute.
​
Pratt Institute recently reimagined their website through a complete revamp of their website. This prompted certain concerns amongst key stakeholders regarding the visibility of key steps taken in the application process, limitations caused by information overload and the general accessibility of their application CTA’s. To address some of these issues, my team and I conducted an eye tracking study to gather data around these assumptions on the usability of the website.
RESEARCH QUESTIONS TO IDENTIFY THE INSIGHTS FROM THE STUDY
My team and I prepared the following research questions in order to understand how prospective undergraduate students accessed and consumed content and approached the process of gathering information about courses and programs on Pratt Institute’s desktop site.
How are users are using the navigation and sub navigation on the website?
How effective is the website's Information Architecture and content hierarchy?
What elements on the website highly contribute to the user's visual load?
Methods of analysis to gather insights
Post doing multiple pilot testing sessions to test the tasks and tools used in the study such as Tobii Pro, the team created the updated final study materials. Post that, our process started with screening and recruiting participants to conduct an Eye-tracking study. A research ops team was formed to combine recruitment efforts and prepare a screening questionnaire which aimed at understanding user's educational background, demographics, their interest in pursuing graduate or undergraduate education and to ensure the recruitment of ideal participants for the most accurate results (participants who didn’t wear bifocal lenses/ glasses and those who hadn’t had any form of eye surgery).
​
The study was divided into three parts starting with Eye Tracking to analyze user's eye interactions with the website, then providing System Usability Scale form to understand the usability and learnability of the website and ended with Retrospective Think Aloud process where participants were encouraged to express their reasoning, challenges and liking while interacting with the website.
PARTICIPANT PROFILE
There were certain key challenges and considerations to recruit the participants for the study:
​​
-
Unable to recruit high school students due to legal as well as parental consent requirements.
-
Expanding the target participant group to recruiting graduate students.
Moderated Eye Tracking Study
Analysis and Synthesis
The analysis helped us understand the SUS score, measure effectiveness and efficiency through usability metrics, minor challenges faced by users, and learn about what worked well for Pratt Institute's website.
HIGH LEVEL FINDINGS: OVERALL PERFORMANCE
HIGH LEVEL FINDINGS: WHAT WORKED WELL FOR THE SITE
KEY AREAS OF IMPROVEMENT WITH A HIGHLIGHTS REEL
KEY FINDING 1A: LOW VISIBILITY AND INCONSISTENCY AFFECTING DISCOVERABILITY
The gaze plots and gaze replays showcase that participants quickly skim through large text boxes due to the excessive visual and cognitive load. This causes them to miss out on important information.
Gaze plots highlighting user's eye movement on the BArch Program page
While the overall structure of program pages is consistent, the inconsistency in content adds to the users cognitive load and reduces the efficiency of the website.
Gaze plots highlighting user's eye movement on the BArch Program page
RECOMMENDATION: HIGHLIGHT VITAL CONTENT TO INCREASE DISCOVERABILITY
Highlighting fundamental information such as program duration in a separate section on every program detail page will help reduce the users cognitive load.
Recommendation to highlight key information such as duration of the course
KEY FINDING 1B: LACK OF VISUAL SIGNIFIERS AFFECTING DISCOVERABILITY
Since the How to Apply section of the side navigation does not visually signify it’s an accordion, it is not intuitive for the user to click on it to find more information.
Additionally users have to navigate to a new page every time they click on a section to access more content.
Accordion of the pages visually unclear
RECOMMENDATION: ADDING VISUAL SIGNIFIERS TO FACILITATE ACCESS TO CONTENT
Making each section an accordion will let users’ browse through and easily access key content thus improving its visibility and accessibility.
Recommendation to make accordion
KEY FINDING 2: BACKTRACKING ON WEBSITE PROVED TO BE DIFFICULT FOR PARTICIPANTS
The gaze plots and gaze replays showcase that participants attempted to go back to the pages from where they started to navigate the website. Often times they looped between pages which affected the efficiency of navigation and discoverability on the website.
User's gaze suggests finding back button
RECOMMENDATION: USE NAVIGATIONAL AID TO KEEP TRACK OF USERS' LOCATION ON THE SITE
Using breadcrumbs as secondary navigation aid will help user easily understand their location on page and their relation with other pages of website. User’s could easily find their way back, go back a step or start over which can improve the discoverability of pages on the website.
Recommendation to add breadcrumbs to facilitate navigational aid
KEY FINDING 3: CONTENT HIERARCHY DOES NOT MATCH USER GOALS
The heatmap show that participants spent a lot of time scanning through Pratt’s programs of study. However, this information is placed last on the homepage. In fact, one participant did not reach this section.
'Programs of study' placement on the homepage
RECOMMENDATION: RESTRUCTURE CONTENT HIERARCHY TO PRIORITIZE KEY INFORMATION
The priority of content should determine the order and hierarchy of the homepage. Since programs of study is key information for users to reach when considering applying, this content should be highlighted higher than it’s current position.
Recommendation to add breadcrumbs to facilitate navigational aid
KEY FINDING 4: CHALLENGE DIFFERENTIATING NAVIGATION CATEGORIES
The gaze plots, heat map and feedback showcase that participants find it challenging to differentiate between categories such as Courses and Academics and find them very similar. This causes them to take additional time and visit different pages for finding important information.
Gaze plot and heatmap of the navigation categories on homepage
RECOMMENDATION: FURTHER ANALYSIS TO IMPROVE CATEGORY TAXONOMY
We recommend conducting further analysis and study using methods like Card Sorting and Tree Testing which can be helpful in improving the terminologies and eliminating confusing or similar tabs on the top navigation.
Recommendation for further study with methodologies such as card sorting and tree testing
Client Feedback and next steps
In the final meeting, our team delivered a presentation with insights, findings, and recommendations to our client, Pratt Institute. The presentation also included a highlight reel and an in-depth problem list of findings we discovered during the study. The client greatly appreciated the thoroughness of the study and how it provided them with data that highlighted some of the issues that have already been a part of the conversations and debate amongst the stakeholders about the future and further changes for the design and development of the project.
This is a body of work that I could think end up genuinely informing the work that we do in the future.
Having this data is going to be invaluable to us.
- Sarah Hromack, Pratt Institute Representative
LESSONS LEARNED THROUGH THIS PROJECT
One major limitation during working on this project was recruiting target key participants who are high school students/ prospective undergraduate students within the age range of 18-20. My team and I also addressed this in our participant profile and tried to recruit participants as close to our target group as possible. We also addressed this by providing the participants with a scenario before the tasks to help them visualize and understand the tasks better. This process prepared us for future projects and we have to be careful with the considerations and time management to best prepare for these studies.
NEXT STEPS
Our study highlighted the challenges faced by users in differentiating navigation categories on the website. As the clients wanted to learn more about user flow in facilitating the application process, we think it will be ideal to conduct further analysis by conducting card sorting and tree testing methods in the future, as they can be useful to create an improved structure before implementing changes.
​
We are also eager and look forward to seeing how the client addresses our findings and incorporates our recommendations to improve Pratt Institute's desktop experience for Undergraduate Students while keeping other target groups in mind.