User Testing & Analysis
Interviewees
Unfortunately, I was unable to contact the original two people that I interviewed for design ideas- especially unfortunate as one of them were one of the main stakeholders of this application.
Tasks
All of the below tasks are to be performed with the screen reader functionality built into mobile devices and most browsers.
| Task | Instruction | Reasoning |
|---|---|---|
| Sign up | Navigate to homepage, use screen reader to sign up | Being able to sign up without any visual assistance is an important portion of the application. |
| Enable and disable unwanted features | In the profile tab, verify which features are enabled. Then, edit the profile to disable features that aren't needed. | Not everyone needs speech based features, and it's interesting to be able to turn them off. |
| Add your friends! | Given the usernames of various friends, add them without visual assistance using screen reader. Once the friend request is approved, try to view their posts. | Reading posts from friends is an important part of the app, and making sure the friends functionality worked was important. |
| Write, edit, delete, and comment on posts. | Write a comprehensive post on whatever you feel like! Use screen reader if time doesn't take too long. Try to reply to another person's comprehensive post. | The only posts you can see are those of your friends and yours. Now that the person has friends, it was time to test out the post functionality and see if it works as intended. |
| Remove a friend that pissed you off! | Did that "friend" send you hundreds of angry messages? Think you don't have time for it anymore? Remove them! ehehe | Human relationships are volatile. It's important to be able to halt it at any moment 😃 |
Reports
Both interviewees were instructed to use the screen reader functionality on their browsers to navigate through the application. After the screen reader test, both users were allowed to test out all the tasks without being bound to the screen reader only.
Interviewee 1
The user was able to sign up to the website without any issues- but noticed that the screen reader was not that good at helping navigation. Once signed up, the page automatically pushed the user to the profile page, which confused the user as the screen reader went a little haywire. Eventually, they were able to edit the profile to contain the short introduction they wanted, and chose to disable the speech generation feature for fun. (Note that this was simply a website-based function, so they were still using the screen reader.) Afterwards, they were instructed to send a friend request to "andrew" and approve a friend request from "gdlskj". The user noted that the friends tab had too many buttons and information that the screen reader had to read out, making it uncomfortable to use. The more friends the user had, the more issues they had finding the correct course of action. The same thing applied to the post list functionality- although it was easy to find a specific user's posts, listing all of them left the user disorientated. They were still able to make a post, and read a post from "gdlskj". "gdlskj"'s post was supposedly infuriating and deserved to be removed as a friend, and the user was able to do that without many issues. However, the UI design would need work if there were too many friends to search through- otherwise it'd be impossible to find the friend to remove in any reasonable amount of time. The post writing function was easy to use, but the reading function also followed the same issue- not to mention the comments. A "max 5" with next page feature was sorely missed. The user was able to sign out without issue, and commented that having all the navigation buttons on the navbar made it easy to navigate with the screen reader. Afterwards, the user pretended to be a "loved one", aka using the system without the screen reader and using all visual components. The UI was intuitive enough for the user, and had no major complaints.
Interviewee 2
Having never worked with screen reader before, the user had a lot of trouble setting it up- but that was out of scope for the application, so help was warranted. Once they had familiarized themselves with screen reader functionality, the user managed to create their account after five minutes of deliberation. The user chose to skip editing the profile, as they already chose the features they wanted during sign-up. The user thought the modals were an unnecessary feature, as they made screen readers act a little weird. In their defense, they made the pages much cleaner visually- and non-impaired users were also part of the app's users. The user also commented on the slogans on the home page- the page itself looked nice but the slogan was "cringy". Moving on, the user first made some posts- and was dumbfounded when only their posts were showing up. After a while, I mentioned that the reason to add friends first was that you could only see your own and your friends' posts. The user requested the addition of a "view my posts only" page, as it would be hard to view through all the posts to find their own if they wanted to edit or delete them. After adding the friends through the UI, the user had the same feedback as Interviewee #1, and hoped that the page could become simpler- such as including removing friends by username and a search functionality by name (in the profile.) The user then chose to leave comments on a multitude of posts and again had the same reaction- the post comments seem to list infinitely, so a single post could show hundreds of comments, absolutely ruining the visually impaired experience. The user was able to remove friends without much hassle, and sign out without problems. Many of the issues they pointed out were due to the project not being up-to-scale, leading to design decisions that wouldn't replicate for larger test groups.
Opportunities for Improvement
- As screen reader tends to be very clunky to use, find a working implementation of speech generation/recognition for Vue.js and apply it correctly.
- Only being able to view friends' posts are fine, but maybe add some other functionality where you can make your posts global.
- Add messages to be able to contact friends 1 on 1.
- reduce many UI elements to list minimal amounts of things, as having too many things to listen to can be clunky.
- Add search by Name functionality, as username communication could be hard- and add a way to list only top 5 posts, comments, etc.