Assignment 6 - User Testing & Analysis
Task List
| Title | Instruction | Rationale |
|---|---|---|
| Set Up an Account and Login | Try to use the app without an account. Then, go to the login page, and provide a username and password to create an account. Then you can use the same credentials to log in whenever you want. | This is required for almost every other action in the app. Moreover, there are a lot of errors that can happen when not logged in, and I want to test if they all work correctly and lead the user to the login page. |
| View Articles | Go to the user list. Select a user that has articles. Go to the user page. Access a free article. Try to access a paid article and try it again after subscribing. Access the article Advancements in Gene Editing: Revolutionizing Biomedical Science by John Smith (populated with a lot of comments). | This is the most core action the user can take, viewing articles. I want to test it to see if the free/paid article difference and its relation to subscription is clear, and also to get feedback on how easy on the eyes the articles are. |
| Comments and Subcomments | On an article's page, look at other peoples' comments to the articles, and the subcomments to the comments. Leave a comment and a subcomment yourself. | I devised an interface that I feel is very unique to look through layered comments. It would be very interesting to see it in action and see if it actually affects the user experience positively. |
| Getting Validation | Create a bio, and then request validation. See that this now allows you to post articles. Delete your bio to see how it affects your validation. | It is crucial that this part of the app works correctly as it is what allows people to post and be sure that the posts come from trustworthy sources. Also, I want to see if linking together the validation and bio concepts this tightly is an intuitive idea. |
| Managing Articles | Post two articles. Make one free, and make one paid. Monitor the articles from the page. | This will be the part where the main content creation comes, so it is very important for this part to be very convenient so that users are likely to populate the app with interesting content. Also, I want to see if the font style I chose that is supposedly very good when reading is also good for typing. |
Study 1
The participant set up an account with almost no concerns, but there was something interesting to consider. While choosing a username, he first typed out his real name as his username, but then opted for a more informal "online name". It seemed somewhat vague to him what a username represented.
While trying to view articles, he seemed to have no problem listing the users, choosing the correct one, and viewing the free articles. However, when he tried to view a paid article, he was at first a bit confused about what to do when he got to the page that had the error message, although he did what most people in a situation like that, go back, and eventually figured it out. Quoting him, the article page seemed "very pleasant".
While looking at the comments, first he was a bit confused as to how the interface to go between layers of comments worked, and mostly because of how fast the interaction was. He seemed a bit confused when all the comments switched after changing layers. However, once he picked up on how to use it, he became very fast with it. He liked it as it allowed you to "choose which part of the content to focus on, often simultaneously".
As we moved on to the validation part of the test, he seemed to understand how the bio and the validation concepts were linked quite well. The fact that you would need to show some more validation did not seem unfamiliar to him. However, I think he again seemed to have the level of "formality" the bio should have, similar to his confusion with his username. After explaining that the bio would potentially be used by the app maintainers to check if he was suitable for validation, he seemed to understand the purpose better.
Finally, the posting of the articles seemed very convenient and intuitive for him, but he wished he "had more access to different formatting options", like italics or different font sizes.
Study 2
The second participant commented that on the user creation page, there is not enough feedback for an action so crucial. In the later parts, when he was posting comments and articles, he said that "these have more weight than setting up your whole account". Moreover, later, he said that this could be even more of an issue when he learned you aren't allowed to change your username.
When viewing users' pages and articles, he seemed to navigate the subscription concept without any problem at all. Observing him, he seemed to understand the access relationships just from the color-coding alone. However, he had one criticism about the user pages. He said if the bio was going to be on the same page as everything else, it might "take up too much attention", and it might be a good idea to "make it expand and close".
While commenting, he was very critical about the look of the buttons. He picked up how the system for managing layered comments worked, but he thought the buttons were "unpleasant with large writings on them". When I asked if he would be able to pick up how the interface worked without the buttons, he said "Maybe a bit more slowly, but I think long-term usability is more important".
While getting the validation, he really liked how when you try to do an action in the app that you currently do not have access to, the app guides with you links really smoothly to where you should be, and he found the whole process to be overall user-friendly.
After posting an article and going to the manage articles page, he had an issue with the color scheme used by the Make Free and Make Paid buttons. He said yellow for Make Paid and green for Make Green did not make much sense. When I asked him which colors would, after thinking for a while, he said he was not sure and maybe "using colors doesn't make so much sense" for this context.
Opportunities for Improvement
User Names Vs Usernames
Right now, it is unclear whether username are supposed to be the users' actual names or semi-anonymous tags to associate users with. The profionalism of the article posting side of the might app might suggest the former to be true, while the apps inspiration from apps like Twitter might suggest the later.
I think the best solution would be to add a section into the bio creation, so that users would define their real names in their bios. This makes both of the nameing options available to users, makes it clear which option should be used when, and adds into the "verification" aspect of the bio.
This is a conceptual issue, and I would clarify it as major, as the way users are identifyied are a big part of the app.
Bio Formality
I feel like previous section also ties into the first users' comments about how he was unsure about how formal bios are supposed to be. Adding some identification requirement, some information about how the bio would be used, and adding a slightly more formal UI for the bio could eliminate that confusion as well.
I would say this is again a conceptual issue, but only minor to moderate in severity. And its solution could be conceptual (adding more requirements for the bio), linguistic (adding a more formal UI), or just having more info available to the user.
Comment Layer Traversal
I got two criticism about the comment layer traversal interface from the participants.
One was that the transitions were so fast that it could be somewhat confusing on what exactly happened, and to find how your cursor related in this new state of the UI. The solution to this would be to add an animation clearly displaying the transition. This is a physical issue, and I would clarify it as moderate.
The other one was that the buttons, although descriptive, were too big and unappealing. This is an aesthetic issue (pleasantness), and I think the solution is a linguistic design decision: to put symbols on the buttons instead of large amounts of text to convey the same information on how to use the interface. I would say this is a minor issue.
Color Confusion
The second participant was a bit confused about how the color yellow made sense for the Make Paid button, and the color green for the Make Free button. This is a minor linguistic issue.
After discussing a bit with the participant as well, we came to the conclusion that color coding Make Paid and Make Free was tough, as those two concepts did not necessarily directly correspond to colors. Instead, using symbols can be a better idea to denote the difference between buttons.
Weight Missing
Finally, there were comments that in some actions like creating a new account, there was some "weight" missing: In other words, there was not much confirmation request or animation when you push the button to take the action. This is a very minor physical issue, and the solution would be to add some visually appealing animations and popups for actions like creating an account.
Notes
AI Usage
The articles and bios in the app were populated with ChatGPT to create a lot of text to make the website feel more realistic.
Moreover, the usage of ChatGPT seems to be such an intuitive solution for website testing that both participants asked if they could use it as well. I said that it was fine, as I felt saying no would be hypocritical.
Website Population
The website was populated so the content was denser and more complicated for parts of the app the task list instructions guided the participants towards, and sparser in parts where they wouldn't interact with.