A6. User Testing and Analysis
Tasks
| Title | Instructions | Rationale |
|---|---|---|
| Register | Register an account for the app | I want to test if the user interface is intuitive and conveys the purpose of the app upon first impressions as well as to see if the layout and navigation can help facilitate user engagement. As the creator of the app, I am too familiar with the app itself which is why it is helpful to test this with an outside user. |
| Create Event | Formally create an event through the app. Example prompt: Say you want to visit Yellowstone Park during IAP to collect local nature samples from different locations in the park. Create a(n) event(s) accordingly through the app. | As a core concept of the app, I want to test if there were any barriers between a user and their goal of creating an event. Do they know where to create an event? How to input the locations for the trail of the event? Understand when their event has successfully been published? |
| Explore and Register for Events | Look for an event, learn more about it and register. Example prompt: Say you’re a beginner hiker visiting your family back at home during Thanksgiving Break. Find an event that might suit you and learn more about it (Who is attending the event? What are the specific destinations of the event? When is the event? What do you need for the event?) Register for the event and ask a general questions pertaining to the event. | I want to test the 'explorability' and 'discoverability' of events. I want to test if the user interface is able to present useful information in a way that is beneficial for the purposes of the user (looking for events meeting certain geographic, temporal, etc. criteria). I also want to understand if the logical flow of registering for an event first and then accessing the general event discussion is intuitive and if the user can discern the function of the event-specific feed. |
| Edit Trails | Now the user has trails from registered events copied to their profile. Say you are currently hiking on the trail or have completed it. Now you want to: 1) Change the location for one of the points on the trail to closer match your position in real life. 2) Attach an image and text post to that location | As another core concept of the app, I wanted to see if users understood the purpose of the trail concept, how it ties to / distinguishes from events, as well as how to use them to fulfill their purpose of documenting outings. |
| Global Interactions | Ask a general question about an outdoor activity or make a general post that anybody can see. | I wanted to test if the user is able to understand the purpose of the general feed and discern it from posts that can be made elsewhere. |
| Discover and Add Friends | Find a person you might be interested in being friends with. Learn more about the events they are registered for and the trails they've been on. Send them a friend request. | I want to test the explorability and discoverability of different profiles and see how the app's structure and functionalities hinders or facilitates socialization. |
Study Reports
Note: Ideally, I would have liked to get users that are currently hiking or participating in an outing to use my app. However, this was not possible at the time of conducting the user studies so I provided prompts and situations to contextualize the study appropriate for the app.
User Study #1: SC
When prompted to register an account for trailMix, SC was able to do so easily but he expressed confusion at the landing page as it was just a large, interactive map. The landing page didn't provide enough context for him to understand the general purpose of the app or how to use it but the interactive map seemed impressive to him as we began to play around and exploring with it (dragging it, selecting on points). I then prompted him to formally create a specific event through the app, providing him the location, purpose, and timeframe for the event. SC tried to create an event by tapping on the location he was interested in creating event for, then he clicked on the calendar button to open the event composer, then closed it and double tapped on the location again before eventually realizing he had to fill out the first page of the event composer's form before he could pick out the locations on the second page. I believe the reason behind this behavior is that the user had default expectations for what interactions the map could provide -- he probably assumed that location-based content could be created through the map itself. This is rather understandable as other apps such as google maps allow us to drop markers on a map. He was able to fill out the event details smoothly and provide appropriate tags for the events. When he reached the second page which defaults to the trail selector, he searched through the pre-existing trails said "no that's not what we want", and switched to the trail composer. He seemed to understand the differing functionalities of the trail selector and composer. On the trail composer, SC read the instructions for how to compose a trail. However, he did not opt to create a trail by dragging markers and instead looked up all the coordinates for the locations and input them manually. I don't think the directions for how to create a map by dragging the points were clear enough. He also provided similar descriptions for the trail and for the event. The trail description described the purpose of the event rather than the trail itself (such as its locations). This suggests that the user was unable to discern the differing functions of events and trails. The user also didn't add their own custom items in the checklist form though their given prompt might have suggested they needed to. This could possibly be because the instructions weren't clear enough or positioned to catch the user's eyes as it didn't seem like he read them. SC's next task was to search for and resgister for an event that satisfied the fake scenario we described. He was able to quickly find the list of events (only because he stumbled upon it accidentally while searching for other functionalities), but he had trouble identifying which events met his geographic needs based on the event previews alone (he asked me "wait, is Yosemite in California?"). He was able to look over the details, understand it was appropriate for his scenario and registered for it. For his next task of adding a post to his newly populated trail (resulting from event registration), he tried to edit the trail by making a post in the event-specific feed. This suggests that the user interface does not reflect the state of the system. Once an event was registered, it was not immediately obvious that the trail was copied over to their profile. When he tried to look at the trail on the home page's map, it was also covered by another user's trail from registering for the same event.
User Study #2
The second user JB experienced similar difficulties and illuminated even more design flaws during her user study. Like SC, she was fascinated with the interactivity of the map, playing and engaging with it when assigned a task to create an event. When asked to create an event centered around a location, she first dragged the map to Siberia and then clicked on the toggle and the icons next to the toggle (believing they were buttons) before opening the event composer. When composing her trail she called the latitude and longitude input "silly" and had to look up the coordinates to Siberia. I believe JB also didn't read or understand the instructions for how to compose a trail by dragging because it was on top of the map and not at the coordinates input which is where her eyes were drawn to the most. It also seemed like she thought the drop down options were also mandatory as she half-heartedly selected them ("sure we can go backpacking"). Like SC, she also put non descriptive trail descriptions and didn't add any custom items. She also added only one location to the event instead of multiple to create a trail. The trail concept may not have been properly introduced to the users. Like SC, JB was able to gather a lot of information from the event pages but didn't really utilize the event-specific feed. When asked to edit her trail, she as confused as to where to find it. I noticed that for both SC and JB, they were confused about what it means to edit a trail and had to be explained that when an event is registered for, a copy of the trail will be made for the user to edit. She went back to the event page and tried to edit it from the trail preview. She then went to her profile and was confused because it indicated that she hiked 54 hours already ("how did I already hike 54 hours?") because the counter adds up hours from events that haven't occured yet. She edited the post through her profile and didn't really utilize the home page except to make events. When asked to find someone and send a friend request, she was also confused where to find all the profiles as she tried going to settings and looking under the "manage friends" section. She said "the only way to find a profile is to go to a discussion, click on a person and send a friend request from there." She also said "if someone doesn't ever post, how can I be friends with them?"
Design Flaws and Opportunities
There is a critical conceptual flaw involving the confusion with trails and events. Both SC and JB's behavior implied that they couldn't discern the difference between trail and events or that trails get copied from registered events into their profiles for further editing and future documentation. I had to intervene with the user study to provide clarifying information. This flaw would be harder to address and would require greater changes to the structure of the code but a good start would be to reflect updates in the system's state in the UI (for example, when an event is registered for and a trail is copied, a toast could further clarify or the app could take the user to the trail on their profile). We can make it more evident that the same trail can be reused for different events by different users by introducing this language of "events" and "trails" more clearly in the composers or on a landing page or redesigning their relations with one another.
Minor longuistic flaws manifest in unclear icons in place of buttons with text and a lack of / unclear instructions. The purposes and meanings of the view toggle and the different icons (grid, map, calendar) were unclear to SC and JB as they were unsure of where to create an event or what switching the toggle would lead to. They also didn't create a trail by dragging or add custom items because of the language and position of the instructions. These issues suggest simpler fixes such as making the functionalities of different buttons and switches more explicit, revising the instructions and adding helper popups.
One moderate, physical design flaw manifested in the interactivity of the map. Both SC and JB had default expectations for how the map would behave on-touch. In general, I can tell that iwas overwhelming for them to drag around such a large globe to look for a specific location. Additionally, all the trails put on the map made it hard to see and identify each individual trail, especially when users subscribed to the same event all had their trails overlaid on top of one another. This could be resolved by adding in more functionalities, such as filtering for different user's trails (the user, their friends, other) instead of displaying them all at once. It would also be usefule to add a location search functionality so that a user can input a search and it would take them there on the map to reduce the stress of looking for the location. Another functionality could be geolocating if the user wants to drop a marker on their current location. SC mentioned that while more expert hikers may speak the "language of geographic coordinates," a beginner like him wouldn't be able to use it to input coordinates.