Skip to content

Project 6: User Testing & Final Release

User Testing Reports

User #1

The user was able to successfully accomplish all the tasks easily with the exception of finding a disposal bin for plastic 4. They were able to successfully navigate to the map, but thought the map was a little too sensitive and was confused about the meaning of the colored markers. While clicking on the bins in the map, they did not pay attention to the bin info panel except for the info about the status since the task was to find a not full disposal bin for plastic 4. Since they were not paying attention to the bin info initially, they did not realize the colored markers represented the different types of bins and thought the colors indicated different bin statuses. However, in the task to find all the types of bins on the map, they realized what the colored markers meant and said they “can catch onto the colors pretty quickly but it would be nice to have it be said somewhere.”

When reporting the bin was full, the user first went to the Search page and was confused as to the purpose and usage of the page since its “just a search bar with no other information” and “home is where you search for the materials you are looking for”. They were also unsure of where to go to report the bin was full since they thought “map is obviously just a map” but ultimately were able to find the report bin full button in the bin info panel by clicking around on the app.

Interestingly, the user never visited the Material Info page because they clicked on “See Disposal Locations” after searching up the material instead of clicking on the material box itself. It was not clear to them that they could click on the box for the material. They also mentioned they were not sure what some of the materials each bin accepted was since it was just an image without any labels.

User #2: Grant

With the exception of the first task, Grant was able to successfully complete each of the tasks without intervention from me.

A shortcut to the search page is provided on the site’s landing page with the words “Where can I put a _ ?” surrounding an input field. They struggled to figure out what search query to use to find “Plastic 4”--trying phrases like “tupperware box” or even “plastic 4 tupperware box”--and mentioned that they were confused about that language.

Grant used a mouse to interact with the website, and found that the map was pretty easy to navigate. When identifying the types of bins associated with the map’s markers, they immediately identified the recycling as blue and the compost as green and noted that the color association was “mostly intuitive”. However, the meaning of the black and orange markers wasn’t particularly clear, and they added that they would’ve found it useful to have a legend. When asked where he could log that they recycled or composted certain items, he immediately navigated to the dashboard page. They weren’t exactly sure that this information would be found in the Dashboard, but deduced that the other pages didn’t make the most sense and felt that a page with that title might be “tracking something”.

Other than the interactions mentioned above, Grant found the UI to be pretty intuitive, and knocked out these tasks pretty quickly. They never actually interacted with the material info page during the predetermined task list. Instead, while playing with the map afterwards, they stumbled upon one of the material info pages by clicking on one of the picture icons by accident.

User #3: Mercy

Mercy was able to complete each of the tasks successfully and efficiently without any intervention from the interviewer. Like the other users we interviewed, they found the colored markers on the map to be mostly intuitive (with the exception of the orange to donation bin association), but would’ve appreciated an explanation of sorts for the meaning of each of the colors. Further, since Mercy was using a trackpad (rather than a mouse) to navigate the site, she found the map to be pretty difficult to navigate.

After the study, Mercy played around with the map a bit. While it was easy to tell from the pictures what bins accepted or didn’t accept certain plastics, she struggled to identify what materials some of the pictures represented. (For example, it wasn’t clear what material was being depicted in the coffee grounds photo.) She was also surprised that the images on the bin information panel were clickable, and she explained that was because “the mouse didn’t turn into the grabby hand thing”. While Mercy appreciated that feature, she mentioned that the presence of a label underneath each of the images would’ve been helpful.

Overall, Mercy “really liked the premise of the app”, and felt it was useful especially in new locations where she didn’t know where to recycle what and where. Further, they commented on how much they liked the font and basic color scheme of the application. (This came as a surprise to me, since we hadn’t touched that at all, but is a good thing to note when doing last minute polishes moving forward.)

Opportunities for Improvement

  • Map legend for colored markers (Minor physical issue):
    • All users we interviewed were confused about what the colors on the map markers meant. They were all able to eventually figure out the different colors represented the different types of bins after interacting with the markers for some time but all mentioned that having “a legend” or “have[ing] it be said somewhere” would have been helpful. To fix this, we can add a legend to the map page explaining to users what the colors for the map markers mean.
  • Search query input box prompt (Critical conceptual/linguistic issue): - Grant was unable to complete the first task of finding a bin to dispose of plastic 4 because they were confused about what to input into the search query on the home page. To fix this, we can add a subtext under the current prompt with examples of things they can search for such as the material or name of the item so that users know what type of input is expected. Additionally, User 1 was confused when they clicked into the Search view page because there was “just a search bar with no other information” and “home is where you search for the materials you are looking for”. To fix this, we can add a prompt to the search page similar to the one we have on the search bar in the home page.
  • Materials Info Page (Moderate conceptual/physical issue):
    • All 3 users we interviewed either stumbled upon the Material Info page when playing around with the app and were surprised to find it or never reached the Material Info page because they did not know they could click on the material in the search result page or the material in the bin info panel. Users on the search results page clicked on “See disposal locations” directly instead of clicking on the material card to reach the Material Info page. We can make the card’s clickability on the search results page and the material’s clickability on the bin info panel more obvious by changing the user’s default arrow cursor to be the hand cursor indicating a component’s clickability. We can also add a “Learn more” link under the “See disposal locations” to indicate to users that they can learn more about the material and provide another method of taking the user to the Material info page.
  • Materials accepted/rejected at each bin (Minor linguistic issue):
    • Users were unsure about what materials each bin accepted/rejected because some images used were not clear. For example, Mercy struggled to decipher the coffee grounds photo shown as a material and said that a label underneath each of the images would be helpful in identifying exactly what the bin accepted/rejected. Adding a label will also ensure that users who may have the same object but with a different physical appearance will know that they can dispose of that object in that bin. Users will be able to match the linguistic label of their material or item rather than relying only on an image.

Design Revisions

  • Sync registering new user with access level none
    • Impossible to manually change the access level of all users. Most users will be normal users who should not have access to app-level changes such as updating a bin location or what the bin accepts as a material. Admin users can be manually added.
    • Further revision and scoping down due to time constraints meant we actually did not end up finishing implementation for different user access levels. Instead, all bins will be prepopulated in the app and users will only be able to report the bin capacity. Any other updates to the bin will require a member of team-w to manually edit the bin or add a new bin.
  • Removed the ‘#’ symbol from the numbered Plastic materials
    • There were strange unicode conversions that made matching the ‘#’ symbol difficult when querying MongoDB.
  • Longitude, Latitude coordinates instead of just x,y
    • Leaflet API requires coordinates in longitude, latitude
  • Colored markers on the map for each different type of bin
    • Decided to use colored markers to easily differentiate between bin types on the map without having to click into the map. This will help seasoned users who are familiar with the map’s coloring scheme to more readily find the bin that they need.
    • Added a legend for the colored markers since this was a source of confusion amongst users. This helps users learn our app and use it more efficiently.
  • User does not need to choose what to search by
    • Instead of making the user choose between item, material, and RIC for their search query, we just search our entire database for the user’s query and return anything that includes their query in the name. This makes it easier for a user without proper waste disposal knowledge to use the search feature.
    • Added example of what users can search for in the home page as a subtext under the search box so that users more easily understand what query the backend is expecting.
  • Did not implement Achievement, Community, or leaderboard due to time constraints
    • The Community, Achievement, and leaderboard concepts/features were all part of our gamification strategy to encourage users to recycle and compost. We decided to not implement these due to time constraints and because we thought the Score concept would be sufficient at gamifying the app and encouraging users to come back to see their sustainability contributions increase over time. Additionally, we decided to not implement a leaderboard since we did not want to risk users trying to increase their waste just to be on the leaderboard. One of our ideas was to use the contamination percentage rather than the number of items recycled; however, we were not able to figure out how to collect data for the contamination percentage with the time we had. Overall, these were all ultimately not required for our impact case goal which was to allow users to look up the proper disposal method for an item and then find the proper bin for it, so we believe not implementing these does not harm our app.
  • Did not implement UserSettings due to time constraints
    • The UserSettings concept would have allowed different users to set their own default locations, however we had to hardcode the default map location for a user (purpose of UserSettings concept) to be at MIT due to time constraints. We still believe users not near MIT campus would be able to easily use the app by interacting with the map itself (i.e. scrolling, zooming in/out) much like one does with Google Maps.
  • Did not implement similar items, commonly recycled items, commonly misrecycled items, recently recycled items due to time constraints
    • It was difficult to find similar materials without using AI or hardcoding, and it was hard to collect and aggregate all of our data for the commonly recycled items, commonly misrecycled items, and recently recycled items. These were all purely nice-to-have features and not necessary for our original impact case, so we ultimately decided that we did not need to overcomplicate our app.
  • Decided to add more textual labels and user action indicators in places relating to the Material concept
    • Users did not navigate to the Materials Info page or were surprised about the page when they accidentally navigated to it. To ensure the app behaves as the user expects, we added cursor indicators to places the user can click to navigate to the Material Info page and labeled each material instead of relying on the user to know what material each image is showing.
  • Implemented bin type filtering on the map
    • When users clicked on See disposal locations for a certain item type (Recyclable, Compostable, Solid Waste, Donatable), they expected to see only the bins for that type. However, our map was showing all the bins, without any filtering which was not what the user was expecting.

Demo

A link can be found here.