Project Phase 6 ​

User Test Report ​

2 Students and one adult were consulted to test usability of TuneTrainer.

Report User 1 ​

Notable observations about gulf of execution and gulf of evaluation, by task

• Task: Song creation and the saving of songs to an existing collection
  • Gulf of execution & evaluation
    • Conclusion: gulfs are small
    • Evidence:
      • Thanks to loading status indicators and automatic transitions of the website to show the UI elements relevant for next steps, the gulfs of execution and evaluation were small for Song creation and the saving of songs to an existing collection. User completed the 2-step process for creating a Song note quickly. They had no questions about the various states of the system during the song generation process (such as the “loading” state when the song was being generated, and the state where the song had finished generating); The user knew when they could begin viewing the generated song
• Task: Saving a songified note to a collection when no collections exist
  • Gulf of execution
    • Conclusion: gulf is medium
    • Evidence:
      • Multi step process was required for user to realize what action should first be taken when saving a song since The app did not communicate well that a collection must first exist before a song can be saved: The “add to collection” button at the bottom of the song had no effect when no collections existed for the account; Confused by the button behavior, the users gave up on attempting to add to a collection and tried to “save” the song directly through the “save button”; it was only when the save button instructed the user (via notification pop up) to create a collection that the user realized a new collection had to be created before the song could be saved * the user did not notice the accelerator “create new collection” button at the bottom of the song until the pop-up instructions appeared
  • Gulf of evaluation
    • Conclusion: gulf is small
    • Evidence:
      • In their self-narration, commented that they recognized the collection they created and could Identify their recently created song within it

Report User 2 ​

This user test revealed the same positive and negative feedback as the previous user. Below is a subset of challenges faced by user 2 not yet mentioned above.

• Task: generating a song: restarting the song generation after having recently generated a song
  • Gulf of execution:
    • Conclusion: gulf is large
    • Evidence:
      • After having previously generated a song, The user did not notice the means (even though there was one: a “discard” button) to regenerate a song after making edits to the input aside from reloading page
• Task: using Study tool
  • Did not pass the study tool game during the session, neither did the user make incremental progress towards successful study-tool completion
    • Why this occurred: the expectations of the study tool are too difficult: expecting a user to memorize specific word-for-word details of a large (the song generated is long) body of text; and there was no aid provided to the user to make incremental progress towards the study tool game’s goal.
  • Flaws pointed out by the user
    • Desired an easier goal: Desired that study tool goal would be either more feasible or be more incremental (to memorize smaller sections of a song rather than the full song)
    • Desired an easy way to get hints / reference the correct answer for the song while using the tool. Currently there is no way to study the song while using the Study tool, so the user has to travel back and forth between Song and Study Tool pages to complete the study tool.

Report User 3 ​

This user test revealed the same positive and negative feedback as the previous users. Below is the subset of challenges faced by user 3 not yet mentioned above.

• Task: Looking back at generated songs
  • While the user was able to sing many song lines in time with the music, near the end of the song, the lyrics and tune did not match up
    • Why this occurred: The generated song lyrics sometimes had more verses than the background tune
  • Flaws pointed out by the user
    • Lack of clarity in how the generated lyrics lined up with the background music; Would have appreciated “verse” and “chorus” labels, and would appreciate, for ease of reading, having the lines broken up by verse
    • The generation of non-student/non-school-friendly lyrics; Curse words were present in the generated song
      • Why this occurred: the method of creating the lyrics (via providing an existing song to ChatGPT) referenced a source with curse words

Opportunities for improvement ​

• Opportunity
  • Flaw description: No users passed the Study Tool
  • Why the issue occurs:
    • The expectations of the study tool are too difficult: expecting a user to memorize specific word-for-word details of a large (the song generated is long) body of text; and there was no aid provided to the user to make incremental progress towards the study tool game’s goal.
  • Class: Conceptual
  • Severity: Major
  • Possible Solution:
    • Make the study tool goal to be either more feasible (e.g. for the user to recall the general ideas of the song rather than specific word choices) or be more incremental (to memorize smaller sections of a song rather than the full song)
• Opportunity
  • Flaw description: User 1, an adult, wasn’t familiar with the tunes for some of the songs provided as options for background music to his notes (because he was not familiar with all the names); user preferred to select an option he was familiar with
  • Why the issue occurs:
    • The website assumes that the names of the background tunes are enough information to provide users with to make an educated choice for background
  • Class: Linguistic
  • Severity: Medium
  • Possible solutions:
    • Provide some information scent to a user when picking their background music for their generated song so that they know the tune and can make an informed decision based on tune
• Opportunity
  • Flaw description: Multi step process was required for a user to realize what action should first be taken when saving a song: that a collection must have been created before a song can be saved.
  • Class: Linguistic
  • Why: See task “Saving a songified note to a collection when no collections exist” in User 1’s report
  • Severity: Minor
  • Possible solutions:
    • Reduce the amount of time it takes the user to identify that a collection must first exist before a song can be saved (either via the way the “add to collection button” behaves when no collection exits, or via an automatic redirection to the collection creation page)
• Opportunity
  • Flaw Description: Difficulty for users to study a generated song due to lack of voiced audio. Singing it to oneself, while a workaround, is not well supported by app due to non-obvious timing of words with the tune
  • Class: physical
  • Why: Minimal information is provided about how the words in the generated song are intended to line up with the tune; Some queues in the lyrics exist (e.g., if the user was familiar with the original lyrics that a song was based off from, and if the generated lyrics appeared to be similar to the original song’s lyrics, the user had a hint about what part of the tune lined up with the familiar lyrics)
  • Severity: major
  • Possible solutions:
    • An animation that indicates the timing of words in the generated song to match appropriate parts of the tune
    • Break the song into smaller sound bytes and show which sound byte corresponds to which line

Design Revisions ​

Collaboration Concept ​

An idea we were planning on having early on in P3 is the concept of Collaboration, where other users can recommend songs to add to the owner’s collection. Unfortunately, wiring the major infrastructures of the more important concepts, like Notes, Sharing and Collection, took longer than we anticipated.

Explore Tab ​

Originally, we had user, shared, and public collections all under the “collections” tab but moved public collections to a new tab called “Explore.” As collections now only shows user-specific content, we believe this change makes more sense.

Study Tool ​

We were envisioning the study tool to be something like Quizlet, where the song’s “answers” (or lyrics) can be flipped to understand the underlying notes. The final study tool ended up being a “fill-in-the-blank” assessment that increases frequencies of a note by how often the user gets it wrong.

Edit Buttons ​

Originally, we wanted users to be able to edit songs but realized that because the option to alter notes/lyrics is given during the song generation step, it is not completely necessary. The lack of editing buttons lets us prioritize the design and routing in the site, rather than spending too much time on forms and API calls.

Create Collection #2 ​

Originally, the “generate song” page only let you make a song and add it to a collection. Our TAs suggested that if a user was not sure which collection to add their new song to, a nice shortcut could be an option to create a collection on that page, as well.

InnerSongComponent ​

At first for the alpha release, our component for a song note’s view inside a collection had two columns, where one half is the lyrics and the right side has an audio element for the associated background music. As the lyrics can get quite long, however, we switched this design to have two columns of text side by side, with the audio on the top right, as it is a much smaller element.

Color palette ​

In the P3 visual study, we were planning to go for a combination mix of the light pastels used in learning apps and the dark background, bright highlight palette for the music aspects of the site. However, as we implemented the concepts, music representations (such as the audio HTML element) became a lot less significant, so we kept the general light, pastel theme all throughout.

Font choices ​

We iterated through several bubbly Google fonts for major labels, such as Oswald, Rubik Bubbles, Rowdies, etc. but eventually settled on Righteous due to its relative conformity with the Arial body font.