F-Droid Lubbock Report – What We Want to Know

 Posted on April 17, 2017  |  Carrie Winfrey

F-Droid LBK Usability Study Report – What We Want to Know

Prepared by Carrie Winfrey

Preliminary Version – April 17, 2017

Introduction

When planning this user test, the team outlined features and flows within the app on which we wanted feedback. From there, we created tasks for participants to complete that would access these areas, and produce insights related to our inquires.

This document is organized by the tasks participants completed. Initial inquiry questions are outlined under each task, followed by the feedback and observations gained from the test. Last, within each section, I’ve listed suggestions for improvement related to the task.

In the Lubbock sessions, 11 people participated in this focused usability study. More information on participant demographics, test methodology, metrics, and follow-up activities and insights will be provided soon.

PAGE_BREAK: PageBreak

TASK A – Search for an app that you would like to download. Install the app you have selected.

Average difficulty: 1.9

Number of participants with critical errors: 0

PARTICIPANT QUOTES

“App won’t install. Found the game easily, though.”

__

__

What we want feedback on

  1. Are users able to locate the search button?
    • Yes. Most users found the search button right away.
    • Some (3 out of 10) browsed through the latest or category view to find an app instead of using search.
    • One participant said he never saw the search button.
  2. Are they able to effectively use search?
    • Yes.
  3. Are they able to find what they’re looking for?
    • Not always. Searching for mainstream apps like Gmail, ESPN, Facebook and Twitter returns no relevant results.
    • “It took about 10 tries to find an app I searched for. It might be difficult for users to find the app they have an interest in.”
    • One participant searched for recipes, but misspelled it, so no relevant results were displayed.
  4. Other insights:
  5. Many users expressed that it was difficult to know if and app had installed.
    • “I found it difficult to figure out if the apps had installed. No notification?”
    • Some participants searched on the home screen of the phone, then in the apps on their phone to see if the app had installed.
  6. 4 participants were unable to install apps all together due to a bug in the app version being tested.

Suggestions for improvement:

  • Location of search
    • Though most people didn’t have trouble locating search, a majority are most familiar with finding it at the top of the screen.
  • Search results
    • Display more relevant search results. Ex: If someone searches for Gmail and we don’t have it, offer alternative email apps.
    • Use suggestive results to help users avoid spelling errors and to make the action of searching quicker.
  • Feedback once installed
    • Provide better feedback once an app is installed for views within the app (installed from list view, app details and updates views).
    • Show a system notification when an app is installed.
    • Give the option to add installed apps to the home screen of the phone.
  • Fix bugs Fix bugs with installing apps.

PAGE_BREAK: PageBreak

TASK B – There is an app that needs to be updated. Find the app and install the update.

Average difficulty: 2.3

Number of participants with critical errors: 0

PARTICIPANT QUOTES

“The update icon at the bottom with red number notifications was easy to find.”

“It took me multiple attempts to update the applications.”

__

__

What we want feedback on

  1. Is it clear where to go to find updates?
    • Yes. Every participant easily located the updates tab.
    • “It was very easy to spot apps that need updating.”
  2. Can users successfully update an app?
    • No.
    • The difference in word choice on app details than in updates view confused multiple users.
      • “The update icon at the bottom with red number notifications was easy to find. Once I clicked on the specific app to update it gave the option to uninstall or upgrade. I wasn’t sure if upgrade was the same as update or not – it sounded like something I’d have to pay for.”
    • Every participant that tried to update an app from the updates view had problems.
      • “Took me multiple attempts to update the applications. Not positive the update occurred. There was no indication of completion.”
    • It was unclear if an app had successfully updated. Participants were not confident that they had updated the app.
    • We need to provide better feedback once an update is complete and successful.

Suggestions for improvement:

  • Word Choice
    • Change ‘upgrade’ to updates in app details.
  • Feedback
    • Provide better feedback once an update is complete and successful within the updates view.
    • Give a system notification when an update is successful.
  • Fix bugs
    • I’ve shared a screen record that reveals the problems.
    • For some users, the red badge indicator didn’t change after they updated the app. This behavior needs to be tested internally.

PAGE_BREAK: PageBreak

TASK C – Find a game app that looks interesting to you. Install the app.

Average difficulty: 1.7

Number of participants with critical errors: 0

What we want feedback on

  1. How do users browse when looking for a specific kind of app, like a game? Do they search, look in categories, browse, etc.
    • Most participants looked through the games category. Some browsed latest. Some used search.
    • Many people found the categories view appealing, and spent more time there than on the latest view, when looking for apps. One participant compared it to the iTunes Store, but commented that it was even easier.

TASK D – Search for a crossword puzzle app within the games category.

Average difficulty: 1.1

Number of participants with critical errors: 0

What we want feedback on

  1. Are users able to effectively search within categories?
    • We saw a variety of behaviors with this task. But, overall, people were able to do this without problems.
      • 4 participants searched in within categories
      • 2 participants searched from search
      • 2 participants browsed in horizontal view of the category
      • 2 participants opened category and scrolled down to find

Suggestions for improvement:

  • Easier use of chip Once you clear a chip, there’s no easy way to get it back if you want to search within that category.

PAGE_BREAK: PageBreak

TASK E – If you had no internet, how would you download an app?

Average difficulty: 3.1

Number of participants with critical errors: 7 of 10 that attempted the task were unable to identify the nearby feature as the place to download apps without internet.

PARTICIPANT QUOTES

“Nearby didn’t give me the idea of downloading while offline.”

__

__

What we want feedback on

  1. Are users able to discover how to download apps offline?
    • A few. But not the majority.
  2. Is it clear that you use they nearby feature to do this?
    • No.
    • 70% of participants never identified the nearby feature as the means for doing this.
    • “Nearby didn’t give me the idea of downloading while offline.”
    • 30% successfully identified nearby. Though, most after looking several other places?
  3. What other behaviors are observed?
    • Turning WiFi off to see if the interface changed
    • Going into airplane mode
    • Downloading an airplane mode app
    • Opening settings and looking in WiFi options
    • Searching google.

Additional Observations

  • People did not make the connection between ‘nearby’ and downloading apps offline.
  • Not one participant read the text on the Nearby view when they initially tapped on it.
  • Multiple people compared nearby to Air Drop on iOS
  • “It’s going to airdrop a virus to your phone.”

Suggestions for improvement:

  • Word choice
    • Reconsider the word choice.
  • Offline mode
    • The experience of the app should be obviously different when there’s no internet connection.
    • Guide users toward the nearby feature if there’s no connection.
    • Consider providing info in settings
  • More engaging visuals
    • If we want people to pay attention to the Nearby view, it needs to be more engaging.
    • If we keep the language about offline, consider using iconography that communicates that.

PAGE_BREAK: PageBreak

TASK F – Your friend emails you a link to a collection of their favorite apps. Open gmail and click on the link they have sent you. Add the collection of apps that your friend emailed to you to F-Droid. Locate an app called Habitica from the collection you just downloaded.

Average difficulty: 2.2

Number of participants with critical errors: Every participant was able to add add the repo. 4 of 11 participants were unable to locate the Habitica app that was added from the repo.

PARTICIPANT QUOTES

“I was able to find the link the email and accept the apps, but have no idea where to find the collection from my friend in F-Droid.”

__

__

What we want feedback on

  1. Are users able to add a repo from a link opened on mobile?
    • Yes. No one had problems with this.
  2. Is the webpage for adding a repo clear?
    • Yes.
  3. Are users able to find the apps from a repo they have added?
    • No. Participants were very confused.
    • 100% participants didn’t know if they had added the repo successfully after completing the steps.
    • “I was able to find the link in the email and accept the apps, but I have no idea where to find the collection from my friend in F-Droid.”
    • Some were able to find one app by searching for it.
    • Average non-critical errors for finding the app—2
      • 5 users were able to locate the app; most after multiple attempts
    • Average critical errors for finding the app—3.73
      • 3 users were unable to locate the app added from the repo
    • Other places people looked
      • In categories thinking the latest apps added would be shown first
      • In the latest view
      • In settings > Manage installed apps
  4. Do users understand that the apps added from a repo are only made available to them, and not auto-downloaded and installed?
    • Not really. It was hard to know for sure since the users never saw a list of those apps on the phone once they added the repo.
    • It seemed that some participants were surprised to see that they still had to download the app Habitica after finding it via search.
    • Many participants assumed the apps from a repo would download.

Suggestions for improvement:

  • Feedback
    • Currently, there’s no indication of completion.
    • The experience of adding an app cannot end after they tap add. People are way too confused. Also, currently after tapping add, the app returns to the screen they were previously on within the app, which could be anything.
  • Make apps accessible
    • The user needs to immediately be able to see the apps they just added to their phone. Note: it should be clear that these are made available, but not downloaded automatically.
    • The user needs to be able to access the apps within a repo later on.
  • Handle delay
    • Is there a delay in the repo updating? We noticed that some people tried to search for an app that was within a repo they added, but even then, it didn’t show up.
    • Update only the repo that was added instead of all. Not sure what the current behavior is.

PAGE_BREAK: PageBreak

TASK G (Last Task) – Add the collection of apps displayed on the computer to F-Droid.

Average difficulty: 2.6

Number of participants with critical errors: 0

PARTICIPANT QUOTES

“I followed the steps on the computer but wasn’t sure where the code from the QR scanner went. I added a new repository and there was a link already in it so I just clicked add, but have no idea what it did.”

__

__

What we want feedback on

  1. Are users able to add a repo from a collection they view on desktop?
    • Yes. Surprisingly, the majority of participants did not have trouble scanned the QR code and adding the repo.
    • Same confusions as mentioned above about knowing if it worked and locating the repo.
    • “I don’t know if it worked. How do I know if it worked?”
  2. Is the word choice clear?
    • Somewhat. It didn’t trip people up from completing the task.
    • Multiple people commented that the word repository didn’t resonate with them. They preferred the word Collections.

Suggestions for improvement:

  • Fix bugs F-Droid crashed a lot after people scanned the code, opened F-Droid and tapped add. I think this was due to the phone being in landscape then portrait mode.
  • Integrate QR code scanner within the add repo view.
  • Word choice

PAGE_BREAK: PageBreak

Additional Observations

  • Concerns of Malware While most participants were not keen on security and the potential risk of downloading apps from a source other than Google Play, one participant was keenly aware. He made multiple comments about how the app is an open door for malware.
    • **We need to address the security risks, and what F-Droid does to mitigate those risks. We need to be open and transparent in our communications to build trust.
  • Tablet Optimize the latest view for tablet. Show more grid tiles horizontally (i.e. display 3 instead of 2 in portrait mode) and increase font size.

More information coming soon!

  • Methodology (Note: All tasks were completed in different orders by the participants with the exception of Task G, which was always completed last.)
  • Participant demographic
  • Desirability toolkit
  • Follow-up survey results
  • Usability metrics & stats
Android  bazaar  F-Droid  fdroid  open source  prototype  usability