Vive Video

Research Process.

Application Walkthrough.

In order to carefully conduct a powerful usability study, we need to thoroughly understand the application, its workflow, UI elements, and system design first. So, we started our study by creating the interaction map to see the flow and connections in the whole application.

Designing Usability Toolkit

HTC would like early feedback on how well their application is received by users. Our goal is to explore the intuitiveness of the UI - whether the user can recognise icons and menus and whether they can easily navigate and perform tasks. We also want to capture any subjective feedback users have on their experience. This will be obtained via interviews, questionnaires, and using the think-aloud technique during the tasks.

Our objectives include understanding any user pain points. This includes discovering blockers, confusion, or distractions users have when performing tasks; identifying any non-intuitive designs and discoverability issues, such as the graphics or layout; and any comparative or subjective feedback users have on the experience of using this application.

Data Collection.

Data in our study was captured and processed across different mediums. The variety of approaches ensured that the data analysis could be supported by more than one perspective. Techniques and their motivations include:

• Questionnaires:
  An online form with multiple choice and open-ended questions. The answers provided quantitative data about the user’s expectations and experiences before and after the interactive session. We explore whether the user’s interpretation of the icon match the application’s intention, and the user’s comfort with the physical controls and the Vive Video VR environment.
  
• Video Recording of the Room:
  Video was recorded using a Logitech webcam to capture the user’s gestures and commentary during the interactive session. The video provided qualitative data about the user’s comfort levels by observing and comparing what they say and do.
  
• Application Recording:
  Camtasia Studio was used to capture the screens seen through the Vive headset. Both qualitative and quantitative data could be obtained, this showed how long the user took to perform the tasks and their attempts they make along the way. Paired with the think-aloud technique, captured in the video recording, this mapped physical and mental frustrations to actions made in the application interface.
  
• Observation:
  Camtasia Studio was used to capture the screens seen through the Vive headset. Both qualitative and quantitative data could be obtained, this showed how long the user took to perform the tasks and their attempts they make along the way. Paired with the think-aloud technique, captured in the video recording, this mapped physical and mental frustrations to actions made in the application interface.
  
• Voice Recording:
  A voice recorder was used during the interview to capture dialogue. This allowed the interviewer to focus on the conversation. Recorded data was transcribed and provided qualitative feedback in the form of user quotes around their experiences.
  
• Interview Notes:
  The interview allowed the participant to reflect and discuss their experiences from the VR study. Notes were used in the sessions to improve the line of questioning and to highlight areas to explore in subsequent interviews.
  

Usability Study Session.

We selected users who were experienced with the HTC Vive. Familiarity with Vive products and the VR application market enabled their feedback to be relevant to Vive Video’s consistency across other HTC VR products as well as competing VR products. Additionally, experienced users allowed the study to focus on the implementation of the application rather than being distracted by the VR experience itself.

Study Timeline.

Participants.

Testing Environment.

Actual Environments.

Key Finding.

We categorized our key finding based on source of issues and 4 level severity criteria from Nielson Norman Group. The results from our study showed that there are 15 issues in total. There are 2 usability catastrophic problems one in technical and another in design aspect. 5 issues under major usability problems 5 in minor, and 3 in subtle. We also included the most 3 common recommendations from users into our key findings.

Example of Key Findings.

Recommendations.

we have categorized our usability issues into bugs, design issues, and user feedback. We found that while some bug usability issues had high severity levels, they cannot be eradicated by design recommendations. Rather they are problems that must be amended in the next release to bring functional improvement to Vive Video. On the other hand, user feedback issues had very low severity levels because theses did not have strong relationship with watching the videos in comfortable manner. Therefore, we chose to focus our usability recommendation at high severity design issues. The identified usability issues are due to poor visual and interaction design. We propose following solutions to improve user interaction for comfortable VR cinematic experience. The design recommendations were inspired from our findings, participant observations, and our own interpretation of the interaction map.

What I learnt from this project.

• Studying / analysing interface and interaction components of Virtual Reality software and hardware
• Creating the questionnaire and usability test kit to capture user experience with Virtual Reality world. There were many aspects that we needed to take into consideration when conducting the test. What were right questions and approaches that can measure user's immersive experience. For example We tried to have minimal interruption during the Vive Testings ( but also wanted to know what the users were seeing, touchig, feeling, and thinking at this time)

Future Work.

• Video Content & Time : usability on longer media (Movie, TV series)
• # of Users : study on bigger audience
• Extended Icon & Typography : test on preference and productivity of icon sets and typography