{"id":9695,"date":"2017-06-02T11:00:39","date_gmt":"2017-06-02T10:00:39","guid":{"rendered":"https:\/\/marvel7077.wpengine.com\/?p=9695"},"modified":"2020-09-03T12:00:22","modified_gmt":"2020-09-03T11:00:22","slug":"creating-user-flows-mixed-reality","status":"publish","type":"post","link":"https:\/\/marvelapp.com\/blog\/creating-user-flows-mixed-reality\/","title":{"rendered":"Creating User Flows for Mixed\u00a0Reality"},"content":{"rendered":"

The rules for mixed reality content creation are still in flux. This is an exciting time to be working in mixed reality because there isn\u2019t a standard way of designing yet. Once more content is created, the audience for mixed reality will grow, and we\u2019ll be able to see which designs work and which don\u2019t.<\/p>\n

Working as a group named Team HoloGlass, my teammates and I designed and developed a holographic artist portfolio for the Microsoft HoloLens<\/a>. Our portfolio allows users to interact with 3D holographic artwork by placing the artwork in the real-world environment, adjusting its size, and rotating it. Users can also see information about the artwork, as well as details about how the artwork was created.<\/p>\n

\"\"<\/a>

A user places a holographic art object on a real wall. This is what selecting an interactive installation looks like in our mixed reality art portfolio.<\/p><\/div>\n

<\/p>\n

A typical user flow<\/a> for web and mobile applications only describes what happens when a user clicks something. In a mixed reality experience, there is much more going on than clicking and viewing. In a HoloLens mixed reality experience, these these basic interactions make up the fundamentals of the experience:<\/p>\n

    \n
  1. Airtap<\/strong>: User selects a holographic object by tapping their thumb and index finger together while pointing at the object<\/li>\n
  2. Double Airtap<\/strong>: User taps their fingers together twice. Similar to a doubleclick on a mouse.<\/li>\n
  3. Bloom<\/strong>: User raises their hand, holds their thumb up, and touches their thumb to their middle finger in a \u201cbloom\u201d gesture. Similar to the Italian hand gesture for capisce<\/em>. The bloom is used as a way to go back or to return to the home menu.<\/li>\n
  4. Voice Command<\/strong>: Users can interact with a HoloLens application by speaking. Users can also use voice commands to access Cortana, Microsoft\u2019s digital assistant, anytime during their HoloLens experience.<\/li>\n
  5. 360 Sound<\/strong>: HoloLens applications use 360 sound. Games, music, movies and more can be experienced in mix.
    \n
    <\/div>
    <\/svg><\/div>

    \"In a mixed reality experience, there is much more going on than clicking and viewing.\"<\/p><\/span><\/blockquote>\n<\/li>\n<\/ol>\n

    \"\"<\/a>

    A HoloLens user watches a sports game while viewing the weather and pulling up a cooking app.<\/p><\/div>\n

    All of these interactions are much different from web and mobile interactions. They call for a new kind of user flow.<\/p>\n

    \"\"<\/a>

    This is a user flow for how a person experiences an interactive installation in our holographic art portfolio. Notice how we have placed a script next to a visualization. Next to the visualization, there is text describing what is happening and icons indicating potential actions. At the bottom of our user flow is an icon \u201ckey,\u201d which shows which actions are available in any given window. Another full user flow can be seen at the end of this article.<\/p><\/div>\n

    <\/p>\n

    This is a user flow for how a person experiences an interactive installation in our holographic art portfolio. Notice how we have placed a script next to a visualization. Next to the visualization, there is text describing what is happening and icons indicating potential actions. At the bottom of our user flow is an icon \u201ckey,\u201d which shows which actions are available in any given window. Another full user flow can be seen at the end of this\u00a0article.<\/p>\n

    Here are 6 ideas to keep in mind when creating mixed reality user\u00a0flows.<\/h2>\n
      \n
    1. Storyboard!<\/strong><\/li>\n<\/ol>\n

      In mixed reality, your user\u2019s journey is a story, and you, the designer, are the author.<\/p>\n

      \"\"<\/a>

      A sketch of a user using a user selecting an artist portfolio to view.<\/p><\/div>\n

      <\/p>\n

      <\/div>
      <\/svg><\/div>

      \"Your user\u2019s journey is a story, and you, the designer, are the author.\"<\/p><\/span><\/blockquote>\n

      Sketch out your story in third person and first person. Sketching in third person allows you to view the experience at a distance, and can help you describe the big picture components of the experience you\u2019re creating. Sketching in first person allows you to understand the field of view of the device you are creating for. Sketching from the user\u2019s perspective lets you make sure that you are including everything that you need to in the user\u2019s line of vision. It allows you to place each element of your experience in a thoughtful way, since you are likely designing an application with 360 movement (i.e. the user has to move her head around while wearing the device to see everything that is available).<\/p>\n

      \"\"<\/a>

      A first person view of an artist\u2019s portfolio in our application.<\/p><\/div>\n

      <\/p>\n

      <\/p>\n

      We found inspiration for mixed reality storyboarding from many articles on Medium about sketching for VR. Virtual Reality Pop\u2019s article \u201cHow to Sketch for VR Like a Pro<\/a>\u201d was especially helpful.<\/p>\n

      2. Describe Every Component and Available Interaction in Your Experience<\/strong><\/p>\n

      \"\"<\/a>

      A spreadsheet we made to describe the nitty gritty interaction details of our mixed reality art portfolio experience. Notice how a script is written for the instructional audio column. Also note the physical space column.<\/p><\/div>\n

      <\/p>\n

      <\/div>
      <\/svg><\/div>

      \"Describe Every Component and Available Interaction in Your Experience\"<\/p><\/span><\/blockquote>\n

      Designing in mixed reality means you\u2019re designing interactions related to sound, voice, sight, and gesture. A good mixed reality experience incorporates all of these elements (this is part of what makes mixed reality so cool!). You must design for a variety of \u201cwhat if\u201d experiences in each window (e.g. What if the user selects the object via an air tap? What if the user selects the same object via voice command? What if the user doesn\u2019t hear the audio instructions the first time they play? What if the user double airtaps instead of single airtaps?).<\/p>\n

      With all these moving parts, it can be difficult to keep track of everything. We recommend writing everything down. Kind of like a story! Ask yourself the relevant \u201cwhat if\u201d questions for your experience, and answer them. If your mixed reality experience is complex, make a spreadsheet with the what if questions and answers for each window. When you\u2019re making your final user flow, it will be useful to return to this document to reorient yourself and stay focused.<\/p>\n

      Also, make sure to describe the physical space<\/strong> in your experience. Mixed reality is great because digital objects interact with the real world environment. It\u2019s crucial to describe the real world environment when outlining your designs, otherwise context can get lost.<\/p>\n

      3. Describe What is *Not* Happening in the Experience<\/strong><\/p>\n

      If you don\u2019t describe which components are not<\/strong> available in your mixed reality experience, the person you\u2019re communicating your design to will assume *everything* is available in *every* window of your application.<\/p>\n

      For example, if your application uses instructional audio in some windows, but not all, show the absence in your flow. Describing these components is key when showing your work to stakeholders.<\/p>\n

      \"\"<\/a>

      Notice how we indicated that no instructional audio is playing in this section of the \u201cPainting\u201d user flow. The green circle is used for emphasis in for this article.<\/p><\/div>\n

      4. Bodystorm<\/strong><\/p>\n

      It\u2019s challenging to express a mixed reality 3D experience in a different format. But it\u2019s a challenge worth tackling. Before you spend tons of time developing in Unity, map out your experience in other ways.<\/p>\n

      Try bodystoming (i.e. acting out your experience with a group of people). You\u2019re designing 3D interactive objects. Humans are also 3D interactive objects! Have one person put on a makeshift headset<\/strong> (we used a sandwich box) and pretend to be the user.<\/strong> The headset is a must, because it allows you to mimic the field of view of the device you\u2019re designing for. Have a second person pretend to be the object<\/strong> the user is interacting with. Have them respond to the user\u2019s gestures accordingly. If you have text in your mixed reality experience, write down the text on paper and use it is a prop\/object. Have the user run through the \u201cwhat if\u201d scenarios you\u2019ve written down. Act out what the user hears, what the user can say (voice command), and what the user can do with gestures.<\/p>\n

      \"\"<\/a>

      Team member Levi puts on the cardboard HoloLens.<\/p><\/div>\n

      <\/div>
      <\/svg><\/div>

      \"Before you spend tons of time developing in Unity, map out your experience in other ways.\"<\/p><\/span><\/blockquote>\n

      <\/p>\n

      \"\"<\/a>

      Team member Levi puts on the cardboard HoloLens.<\/p><\/div>\n

      <\/p>\n

      \"\"<\/a>

      The first person field of view from our cardboard HoloLens.<\/p><\/div>\n

      <\/p>\n

      Doing this is the quickest and most effective way to get a sense for how your experience will feel<\/em>. Bodystorming will allow you to design a natural feeling flow that is intuitive. It will also help you understand how to improve the clarity of your experience from the user\u2019s perspective. Creating a makeshift headset builds deep user empathy, allowing you to understand what the user feels like when they have a limited field of view. Lastly, bodystorming helps you see which interactions are essential and intuitive, and which are not.<\/p>\n

      \"\"<\/a>

      The sandwich box in all its glory.<\/p><\/div>\n

      5. Prototype in 2D First<\/strong><\/p>\n

      Prototyping in 2D is not the best for user testing, because the actual mixed reality experience will be drastically different. However, it is incredibly helpful for determining flow.<\/p>\n

      Nail down all your interactions in a 2D, clickable prototype. We used Marvel. This will save your developers time and energy \u2014 they will be thankful!<\/p>\n

      \"\"<\/a>

      This is a Marvel render of what it would be like to watch a video in our mixed reality portfolio app. Each icon is clickable and brings the user to a different window.<\/p><\/div>\n

      Prototyping in an app like Marvel is more clear and concrete than bodystorming. While bodystorming helps you design feeling, Marvel helps you design for usability. Make images clickable to represent the holograms in your mixed reality experience. When there is audio or text, write it out in your Marvel window. It\u2019s okay if your Marvel window gets cluttered with text. Creating a 2D interactive prototype for a mixed reality experience isn\u2019t about making something beautiful for your user tests. It\u2019s about making your user experience clear, and planning for all the different answers to the \u201cwhat if\u201d questions you\u2019ve been working with. A 2D prototype is a pragmatic, time-saving approach for designers who need to describe detailed concepts to developers. The 2D prototype serves as a roadmap for your developers to follow, and lessens the chance that there will be major mistakes or communication errors.<\/p>\n

      \"\"<\/a>

      Note the instructional audio written on the left. This could be even more detailed if an exact script was written instead of placeholder text.<\/p><\/div>\n

      Here is the Marvel prototype:<\/p>\n