Designing Virtual and Augmented Realities is a new art, and the production skill set takes managing a variety of very specialized workers. There is no single profession that is fully equipped to tackle the new challenge, but film makers, playwrights, movie producers, visual effects directors, and scrum masters are some of the leaders in the pact for diving into the new medium, applying the knowledge and adapting as best possible. There are many unique considerations when designing. Especially when you consider most of the above professions weren’t heavy in the soft-ware game until now. Therefore, we want to avoid forcing AR and VR designs to fit old mold ands really appreciate it for the potential it holds. Let’s not suffer the limitations of its predecessors. The major considerations we will make VR or AR are to design on (1) Human comfort, (2) Environment Building, (3) Human Experience and (4) (Tech Specs.
Environment building has two key areas to serve when building for the user. These are attention and immersion and exploration. When someone is playing in a VR or AR, the are immediately “In the environment” and the experience starts as the headset is domed. Loading screens, menu options, opening design, and everything that occurs after the all white compositor room (known as an upload-platform) is a part of the user’s enjoyment or displeasure. Therefore, we must enable a sense of presence and care for the small details that provide either authenticity or dissonance, and opt for the former. The user will walk around, they will explore, they will experiment with six degrees of freedom in a 360º space – we must always be aware of this.
Our content needs to cover everything, and that includes the ability to engage the content. If a user goes to pick up and item they would in the physical world, but they cannot I this virtual world, then the presence will be broken. If they toss an item, it should be tossed, not stuck to their hand or just dropped to the ground. Even if it is not totally realistic, we should be providing them with a good-enough example of the object be projecting to offer the satisfaction that the surroundings can be engaged as they expect them to. As the user tests the limits, the environment should react relatively within the current realm of expectations, where things are handled, moved and broken in relation to the force applied.
Games like Job Simulator do a great job at making everything in the room interactive. And they went as far as considering the different ways in which someone would accomplish a simple task, like making tea. Though for the most part, we all make team about the same, we might order the tasks differently. If I always make tea by first boiling water, and I am not able to in the VR or AR experience, until I’ve selected my tea, then then I am rushed to realize I am but in an environment where my senses are being artificially stimulated. I suddenly remember this is all just a computer simulation. This is the kind of deflation we feel when we realize our dream is but a dream and not life. That disenchantment of finding out Santa is not real. That void of this is not the “real-world”, and the consequences of gloom that are baggage with it. and Augmented Realities share this undertone of believing the environment and when you betray that trust you lessen the experience. Don’t betray your users trust.
Lastly on our environment build, we segue into Human Comfort by being cognizant of the size of our space. If we design the environment not too small, and or too big we can make the user uneasy, even afraid. Spaces too tight and closed off can induce claustrophobia or whereas spaces too open and infinite can induce agoraphobia. Neither of these are desired unwittingly. The reversal of this guideline is realizing that using spaces that provoke fear, fear-triggers if you will, as part of the storyline or the narrative of our content can be very powerful. As we engage biometric measures we will becoming more and more adept to manipulating the user feelings, not just elevating and elating the user, but making them feel fear, or even terror. But, of course these tricks will take skills and in time, with increased attempts, the skills will develop and maybe even be mastered.
A simply a smooth design can do far more for the user’s comfort and overall experience than the amount of space they have for play. A destination the size of a single room can be infinitely engaging and superior to an experience that is partly interactive but goes on and on. Design the environment for optimizing comfort by making it simple, purposed and precise. These designs are the best approach. Focus on quality and then we will build quantity with artificial intelligence generatively designing more spaces for more play. Maybe these will even be created in real time based on mood and feedback loops.
Part of quality design is our 3D models being the appropriate sizes. Textures are important too, but more critical is the size of the item. If a baseball is the size of a golf ball, then presence is broken. But like before, if done tactfully, this can be intentional.. In virtual and augmented space we can be any size we want. Maybe we are a giant romping across huge terrains, where trees should be small. Or we are shrunk to the size of an insect and we explore our computer’s interior mechanics. Of course, it depends on your end goal but keep in mind that objects of unrealistic proportion can break presence.
Other environment designs include what is the user is seeing now that he/she will not see later? Maybe something that was previously not in view becomes the sole focus of attention and fully engaged as the only item the user realizes for some duration. When we design to should be for the user to look wherever, whenever, even if that is not the norm. If we provide an environment that allows for curiosity and rewards their experimentations will will bring them happiness and satisfaction. People like having their curiosities rewarded. They like having control. When they are in control of their own story, or at least as far as they are concerned, they perceive the experience more fondly.
We must keep this ongoing relationship, where the user is the alpha and the world is responsive, reactive and catering to their moves. So how do we get their attention when we need it? We need to go about it my psychologically rather than forcefully. It would be jarring, disturbing and unpleasant for the user’s view to change if they didn’t initiate the change. We cannot force our users to look somewhere. Instead, we suggest they look somewhere by speaking to their subconscious and peaking their curiosity. They see something out of their peripherals that is of interest (relative to the rest of what is within view) and they will look that way. We can estimate that 80% of the time they will be drawn to partial information in a specific direction that minimal-to-no information that is right in front of them.
Additionally, audio and cues can compliment the visuals. Where visuals like a flying bird or an emerging light can grab the user’s attention and lead it down a path, audio cues can do the same. If a monster is invading from behind you can have a voice scream “OMG! BEHIND YOU IS A GIANT DIGUSTING MONSTER!” But that cue is obvious. More subtly, we hear his foot steps creeping closer, louder, crisper with each haunting step! It is clearly coming from behind, and the user turns to see what can be approaching. Soundscapes and audio tracking are of the most important when designing engaging environments. These are the subtle cues that deepen feelings of presence. Because audio has an intrusive way of entering your head and speaking to our brain, even if on a subconscious level, we are enamored by sounds. We cannot avoid sounds. We hear our name spoken from across a loud room our attention is suddenly averted to wherever our name was spoken because we are constantly listening and filtering for audio relative to us. Our name, is obviously important information and therefore we we should tune into that conversation.
These same tactics can be applied in environments for either gaining or diverting attention. Much of these special tactics are used in theater and live performances. We are seducing our users into where we need them to be through a variety of different ways. At the end of the day it is a game of allure. Attract and lead – don’t rush in. It’s almost like dating!
All of this continues to harp the importance of the human experience and human comfort. The quickest way to cause a bad experience and discomfort is causing “sim-sickness”. It is easy to make this fatal blunder if you are changing the environment unnatural to human expectations. Sim-sickness is physical discomfort, including nausea and fatigue. Everything we design we want to have replay-value and share-value. If the adventure the user virtually journey is physically discomforting, then we cannot expect them to share the experience with others. We can also expect them to not return. Both these will ruin your brand. This is detrimental to monetization, too. Always keep them coming back for more! Always turn them into your ambassadors!
Much of the physical comfort has to do with proprioceptive system, which is one’s relative position to neighboring parts. The brain integrates information and from proprioception (Latin for “one’s self”) and the the vestibular system (also known as the human balance system) to orient itself in the provided space. Where is my elbow? How far is the floor? And other measures of our whereabouts, location and proximity are computed innately by these systems, without our conscious awareness for the process. Further, these systems are also responsible for understanding movement, acceleration and most kinesthetics. If not designed for, we can easily cause the same kind of motion sickness one might get in a car, riding a rollercoaster or any scenario where near items are moving at one speed (or none) and far items at another. This is the biggest concern in VR today because it is very difficult to evangelize the industry to someone ready to puke.
This forces us to realize that everyone is different. My mom is going to get nauseous more more easily than my little cousin. This is true for moving objects and landscapes when we seek avoid discrepancies in the proprioceptive systems, but also for more simple requisits of any experience. What if one is engaging the surroundings? Someone who is six foot three is able tor each an item someone who is five-foot-one might not be able to. We never had to worry about the size of our viewer when we were designing or a framed screen. Previously, the user’s size and energy hardly mattered. Now, however, if one is not up for the physical challenge the experience may be limited. How adaptable is our experience? Who is at the center of the design? Why? How can it evolve or adjust? What settings can we allow the user to change and how does it initially default? These questions and many more need to be asked throughout the process.
Lastly, and briefly, we will look at technical design. The focus for this particular article, in regards to technical design is anchoring objects and depth design. When refrencing anchoring here I mean physical (virtually physical) objects. Of course, especially when delving into psycho-states and the human condition, we can also “anchor” sounds and emotions through Pavlov-like learning, but that is a deeper diver into a different realm of design. We’ll go that route when we further discussing psychological influences. Here we are anchoring physical environment, like the base of our vehicle.
Imagine you are paragliding over Rio De Janeiro. The parachute above and the bars that connect us to it stay still (relatively) and serve as a way to anchor us into the scene. Even when we spin and swirl and look around 360 degrees, we have that constant anchor of the parachute keeping us (for lack of a better word) grounded in the experience. Or, you can use a vehicle. Or another form of transportation vessel is common. Recently, directors have been experimenting and finding success in adding a nose between the eyes. Creating a small flesh-looking nose to be subconsciously aware of, as we are in the physical world, to lock us into the perspective and be anchored.
The brain is powerful when filling in the gaps, and we should let it work its magic when it can. Which means that if we cannot track the true position of something, as in we cannot replicate movement naturally, then do not. We might not know the real position of our elbow or torso because there are not sensors on them Therfore, it is better not feign their existence because our mind with be perturbed buy the inaccuracy. The user’s mind and imagination will fill in those gaps more accurately than we can design to.
Additionally, for technical design, we want to avoid making the user look near, then far. It is a simply mistake that causes eye strain and ultimately, fatigue. This can be common when the user is reading a text box that is floating in space. If the box is not rounded and equal distant to the user from all angles then it forces the user to focus at differing depths. The easy solution is to round the text box, but we will come up with better designs soon, as floating text isn’t the future. We have also been anchoring menus to the body. Like an NFL quarterback looking at his wrist for the plays he is going to call, our menu options are on our body, mitigating depth challenges by anchoring it to our own body. The simple take-away here is respect depth and anchor wisely.
Overall, the user interface is much different than on a mobile or desktop as there is much more that needs to be considered. We do not just design wireframes to review flatly, because we are not looking from just one angle the way we do with a screen. We must think of shadows and light and contrasting objects that will affect the surroundings, or depth, our perception and our sensation of being present. We’re not simply adding 50% more by upping from 2D to 3D, but nearly 500% in the complexity as we are moving in more directions, free to change position forward/backward, up/down, left/right, and in orientation through rotation about perpendicular axes pitch, yaw and roll.
At this stage most designers are reflecting, and should be. We will better understand preferences, and the colors, textures, features and patterns are best received, but that is with time. For now, design with the user at the heart of the equation, and build around their experience and comfort. Our day will come where we can design anything anywhere, but for now it is critical not just for the user, but for the industry, to optimize comfort.