Augmented reality (AR) technology has been around for a while (the launch of Pokémon Go seems like a lifetime ago), but maybe right now, during the pandemic, it’s the perfect time to bring AR even more into focus.
Currently across the globe, many of us find that we are closed off to the outside world. AR technology allows us to bring things into our homes that we’re missing right now.
Advertising, retail, construction, education, travel, and many more industries have used AR to create immersive experiences that go beyond the flat screen of a smartphone. As designers we’re used to adapting to change. This current crisis puts a new perspective on designing experiences that can help our users, since AR technology does have the potential to help in situations like this.
We might feel disappointed when travel was cancelled or we feel out of place when we can’t go to a physical store to see items in person.
For students who may not be used to the school experience that’s taking place in their homes, AR can serve as a way to both educate and entertain them.
And many of us are feeling pretty scared or anxious at the moment, so using AR can be a way to temporarily take our mind off of things.
Here’s our guide to making these experiences as user-friendly as possible.
Augmented reality (AR) refers to an interactive experience that takes place in a real-world environment.
With AR, a user interacts with the image of a real-world object or environment via a screen (mobile phone, smart glasses, etc). The device looks for a ‘marker’ and when it finds the marker, a virtual object is added on top of the real-world image.
Please note: When talking about AR it’s easy to get confused with virtual reality (VR). Although there are ‘virtual’ elements in AR, with VR the real-world environment is entirely replaced and you’re immersed in an entirely different environment and experience. One is not better than the other, they both have their advantages and best use cases. This post will focus on AR.
As you interact with more and more AR experiences, it becomes apparent which are the ‘good’ experiences and which aren’t. Whether you’re an AR pro or just getting started, these are some tips and tricks for designing in this medium.
One of the most important question that needs to be answered is, “Why would the experience be better with AR?”
There are many tactical design tips for creating a user-friendly experience. However, before committing to creating an AR app, make sure you’ve done your research. It’s important to tell the story of how and why your users will use and benefit from your product.
There are so many opportunities with AR. It can help with employee training, bringing experiences to those unable to travel, education, and so much more. It’s important to check out as many AR apps as you can, some in your target market, but some that aren’t. The best inspiration can come from other industries.
Along the way, you’ll get plenty of design ideas, but it’s also important to have an understanding of what is and isn’t possible. There needs to be a balance between the functionality and the technical capabilities. AR technology is always improving and there’s every reason to be optimistic about what the future will look like, but always keep the user at the center of everything you design.
There are so many opportunities with AR. Because it is so new and exciting, it might be tempting to include everything and give the user a lot of features right out of the gate. However, as enticing as it is, keeping it simple is the best approach. Too many features could introduce unneeded complexity. The user can only do so much at one time, so keeping it simple will keep the user focused.
It’s not everyday that you have the Solar System in your dining room. The Healium app is intended to reduce anxiety. This is a case where AR has the potential to help students who may not be physically in a traditional classroom, which is the current reality for many.
One major constraint of AR is the viewfinder. You can only see and interact with what’s in view at the current time. It may seem obvious, but the user is in control of movement, so they are controlling the viewfinder. From a user’s perspective, navigating the view, might take a bit of getting used to at first.
There may be an ideal starting point, but this can’t really be controlled, since the user controls the camera. Since you can’t control where the user is looking, AR creates an interesting design challenge.
To fully utilize the full AR experience, the user has to move the viewfinder around, or in many cases they might be moving around in their environment. The ‘keep it simple’ rule applies since controlling the camera can be a challenge when doing more complex tasks.
It’s important to keep in mind that in most cases, the user will be holding their device with two hands, which makes it necessary to plan the interactions accordingly.
I was recently using an app and had to line something up with a marker and then tap the screen when I was done. I have a relatively large phone with a heavy case, so I found it difficult to do this. It was especially challenging because the position shifted and I had to readjust and then tap a location that was in the center of my screen.
Live Home 3D has directional control (bottom left) and zoom. It’s easy to move around the space by tapping and dragging.
Can users take one hand off the device to touch the screen? In most cases, yes. However, it can be a challenge for many. This is something to consider with larger devices. Sure, some users have access to a hands-free 3D viewer, but many do not. If it makes sense for the app, directional buttons or quick options may help the user stay engaged so they can focus on navigating.
Not everyone has tried a lot of AR aps. Your app might be their first AR experience, so it’s important to have a great onboarding experience for your users. You might be aware that a flat surface with light is needed to get the best experience, but users may not know this. It’s important to give clear instructions before starting that introduce them to the basics. Simple instructions with images can go a long way.
Angry Birds includes a helpful visual to help guide users so they can find the best surface to play the game.
For the pros, the ability to skip should be presented, as they might just want to get started.
Where will they be using the app? Will it be indoors or outdoors? This will really set the stage for the ideal use case.
Options are easily accessible in AR Dragon.
Defining the experience size for the user is key to a strong start. It’s important for them to know right away if it’s tabletop experience or if they’ll need a larger space like an entire floor (think room scale). If you’re working with larger assets, like furniture, the room scale and environment would be different from an app where you’re studying something smaller like stacking blocks.
A larger environment like the floorplan model on the left (Live Home 3D) needs a larger space than a stack of blocks on the right (Stack AR).
There are limitations to the environment and things won’t always be optimal for AR in general. Some elements might require a very large horizontal space. If something like this is preventing a good experience, it’s worthwhile exploring different solutions.
Maybe you’re trying to design an AR app where a user is bowling. That’s quite a bit of space to ask of the user. It might be a good idea to add a virtual bowling alley so they can concentrate on the app, rather than trying to find a usable space in their environment.
It should be apparent to users which elements they can or can’t interact with. A visual indicator to let users know certainly helps. Some apps use outlines or other visual indicators. Depending on the interactivity, it might make sense to show which object is highlighted by showing a ‘select state’. If there are multiple items on show, this can be very helpful.
In the IKEA Place app, it’s easy to see what’s selected because there’s a shadow and a ring under the chair. When the chair is put into place, it is no longer selected and the ring disappears. This allowed me to try out some new furniture ideas while being at home
Users should also be able to quickly pick up on other interactions. If they are able to move things around, it should be apparent. Is it a single touch and drag? What happens when it moves into space? Typically, the item should get smaller so the user can confirm that the object moved. Does the standard rotation interaction work? What about scaling, is it a simple pinch action?
Not everything goes smoothly for users. Maybe their environment is too dark. Sometimes they might find themselves out of bounds and not sure how to get back. Some apps have an off screen indicator with an error message that helps the user get back on track.
It’s important to provide a realistic environment for the user. Objects need to look real so the user believes that the virtual elements belong in their environment. Light and space are super important. The height of objects should be maintained if you are moving objects around in space. For example, if an element is moving back into space, it would get slightly smaller. Lighting is key when creating realism. If the lighting looks off, it’s not going to be believable. Highlights, cast shadows and reflections will help things look more realistic.
Detail and texture is extremely important in the virtual elements. Textures help things look more lifelike. Flat colored surfaces however might look too perfect and fake to the user.
The avocado from the Avo! app has a very realistic texture. It’s very believable that there is an avocado wearing star glasses on my couch.
What should be ‘in the AR world’ and what should be ‘out of the AR or virtual world’? At first, it may seem obvious from a design perspective. The ‘virtual’ is the new stuff? However, some things may not be as clear. What about game scoring information, messages, etc? Which environment does that belong in? It all depends on the app, but try to avoid placing controls and messaging in the viewport unless they’re necessary for the experience.
It’s kind of related and worth mentioning, design consistency is an important consideration. AR technology is relatively new, so best practices are still being established. With that said, it’s still important to create the most consistent UI as possible. Common design elements should be used consistently. Things like icons, UI components and messaging should be predictable throughout the app.
This is an important one. Immersive experiences are great, but there’s also the potential for people to forget what is happening around them. It’s important to design experiences with safety in mind. Make sure that you’re not asking your users to walk backwards or anything else that could potentially make them stumble or put them in danger. AR navigation apps benefit from ‘reminders’, since users are most likely super focused walking around, holding their phone up.
Also, holding a phone up for long periods of time can be tiring, so resting points are important. If a user pauses, make it easy to store progress. Users should not be required to do anything physically demanding or that may feel uncomfortable. It makes sense to include stop-points or in-app notifications that suggest they should take a break.