The UX & practical applications of mixed, augmented & virtual reality Creative Circle @ Rainmaking Loft (Weds 19th October 2016) Review

augmented virtual reality

Photo credit: pestoverde via flickr

Is this the real life? Or is it fantasy? If questions like these give you sleepless nights, you should’ve come along to the Creative Circle’s inaugural event on augmented reality (AR), virtual reality (VR) and mixed reality (MR).

That’s right… shit got very unreal—thanks to presentations from professionals working with this surreal technology to overcome very real challenges.

The best thing about this event was that all of the speakers focussed on practical and meaningful applications of this technology.

We even got answers to the question on many a designer’s mind, “What do you need to be aware of when designing for virtual and augmented reality devices?”

3 speakers discussed how their companies are creating and applying VR, AR and MR experiences, as well as how they believe these technologies will evolve:

  1. “Alternate realities: moving from marketing gimmicks into our daily lives” – Chris Scattergood, Founder, Fundamental VR

  2. “Architectural design through virtual reality” – by Ekke Piirisild, Director, VRtisan

  3. “How do we design for the experience of mixed reality?” By Maximilian Doelle, Chief Prototyper, Kazendi

Ekke and VRtisan provided a pretty cool virtual reality station that attendees could play with.

Nonetheless, the star of the show (to me) was Maximilian, whose talk delved into the design and mechanics of the Hololens. He even gave a live demonstration. So, I’m going to start this article by covering his talk.

How do we design for the experience of mixed reality? By Maximilian Doelle

Max got us off the mark by asking the room to define the terms augmented reality and virtual reality.

Definitions of augmented reality, virtual reality and mixed reality

Virtual reality is an experience where your visual (and perhaps other) senses are completely isolated from the real world, and placed in an artificially generated one. That is, images are generated using only artificial light.

Augmented reality is an experience where as well as receiving light and images from the natural environment, your eyes (and perhaps other sensory organs) receive stimulus from artificial light, and perceive visuals superimposed on the real world.

Mixed reality is (of course) a combination of the two – an artificially generated world superimposed on the real one… with the added dimension that MR is aware of what’s going on in your real, immediate environment.

VR lets you block yourself out from the real world and create a new, artificial one. AR lets you copy and paste elements of an artificial world into the real one. MR lets you create an artificial world that is aware of and interacts with the real world.

If you project a model of an engine across a table, MR won’t just paste a visual of the model over the table – it will sit the model on top of the table (just like a real, physical object would).mixed reality spatial mapping

Max stressed that Kazendi is not affiliated with Microsoft – although the technology it uses to deliver this mixed reality is the Microsoft Hololens.

Then he discussed some of the use cases for this technology – including for visualisations, education and virtual assistance. I’ll cover 2 here, which weren’t discussed (at least not in the same way) by the other speakers.

Use cases: virtual assistance

Apparently, a version of Skype that’s compatible with Hololens already exists!

Using this technology, a handyman (for example) can remotely give assistance to a hapless parent, trying to fix their sink.

They can draw 3-dimensional instructions which spring to life on the Hololens, allowing the user to know exactly what needs to go where and how, and which bits screw into others.Skype Microsoft Hololens

Use cases: education

According to Max, researchers say the more sensory input people connect to a piece of information, the more effectively and efficiently they learn it.

Well, with MR, more senses are being stimulated, and in deeper ways. Simply put, there’s visual, audio and spatial stimulation, all at once.

Some very lucky kids are already using this mixed reality technology to learn, by visiting museums and far away, exotic lands.

All I had was a dodgy V-Tech computer that sometimes told me off by dispensing an electric shock…

New interfaces, new interactions, new user experiences

How does Hololens allow this level of compatibility between unreality and reality? Max explained that it’s constantly tracing and retracing what’s going on in the real world, through a process called spatial mapping.

At one point, Max created a model of a ballerina in the room, blew it up to life-size, then made it dance around the room. It was pretty awesome. Below is another part of the live demonstration. 

As well as spatial mapping, the Hololens has 6 microphones – 3 facing the user and 3 facing the external world. This way, it receives feedback from both and provides a 3D audio experience.

Hololens doesn’t need a computer to run – it’s a computer within itself – and Max explained how the kinds of interactions people might have with this “computer” are so different.

For one, your head can now be used as an input device. You can fix your gaze on something and Hololens will recognise that, making the object pop out and enlarge.

We now also have air gestures – Minority Report-style… except Max didn’t even need pimped out, Michael Jackson-style gloves.

You can hold, move, pinch, pin, enlarge, shrink, flip, flop… you can do damn near everything you can in the real world! From a user experience point of view, there are entirely new user journeys to contend with.

Mixed reality user experience

Click on this image to enlarge or download

Max admits a lot of education needs to be done before this technology can gain proper adoption within the consumer market.

Alternate realities: moving from marketing gimmicks into our daily lives, by Chris Scattergood

Chris began his talk by noting that although his company is already doing some amazing work (mainly in the medical field), he planned to make several semi-unfounded predictions.

Why? Why not? After all, we now live in a world where reality can be changed in an instant.

Chris acknowledged something I suspect many of us already feel – that the hype about VR is over-the-top and unsustainable. Nonetheless, he believes there’s cause for optimism and bases his predictions on things we’re already doing with virtual reality technology.

For example, VR is already being used during the training of surgeons – I suspect not as a replacement for real surgery, but as a supplementary tool. Out of the typical learning environment, surgeons-to-be can use VR and haptic devices to simulate the human body and the tools used to fix it.

These haptic devices are so clever that they can tell the difference between cutting through bone and skin in the virtual world, and provide accurate sensory feedback in the real one.

Haptic devices can also be used as scalpels, needles or a list of other medical tools, and the feedback received by the user will be adapted accordingly.

This particular FealReal™ technology is actually the intellectual property of Fundamental VR, so don’t get any funny ideas…

11 (good and bad) virtual reality predictions by Chris Scattergood

  1. By the end of 2017, mass media will blame at least 1 mass shooting on VR
  2. By the end of 2018, there will be at least 3 movies about fear of VR
  3. By 2025, our most treasured possessions will be VR family movies, weddings, picnics, holidays etc.
  4. In 2025, the majority of VR revenue will be from social experiences
  5. By 2018, virtual reality will be integrated into all independent UK schools, and in the process of being implemented in all state schools
  6. By 2020, augmented reality lectures will be commonplace in universities and classrooms
  7. By 2020, at least 20 reality shows will have VR as a key component
  8. By the end of 2019, Netflix will have a VR channel
  9. By 2020, augmented reality experiences synchronised with television will become a thing
  10. By 2025, virtual reality feature-length films will become successful by including social experiences
  11. By 2020, trackable/wearable props for VR and AR will be common in business and the home

Chris ended his presentation with a self-aware, tongue-in-cheek quote by Niels Bohr – “It is difficult to make predictions, especially about the future.”

Architectural design through virtual reality by Ekke Piirisild

Ekke explained how VR is helping professionals in the architecture and construction industries collaborate better and more easily.

  • Designers can go into 3D models of their designs, at 1:1 scale
  • They can move, modify, and build, among other things, in real time
  • Architects, clients and other users can interact with and modify designs
  • The experience is immersive and can give all parties a sense of what it’s like to be inside their designs

All these factors enhance multiple stages of the workflow – communication, collaboration, concept design and design development.

The tools and methods currently used across these stages include:

  1. Sketching/mass modelling
  2. Scaled drawing/scaled modelling
  3. 2D/3D/CAD/BIM
  4. Collaboration platforms
Video Thumbnail

Interactive Virtual Reality Architectural Visualisation – VRtisan – Examples of interactivity

Ekke also noted that there’s import/export compatibility between 3D VR designs and other types of 3D models.  So, professionals are able to use this technology based on their current skillsets alone.

The levels of proficiency across users also needs to be taken into account when designing for VR. You need to make sure the novices can easily do the basic stuff, and the pros can easily do the complex stuff.

In architecture and construction, these levels of interaction can be categorised as follows:

  • Basic – locomotion (movement in the virtual world)
  • Medium interactions such as toggle on/off, pick up, put down and clone
  • Complex choose, place, scale and rotate

And you can get different versions of this software, which is based on the Unreal engine,  according to your needs. There’s also a full version which allows all levels of interaction.

Consumers and users get the freedom to experiment with designs and leave markups. Industry professionals get to collaborate with others from different fields. VR designers get a tool that allows more detailed development.

Overall verdict

I’ve been to a couple of events about virtual reality and Creative Circle has provided, by far, the most informative experience.

Because the technology is still inchoate, most discussions veer towards baseless speculations about the future or, more recently (and perhaps, annoyingly), Pokémon GO.

But this event struck a good balance between explaining how AR, VR and MR technologies are being used, and how these use cases might evolve.

Our UX blog is on the move...

We're now publishing all of our brand new content on the UserZoom UX blog. All of our previously published articles will also be migrating to UserZoom over the coming months.

Don't worry, we'll still be just as accessible, interesting, helpful and entertaning as ever. We just have a different name and an owl instead of a question mark for a logo.

Come say hello!

Timi is a London-based copywriter and full-time marketing sceptic – there are now more unvalidated opinions out there than ever.

He became a UX testing enthusiast after seeing its power while working at TUI – the world’s largest travel, leisure and tourism company. He then joined WhatUsersDo to sharpen his UX knowledge and work side-by-side with the field’s best and brightest.

Leave a Reply