XR and the Self-Inflicted Trough of Disillusionment
Or, “Snow Crash Is Not a Real Use Case” (with all apologies to Neal Stephenson)
We’ve all seen the fevered reactions over the last few years from people who experience VR and MR demos: gasps, hoots, unbridled joy, and child-like wonder. People are quick to become enthralled by immersive multi-sensory novelty, and design teams are excited by their excitement. We’re all very excited! In fact, we’ve been very excited for a long time now without much actual payoff.
A variety of XR devices have been available for years at this point and they have yet to penetrate mainstream markets or deliver content sufficiently compelling to capture the sustained interest of mainstream consumers, even as they continue to dazzle us in demos. It’s easy to blame the stagnation of XR in the market on the understood-to-be-inevitable tech hype cycle, but in this case the trough of disillusionment is being extended by developers who seem to have forgotten some fundamentals of user-centered design: the products people use and the products they say they want are frequently miles apart.
Trending AR VR Articles:
2. Infographic: The Future of Virtual Reality
3. Mario Kart in a real vehicle with VR!
4. Five Augmented Reality Uses That Solve Real-Life Problems
What’s even more demotivating about XR content continuing to miss the mark is that consumer perception appears to be hardening around the notion that these devices are curiosities best suited for a hardcore video game audience and entertainment content. “It’s cool, but I don’t play video games so it’s not really for me” isn’t the kind of feedback you want to hear about what you hope will be the next ubiquitous human-computer interface.
As user-centered designers we have reliable processes and methods to help identify priority use cases and separate them from novelty, so why are we spending so much time spinning our wheels in the XR novelty trough when we’ve been doing design research on these devices for years?
There are answers here, but we first need to revisit the recent past: 2010, to be specific…
Looking at XR through the lens of Microsoft’s Kinect: Echoes and Repeating Patterns
A post mortem on the rise and rapid fall of Microsoft’s Kinect should be required reading for anyone working in XR. If you need a refresher, Kinect was the Microsoft embodied controller everyone believed would usher in a new era of interactive possibilities, opening the floodgates for myriad new experiences. Expensive and flashy demos (some of which are unfortunately now rather notorious) were rapturously received by consumers and press at conventions around the world, tens of thousands of hours of user tests were completed by the Kinect design team who felt confident in the soundness of their use cases, and when the device shipped Microsoft was quick to declare it the most successful consumer electronics launch in history with 10 million units sold in the first three months.
It was only a short time after Kinect’s launch, however, when serious cracks started emerging in the hype: the devices were notoriously fickle and plagued with errors, many users didn’t have sufficient room in their living spaces to comfortably enjoy experiences that required them to stand a considerable distance away from the device for it to work (and to prevent injury and property damage), experiences excluded a wide range of people with mobility impairments, the available content was sparse and underwhelming, and — perhaps most embarrassing — the palette of interactions hailed as a breakthrough proved to be starkly inferior to what was already available (e.g., navigating menus using hand gestures was far slower, more physically burdensome, and more unreliable than using a controller, which was true for many of the Kinect interaction conventions).
Any of this ringing a bell re: XR?
It’s tempting to say that hindsight is 20/20 and nobody could have predicted Kinect’s failure to capture the imagination of the market, but that seems rather generous given the facts: Microsoft released to a mainstream consumer audience what was still fundamentally a research prototype, with a modest set of narrowly-defined and not fully baked use cases, and a palette of interface conventions that should have been easily identified as deeply problematic in user testing by experienced interaction designers.
Even ignoring problems with device reliability and performance, Kinect broke the golden rule of interaction design: the best interfaces allow users to accomplish the most with the least effort (both cognitive and physical), and there needs to be compelling justification to stray from that maxim. In the case of Kinect, Microsoft’s justification seems to have been “because it feels cool and futuristic to use your hand to navigate menus or to stand up and pantomime pointing a gun at your television to shoot things in a game”, and they’ve never offered a reason for designing those overwrought gestures that holds up to user-centered process scrutiny.
Hot tip: If you’re in the mood to laugh and sometimes wince or cringe, search Youtube for “Kinect Fail Videos”.
How, one might rightly ask, did a company like Microsoft — with all the resources at its disposal, expertise in shipping experiences and devices, and super-smart design and development teams — ever think the Kinect product they launched was viable? Many have attempted to answer this question, but I suggest two culprits that haven’t received attention:
First, The teams designing these devices and platforms are composed largely of young and able-bodied video game industry people who work in fairly homogeneous environments, where virtually everyone has the same canonical cyberpunk references about future tech, leading to a lot of unexamined confirmation bias and group-think.
Second, Multi-sensory curated experiences presented in immersive platforms possess an outsized power to dazzle users in brief demos and lower our objectivity about how useful the products will be in daily life relative to what’s available today. When designers who are already infatuated with pre-existing notions of what a thing should be meet users who react to those notions with cheers of joy when experiencing the novelty of an interaction that feels lifted directly from our cultural science fiction canon, it leads even experienced teams to make bad decisions.
And with the above two points, the Kinect conversation becomes indistinguishable from much of the XR conversation.
The outsized presence of video game industry talent on XR teams is bad for [mainstream] business
From the promotional materials, marketing videos, and major demos released by the HTC Vive and Oculus product families, and even MR devices like Microsoft’s Hololens and Magic Leap, you’d likely surmise that these devices are mostly interesting for playing video games or indulging in entertaining novelties like aliens popping out of your living room wall or holographic Minecraft on your dining room table. While some other kinds of use cases get a few mentions here-and-there, the most visible marketing and demos tend to focus on video game content.
If device makers are hoping their products will become as ubiquitous as PCs and mobile devices one day, a widely-held perception that the devices are only interesting for gamers is going to make realizing that hope a much heavier lift (just ask Microsoft about their experience trying to get consumers to re-imagine the Xbox as a general-use device; changing consumer perception once it’s hardened is not for the faint of heart).
Again we might ask why entirely new interactive platforms like VR and MR — each with unlimited use case potential — have chosen to focus so much effort on video game content; the obvious answer is that the people making the devices and platforms are disproportionately drawn from the video game industry and that’s all they know. Imagine what might have happened at Xerox PARC if everyone exploring use cases and device standards for PCs and GUIs, or at Apple if everyone working on use cases and device standards for the first iPhone and iOS, came from the video game industry and had spent their careers making devices and experiences exclusively for gamers.
And so, the state of XR today is eerily reminiscent of Microsoft’s experience with Kinect: the retail devices marketed to mainstream consumers are still fundamentally inelegant research prototypes with a modest set of narrowly-defined and not fully baked use cases accompanied by a palette of wobbly interface conventions; complete and polished experiences that clearly demonstrate the power of the platforms remain hard to come by even years after commercial release, and video games and assorted novelties represent the vast majority of available content (most of which have proven to be underwhelming even to the game community). There are hundreds of potentially compelling non-game use cases being explored by small teams or niche industries, but they’re not what’s being showcased by device and platform manufacturers, they don’t present a unified vision for how these devices might exist in our daily lives, and they’re not top-of-mind with consumers.
Indeed, as of late 2018, device manufacturers appear to be abandoning the consumer market for now and focusing on more narrowly-defined business and enterprise use cases.
The teams making the HMUs and their supporting software platforms acknowledge (sotto voce and away from marketing) that the hardware is still multiple generations away from offering the same fidelity and performance as our current 2D screens, and that processing and display limitations constrain the kinds of mainstream commercial products that can be delivered today (and tomorrow, even). These devices are clearly years away from being ready for mainstream use, so we can and should ask why the marketing messages suggest otherwise: once people get the devices home and realize the gulf between what’s printed on the tin and the actual experience it becomes obvious that a focus on short-term gain by manufacturers is inviting long-term pain by consumers who won’t be fooled twice.
Gamers, however, have proven willing to part with large sums of cash, jump through elaborate setup hoops, and deal with considerable bugs to be among the first to experience new tech, so one might conclude that an early focus on this audience was a smart strategy to maintain momentum until some theoretical future generational leap happens where the hardware, industrial design, platforms, and interfaces have been refined sufficiently to be usable for a mainstream audience. Unfortunately, what appears to be happening is that the world is moving on and interest is waning, and the hope that one day a ready-for-primetime device will appear that developers will suddenly start populating with stuff mainstream consumers see value in is starting to seem a bit like wishful thinking in the near term; more likely is that something unexpected representing a fundamentally new interaction and/or display model will come out along with a killer use case that captures the popular imagination from one of the big companies who have a history of doing this stuff successfully, and the decade that HTC, Oculus, Microsoft, and others have spent allowing the mainstream use case narrative to take a back seat to video games will come to the same sad end as Kinect’s brief moment in the sun (just much more slowly and painfully and even more expensively).
There’s still hope for an eventual mainstream platform winner to emerge from this generation, but for that to happen the industry needs to do some serious realignment.
Some thoughts on what it will take to right the XR ship
1. Stop leading with entertainment: unless we want HMUs to become just another video game console, it’s time for device makers to stop leading with entertainment and to start telling comprehensive stories about how their platforms will be able to address mainstream use cases better than current devices, and to invest more resources in marketing and messaging to support those use cases. Three years ago was the time to start changing consumer perception, so there is catch-up work to do here. People who don’t come from the video game industry need to lead this charge.
2. More diverse leadership: Speaking of people who don’t come from the video game industry, major HMU players need more diverse senior leadership, and more team diversity in general, to eliminate confirmation bias, group-think, and limited visions of what our future will look like with these devices as a core aspect of daily life.
3. Avoid the novelty trap: Beware the power of immersive novelty experiences — the non-nutritive sugary treats of interaction — to dazzle and trap both users and developers; that new arty thing you made in VR is gorgeous and cool and yep people loved it, but does it really help us discover use cases that allow us do things better than we could do them before?
4. Accessibility NOW: It’s time to address the long-standing accessibility elephant in the room: much like with Microsoft’s Kinect, too many experiences developed for HMUs exclude large numbers of people with physical disabilities and even those who just aren’t young and fit. It’s moderately tiring for a healthy 25 year old to participate in an experience requiring controllers to be continuously held in outstretched arms, but it’s impossible for many older users or users with other mobility impairments. Room scale VR experiences need to include seated or reduced mobility options. We should all be committed to using this technology to increase access, not constrain it.
5. Get serious about interface and interaction conventions: It’s time for a coordinated laser-focus on developing design standards and patterns for interface conventions that don’t represent a step backwards from what’s currently available with mouse-and-keyboard (and again, interface design teams should be diverse and not dominated by designers from the video game industry). Progress here has been spotty and only loosely coordinated.
6. Honesty in representing current products to consumers: Magic Leap and Hololens are both notorious for producing marketing videos that bear no resemblance to the actual user experience, especially around their tightly constrained fields of view, uneven performance, and graphical fidelity.
We can learn from prior efforts like Microsoft’s experience with the Kinect, and we can avoid making the same mistakes. The biggest priority for the XR ecosystem at the moment should be diversifying talent beyond the video game industry, particularly within HMU leadership, teams doing design research, and teams responsible for marketing: more diversity will give today’s XR devices the best (and perhaps only) shot at discovering a path into the mainstream before they’re leapfrogged by something else.
About me: I lead design teams and look for ways design processes can more effectively serve customer needs. In a prior life I led feature and product design teams for Halo, Windows, and Visual Studio at Microsoft, and I’m currently running a studio at the University of Washington exploring ways to make design and development culture more innovative, impactful, and inclusive.
Don’t forget to give us your 👏 !
