My intention with the visual effort was to have the look and feel draw inspiration from my extensive design work on the 4D Studio platform (previous case study). Intentionally utilizing the same types of transparencies, overlays and subtle design color cues in the UI, would serve the user's needs and maintain consistency. While equally allowing the UI to fade into the background, thus exposing the real-world settings, to take center stage. The color palette was carried over – simple, clean and professional. Delightful subtle user experience interactions were meshed throughout to heighten engagement and delight.

 

The final look & feel is readily recognizable from my design work featured on 4D Studio. This is intentional and a natural extension as well as progression of my product design vision.

While the design work was wrapping up on this tablet application, it was time to say goodby to the contractors. I was already shifting my core focus onto the app suite for our pending hardware device. The tablet app would allow us to test various aspects of 4D Studio publishing, while ultimately re-directing all discoveries towards the smart device's app suite. Therefore, I began to ideate again on what this experience would be on a hands-free, gaze-controlled device.

Who would have guessed? Well, me for starters and anyone else that was paying attention. I was accustomed to change and re-direction. I feed off of challenges and I rise to every occasion. Therefore, when asked to additionally take on the design of the software for our pending smart device – which was still yet to be created – I jumped at the opportunity.

 

The first challenge here was the same as with our tablet app – design and deploy a mobile application that would work in conjunction with 4D Studio. However, this time it was different. Very different. Our device would be using gaze-based interactions to control applications. A digital artifact, called the reticle, appears on the displays in front of the user in space and follows their head motion. There would be no mouse for interacting with the UI. The decision came down from above to use a gaze tracking and a reticle as a controller.

 

The Daqri Smart Helmet (DSH) was conceived to make industrial jobs easier by overlaying instructions on complex equipment — a relatively common use for AR headsets. It was envisioned as a combination safety helmet and AR headset that overlays virtual instructions, safety information, training and visual mapping on a real-world background. Workers in the oil and gas, automation and manufacturing sectors often need to understand complicated instructions to perform complex processes. The DSH would enable them to see digital information overlying different contexts — a physical controller, a scanning device or quality control meteorology equipment.

 

The core user of these apps were typically field workers. The worker that is out there day in and day out, in the physical environment. A field engineer may be responsible for 600 different work packages, using some perhaps only once in a career. Having the added data when it’s needed makes a big impact. With the 4D Studio platform users were desktop bound. However, these users were the "consumers" of that content and on the go.

 

The first major roadblock of this challenge was the fact that our hardware wasn't even out of the lab.

"I know you don't have access to our hardware, but I need you to create the mobile apps that will be running on it when it's ready. Is that going to be a problem?"

Eventually, I would get my hands on our hardware. But amazingly, I still had to progress with design work without one to test on. I had to get creative and employ numerous clever "work-arounds" in the meantime. One such approach was securing access to the same display lenses that we would ultimately use in the smart device – to mirror my design to for testing from my laptop. This allowed me to realize and alter transparency, depth of field issues, lighting and color issues otherwise unforeseeable. Prior to this approach, I was literally designing blind!

"We need to switch our tech stack immediately. I know I said we were betting on Android, however that's changed. We will now be developing our apps using Linux and 4JS. Sorry guys, you'll have to redirect all your previous efforts." – Daqri CTO

"Hey Neil, can you design a logo and App Store icon for our custom 4JS framework in between your current work? We will be releasing this shortly to our developer network as a company-branded offering." – Daqri CTO

As an incredibly versatile and adaptable designer, I have been tapped many times in my career to create original branding and marketing materials. This app store icon request, simply brought me back to previous comfort zone where I was executing custom iPhone applications in the early years of Apple's App Store.

You may recall that from the start I never really had internal support for actual user testing. Our CEO felt he knew what our users needed and didn't care much to interview them or get any feedback. Due to my gorilla efforts though, over time I was able to amass some user data. This data informed me where to place our focus and what to iterate. I was able to employ the data in assisting my decision making and adjusting my assumptions. To help gauge our success, I settled on a series of user-centered metrics that allowed me to measure the changes we were intending and the eventual outcome. This all improved over time as we eventually got the benefit of a dedicated user researcher added to staff.

Eventually I had a handful of studies available to draw from to better understand the pain points out in the field thanks to the addition of a researcher to the team. From these, metrics were identified. I had to take into consideration the challenging conditions that would compete for our user's focus and attention while using the UI.

Our initial team was primarily comprised of myself, a Product Owner, a Front End Dev, three Back End Devs and one QA. I was charged with owning the end to end Product Design – inclusive of conceptual, visual, interaction, research and testing. We were small. We were scrappy.

 

But the design team was now growing. Fortunately as time passed and we progressed, I joined forces with not only a dedicated researcher, but also two other product designers to help me hone and perfect our outcome.

I created the following design approach that was based on our switch to Linux and 4JS. In it you will recognize many patterns that followed us through the various iterations. But with this change, came new and different technical constraints. It was essential that I continued my close collaboration with our DEVs to ensure what I envisioned was technically possible. Note: Background imagery is suggested as this UI would be displayed on lenses over the user's real, physical environment.

You may remember, at the start of this saga – AR Authoring Platform Case Study – where I stated that "I crafted endless creative solutions for seemingly random requests, shrouded in ambiguity, from a revolving door of cast members". This stage of work saw the introduction of yet another new Chief Creative Officer's (and his short stint at our company). With his arrival, came a new design direction mandate.

"I'd like to introduce you all to our new Chief Creative Officer. He will be taking over for name redacted and guiding us towards his new direction for our suite of apps. Now let's give him a warm welcome and all get back to work." – Daqri CEO

Throughout, interactions were vitally important to support and enhance good user experience on our mobile interfaces. On our proprietary hands-free device – The Smart Helmet – thoughtful, intuitive and efficient interactions were beyond essential.

The users of the Smart Helmet were to navigate with a pointer (reticle) on screen using subtle movements with their head. Therefore, every interaction had to be conceived, designed, mapped and tested time and time again. Imagine if you will, navigating your computer screen with a laser pointer in place of your mouse and you'll get an idea of what we were solving for within our designs. Unity was a go to choice for building out the production-level 3D experiences. However, I often employed some much faster techniques – simple animated frames – to quickly test out a theory before allowing Unity to consume so much of one's time.

With my ability to adapt and thrive in an ever changing dynamic environment, I was able to succeed where others may have failed. Furthermore, with my unparalleled design experience within the group, I was able to help our junior team members reach their own levels of success. That is to say, until one final change was too much to overcome.

This was as true now, when refining our design iterations, as it was at any point in my journey to create a streamlined, meaningful and friction-less experience for our users.

Despite being bounced between the whims of revolving door of Chief Creative Officers, we became quite good at selecting, refining and iterating our approach. We took the best from the worst of situations and used it to our advantage. We continually advanced the design to the 'final' outcome following this late stage iteration.

I learned a lot throughout this experience. I learned that solving problems for an industry and technology I had no prior knowledge in, was a sweet spot for me. I fed off of the challenges and thrived in the environment. As a designer who had no prior background in Augmented Reality, all I needed was empathy, curiosity, taste and constant iteration to design incredible experiences. With these elements, I've learned I can create meaningful relationships with our users that can ultimately shape our product into one to be proud of.

 

Users of our product are on a mission. They came to our product with a goal in mind. My job is to help them accomplish their goal in the most direct and pleasing way possible. I believe we would have received favorable results with our work had we been given the chance to fully complete the design cycle. Our influence would have continued to shape the future of our product and help users reach their goals. But we would never get the chance to really know the impact of our work.

Make no mistakes, I didn't enter any of this with rose-colored glasses on. I was very excited about the work and the company's potential when I first started. However, I always felt we were either going to the moon or going bust. There was not going to be a middle ground. As we were nearing the end, a bit more of me felt more strongly towards the latter.

Empathy is trying to see the world from other people’s perspectives. My job as a designer is to help users accomplish their goals in the most direct and delightful manner possible. This characteristic defines me. When it comes to business though, there is seldom room for much empathy, especially during a downturn. This wasn't my first rodeo. I had been through other general tech downturns and I have seen it all. I felt for my less experienced counterparts. Initially, this was going to be tough for them to swallow. However, I was massively confident that a handful of them were going to reach higher highs in the next stage of their career. I too, would carry on and adapt.

 

The following production-level UI release wouldn't have been possible without the close collaboration between myself and my fellow designers Sterling & Anthony.