20 Augmented Reality And Mixed Reality Use Cases
Augmented and mixed reality (AR/MR) are widely expected to go mainstream in the not too distant future.
While current virtual reality technology lends itself to games, AR and MR are suitable for many other types of applications, games and otherwise. Some people predict that their impact will be on the same level as that of the internet or smartphone. Job positions for developers with experience in the AR/MR field are already seeing a surge — this trend is likely to continue.
Many problems in this space are yet to be solved, but research is quickly driving innovation. To stay ahead of the curve, you might want to consider working on a personal side project in this domain. Here are 20 ideas you could get started (or keep going) with.
Disclaimer: I did not research whether any of these already exist or are actively being worked on somewhere. In fact, there’s a pretty good chance that is the case for many.
#1 Sticky Notes
Imagine walking around wearing a stylish pair of newly purchased smartglasses. As you approach the fridge, a small virtual display resembling a notepad animates into existence next to it. On the notepad, you or your significant other have listed a few items you need to pick up from the grocery store. You inspect your fridge and add another item. The list is synced to your smartglasses and smartphone automatically, via the cloud. As you walk away from the fridge, the note disappears with another satisfying effect, perhaps accompanied by a small sound clip. You go to leave your apartment and another note pops up by the door, reminding you to take out the garbage.
Features
- Creating, editing, and deleting (archiving?) notes.
- Customizing notes (color, font, checkboxes, radio buttons, etc.)
- Saving and loading notes.
- Syncing notes across devices.
- Attaching notes to specific locations.
#2 Presentations
PowerPoint, Google Slides, and similar programs are still the de facto standard for presenting to groups of people. This stale area is begging for a better solution — AR/MR might be that solution. The possibilities for presenting and audience with text, images, videos, charts, and other media is only limited by your imagination. 3D data visualizations are often difficult to create and consume nowadays, but AR/MR might be a gateway towards their wider adoption.
Features
- Preparing slides with text, pictures, video, etc.
- Saving and loading presentations.
- Hosting presentation sessions other devices can connect to.
- Showing slides in a centralized location or a unique spot per presentee device.
- Manipulating 2D, 3D, and other data in real-time.
- Customizing slides (text, background, data visualization, transitions, effects, etc.)
- Creating custom virtual presentations spaces in large spaces.
- Making presentations interactive and individualized, with presenter and audience controls.
#3 Gamify Everything
Working out, watching a movie with friends, commuting to and from work, walking the dog, cooking dinner, or doing house chores — AR/MR gives us the tools to make pretty much anything more engaging.
Your next side project could be a smartglasses app that tracks vital statistics (perhaps by connecting to a smartwatch) and awards the user points for calories burned in a workout session. Those points could be converted to discounts at relevant stores, or just collected in order to compete against others. Or maybe it could track how often the user hugs their loved ones, awarding them a score for how emotionally outgoing they are over time. Many gamification apps already exist, but smartglasses will allow those apps to be reinvented and integrated seamlessly into the activity being gamified.
Features
- Engaging users in everyday activities through pre-existing and new gamification techniques
- Sharing progress with other users
- Competitive (e.g. leader boards) and cooperative features to make everyday activities more socially engaging
- Visualizing progress across time
- Offering and visualizing advice and tips for users to become more efficient or proficient at the gamified activity
- Rewarding users through joint programs with (retail) companies
#4 Book Illustrations
This one is an obvious choice for the technology and close to work already being done in existing augmented reality apps. When reading a book—whether it’s a children’s book, a high fantasy novel, a biography, or a textbook — the reader’s smartglasses could produce interactive illustrations or models of the content. This could work in one of two ways, via a QR code or similar visual cue, or by reading the text — in which case the book wouldn’t even need to be changed.
Features
- Displaying images, animations, videos, or interactive 3D models
- Playing audio clips with definitions of specific terms or additional background information about a character, time era, activity, etc
- Recognizing visual cues such as QR codes or using optical character recognition and natural language understanding to determine what visualization to play
- Interaction with visualizations
#5 Calendar
How great would it be to not have to buy a new calendar every year? Of course, smartphones already do that for us. However, phone screens perhaps aren’t as natural a place for a calendar to be located in than, say, a wall in your living space. AR/MR apps should be able to easily integrate with existing calendar apps, making them usable in more traditional ways.
Features
- Integrating with existing calendar apps
- Placing calendars onto walls
- Saving and loading calendar locations
- Syncing calendar locations across devices
#6 Timer and Countdown
Another idea that already exists but could be enhanced with real-world location context. Rather than interacting with several stopwatches in a centralized place such as your phone, being able to put them near different areas, such as the stove, microwave, and fridge could make this much more helpful.
Features
- Placing one or more timers in specific spots
- Adding additional context via text or images
- Saving and loading timers and countdowns
- Syncing timers and countdowns across devices
#7 Smart Home Control Panel
For the more visually inclined smart home users, talking to a disembodied voice assistant might not be the most convenient way of controlling their various gadgets. Enter the mixed reality control panel!
Apart from integrating with existing smart home systems, all that’s needed to accomplish this is wrapping interactable options in some intuitive 3D user interface.
Features
- Interacting with smart home systems through a mixed reality interface
#8 Picture Frames
Another not too far-fetched idea is to replace digital picture frames with virtual ones. Different implementations could place them either anywhere or on predefined spots, via visual cues. A slightly different app might look at physical images and animate or superimpose other images onto them.
Features
- Saving and loading image locations
- Overlaying images onto visual cues
- Applying effects to virtual images
- Applying effects to physical images
#9 Backyard Guardian
Do you have children or pets? If so, you might be familiar with this situation. Picture yourself once again in your kitchen or living room. You are busy preparing dinner or watching TV, while the kid(s) and/or pet(s) are playing in the backyard. You find yourself checking on them all the time. What if somebody gets hurt, or worse, someone looking to nab them is stalking your fence? Either way, what you could really use now are a pair of X-ray goggles allowing you to see through those pesky walls. Enter smartglasses. By connecting to either an external camera or a tracking device on your kid or pet, this could be achieved (both for the backyard and inside). You could further enhance this by mapping out your living space and designating safe areas, so that if someone leaves that area, an alert is created.
Features
- Connecting to cameras or tracking devices
- Overlaying camera output onto walls with correctly transformed perspectives to match the real-world 3D view of the user to the area of interest
- Highlighting tracking devices through walls
- Designating, saving, and loading safe areas
- Alerting the user, if a device leaves a safe area
#10 Tooth Brushing Trainer
Most people don’t enjoy going to the dentist. Despite that, many don’t brush their teeth correctly. Some may not brush long, or hard enough, while others may do more damage than cleaning. Augmented reality could be used to create a toothbrushing trainer — the user looks in a mirror while brushing their teeth and the app analyzes the brushing. How the analysis would work, I’m not sure — perhaps with the help of AI and dentistry experts.
Features
- Recording smartglasses’ video stream in front of a mirror
- Analyzing video recording for brushing technique, duration, and strength
- Advising the user on how to improve their brushing
- Rewarding the user for improvement (did someone say gamification?) or through joint programs with dentists
#11 Speech Trainer
Along the same lines, but perhaps slightly simpler, is a companion app that helps users improve their speech, analysing it for grammar, usage of unnecessary fillers, or maybe even eloquence. We should note that there will be privacy concerns with an app that records your every word.
Features
- Recording user’s voice
- Analyzing the user’s speech
- Advising the user on how to improve their speech
- Rewarding the user for improvement (did someone say gamification?)
#12 Fantasy Sports
This technology lends itself very well to all things sport, but fantasy football and its kin, in particular, could stand to dramatically benefit from it. Not only could individualized league, match, and player statistics be overlaid onto actual coverage in real-time — the user could also be given the means to interact with the information and other users directly.
Features
- Enhancing sports coverage with fantasy-related information
- Manipulating fantasy-related data
- Interacting with other users
#13 Eating at Restaurants
This is another fairly natural fit for AR/MR technology. Customers, when looking at a menu, could be presented with high-quality 3D models of the various food choices. The models could further be made interactable (i.e. moving, rotating, or resizing a model). To top it off, nutritional stats or a cost breakdown could be provided for a meal and its ingredients upon request.
Features
- Visualizing menu items
- Interacting with menu item visualizations
- Displaying nutritional statistics for meals
- Displaying cost breakdowns for a meal’s ingredients
#14 Working at Restaurants
On the flip side, the lives of a restaurant’s waiters could also be made easier. One of the challenges of being a waiter is keeping track of how long customers have waited — for being greeted and handed menus, for their drinks and food orders, and for the bill. Another challenge is taking guests’ orders (especially, if they have specific requests) and answering questions about the menu (e.g. what are the daily specials). Both of these could be solved easily with the ideas explored above: timers and notes. Simply place a timer and note onto each table with guests. A timer might reset every time a waiter talks to guests of a table, or when specific events occur (the order has been taken, food has been served), while notes can keep track of a table’s orders, dietary restrictions, or a running total of the bill.
Features
- Placing timers above tables
- Resetting timers based on different conditions (proximity, talking to guests, predefined events occurring)
- Creating and editing different types of notes above tables (orders, dietary restrictions, running total of bills)
#15 Enhanced Vision
Smartglasses enhance vision in a Predator-esque fashion. Imagine walking, driving, or perhaps skiing or snowboarding through rainy, snow, or foggy weather. Given a device with advanced cameras capable of infrared or other types of vision and a decently powerful processor (spatial computing tends to be expensive), one could conceive of an app that highlights terrain in a way that’s more friendly to human eyesight (for example, by placing a digital mesh on top of recognized surfaces — something some existing apps can already do).
Features
- Recognizing surfaces in low-visibility conditions using additional sensors
- Highlighting recognized surfaces
- Plotting and displaying a navigation path forward or towards a destination that minimizes obstacles or uneven terrain
#16 Autocorrect Handwriting
How often do you write something by hand? How often does correct grammar matter in those pieces of writing? A personal touch in writing can mean a lot in certain situations — writing a letter to a friend or family member, for example.
But in this world of emails and text messages, writing by hand often does not come naturally, so we make spelling mistakes, or our hand-eye coordination is not what it used to be.
Either way, a trusty pair of smartglasses might be able to alleviate this pain by scanning your writing and pointing out mistakes. Such an app might even make suggestions for what the user had intended to write, like existing auto-correct solutions.
Features
- Scanning and understanding handwritten text in real-time
- Detecting mistakes and offering corrections
#17 Reading Pet Emotions
If you’re like me, you know what it’s like to wonder what pets are thinking or how they’re feeling. I’m more of a dog person and can usually read our canine companions with ease. Cats on the other hand usually elude me.
AR/MR could be leveraged to understand how an animal feels at any given time, perhaps suggest ways to interact with them (for example, something along the lines of “This dog is teasing you, consider participating in a game of chase” or “This cat is scared, it might be advisable to leave them be”).
Features
- Analyzing video streams of pets for behavioral cues
- Suggesting suitable reactions
#18 Pregnancy Safety
Those who have been or are currently expecting will know the inconvenience of having to check what foods a mother-to-be can safely consume. It is a constant hassle that might be eased by an augmented reality app. This could be achieved by taking images of and analyzing what the pregnant woman is about to eat. Naturally, such results should be double-checked, as not all ingredients of a meal might be visible at any given time. But this app could be a great first step in making it easier for expecting parents to know what’s safe to eat and what isn’t.
Features
- Analyzing images or video streams of food
- Displaying results of whether the food can be safely eaten by someone who is pregnant
#19 Learning to Read
Along similar lines as the book illustrations, AR/MR could be leveraged in concert with natural language understanding to help people learn how to read. Smartglasses could scan a page being read, listen to the reader attempting to read the text out loud, analyze that and check for pronunciations and gaps and such, offering help when a reader appears to be stuck.
Features
- Scanning and understanding text on book pages
- Listening to and understanding the reader
- Analyzing audio for pronunciation errors and reading speed/flow
- Offering advice when the reader appears stuck
- Highlighting sections or specific words where errors were made, and prompting the reader to reread them
- Rewarding readers for good performances and encouraging them to stay determined (did someone say gamification?)
#20 Notifications
Everyone is familiar with notifications. Got a new email, text message, or a nearby unlocked WiFi hot spot — notifications are sure to let you know.
Phones have a couple of ways of alerting their user to a new notification: Via vibration (tactile cue), sound (audio cue), and some sort of message being displayed (visual cue). While we’re all used to and pay attention to them, notifications and their cues could be revolutionized in AR/MR. Perhaps an animated mailman appears out of nowhere to deliver you a new notification via virtual mail. Alternatively, each notification could be marked by a shooting start appearing in the background alongside some related (or custom) sound effect.
Features
- Implementing notifications in well-known ways.
- Enhancing traditional notification systems.
- Innovating notifications from the ground up.