Getting Started With Augmented Reality

Introduction

“If we are thinking and living in a world with colors, then why are the pixels on the screen black and white? Bring color for God’s sake.”

Isn’t there a need of enhancing the overall experience?

Making those objects appear in the real world rather than a box?

There might be a lot of you who are enthusiastic about AR/VR applications and are eager to learn more about it. You are probably wondering where and how to begin and sharpen your skills in this evolving field. However, before jumping into this exciting field, let me give you a behind the scenes experience —

Developer Experience

Let’s understand what a developer goes through while developing any AR experience. These are like the pre-requisites that a developer should know before diving into Augmented Reality.

1. The developer should know how to program in C#. That includes but not limited to data structures and algorithms and OOP concepts.
2. The developer should be well versed with Unity Fundamentals and that includes playing with Unity Physics, Animations, Sound, UI, Scripting with C# & couple of other stuff that have a steep learning curve.
3. If the developer is writing code at enterprise level then he is expected to write production level code which includes clean coding principles, highly maintainable code, writing unit tests, following design patterns and generating code analysis reports.
4. The developer should have a good understanding of the difference between world space and local space.
5. The developer should know a little about designing in 3D space and thinking outside of your phone’s screen. In theory, it’s just the environment around the developer. The challenging part is to imagine what kind of objects will fit in the user’s environment. Once you start thinking of your user in different user environments, designing a so-called perfect AR experience gets much more complicated.
6. The developer needs well designed 3D models that look as realistic as the real chair in your living room. This is not really a developer’s work but rather a designer’s work. The designer is expected to have experience in Blender, Adobe XD, Maya, etc. which takes time to learn and master. I believe that the demand for motion graphic artists and VFX artists will skyrocket when this field booms.
7. The developer should have a wild imagination(almost equivalent to a kid).

Resources needed

Alright now lets come to the resources part. What all you need to buy to get started with AR Development-

1. Developers need a device that supports AR and if you check out the prices of these phones, it’s very high for a student who is enthusiastic and wants to experiment, but also has to pay for student loans for Bachelors or Masters degree.
2. A high end PC. Your computer or laptop should literally be a beast. It should have high specifications that can run bulky applications like Unity and Blender smoothly. If it doesn’t then be ready for a “frustrating experience”.
3. If you are developing for IOS using ARKit, you should have a mac and an IOS device. You will need to make a developer account if you plan to launch the application in the app store which is $100 per year, but if you just want to develop and test an app on your phone, then that is free.

Summary

So to sum up, what are the things you need essentially to dive into AR —

For developing AR application in IOS, you need -

1. A MacBook
2. An iPhone(yes the expensive one)
3. An iPad (yes an expensive one with Lidar sensor, but cheap ones also work fine.)

For developing AR application in Android, you may use the above or if you find that crossing the budget then you will need -

1. A high end machine with at least 8GB RAM and an i5 processor.
2. An Android Phone that supports Augmented Reality.
3. Configure and install Unity in your PC
4. Configure and install Android Studio in your PC
5. Visual Studio for writing C# scripts for your Unity application.

I created a video on how to setup Unity in your system👇

Advice

Fields like Full-stack web development and Mobile App Development are very much stable and requires lesser investment in terms of time, energy and resources.

If you don’t have the above resources then calculate the Investment and the Return on Investment and then decide whether you should pursue this field or not right now. See if you have any other commitments before going down this rabbit hole.

Is your university kind enough to sponsor the resources for you? Can you get an internship at a company’s R&D department in order for you to continue doing your research according to that company’s needs?

If you already have the resources then go ahead. Setup Unity and develop your first AR experience👇

Developing an Augmented Reality Hello World App with zero coding in Unity using ARFoundation and XRInteractionToolkit

Before diving into the practical steps, let’s understand a few things first. A lot of the steps that you see ahead will seem like “magic”. It’s as if you are just following instructions, blindly. You won’t understand the underlying concepts and that’s totally fine. Get it working first. Then we can jump into the “Why” aspect of each step involved.

So what are we developing here? Let’s take a look. Here I have my phone and I will launch my app. Once I open my app you see that my phone’s camera starts capturing my room. This is the user’s environment.

As I move the phone around slowly and scan my room, it starts tracking the plane surfaces in my room. This is being done through Environmental Understanding and Plane Detection.

What is Environmental Understanding?

Environmental Understanding refers to understanding the real world environment by detecting feature points and planes and using them as reference points to map the environment. Also referred to as context awareness.

What is Plane Detection?

Plane Detection refers to the process by which horizontal and vertical surfaces are detected in your environment. These surfaces can be used to place and orient digital objects.

You can find more terminologies related to AR/VR in this article👇

Top 50 Terminologies related to AR/VR

This list is a compilation of AR/VR terminologies from all over the web. I used to refer to this list each time I came across a term, but didn’t remember what it meant exactly. I am sure you might…

AR/VR Journey: Augmented & Virtual Reality MagazineRajat Kumar Gupta

Here is a video on how to place a cube in AR (your AR Hello World project in Unity). I have enlisted the steps below so that it becomes easier for you to follow along.

Step 1

The first step is to open Unity and create a new 3D Project. Name it whatever you want to.

Step 2

Download the following 4 packages —

• XRInteractionToolkit
• XRManagementPlugin
• ARFoundation
• ARCore XR Plugin / ARKit XR Plugin

Be cautious about the versions.

Step 3

In the hierarchy window, remove the MainCamera. Right click and bring in the following GameObjects -

• ARSession
• ARSessionOrigin
• ARDefaultPlane
• ARPlacementInteractable (You will see that InteractionManager also gets added. That’s fine. Ignore it for now)
• Cube

Step 4

Add the following components to the ARSessionOrigin Game Object —

• AR Raycast Manager
• AR Plane Manager

Step 5

Add the MainCamera tag to the ARCamera Game Object

Step 6

Add the following components to the ARCamera Game Object —

• ARGestureInteractor

Step 7

Make a prefab of the Cube GameObject and then remove it from the scene.

Step 8

Select ARPlacementInteractable. Drag and drop the cube prefab in the PlacementPrefab field of the ARPlacementInteractable Script component in the ARPlacementInteractable GameObject.

Step 9

Make a prefab of the AR Default Plane GameObject and then remove it from the scene.

Step 10

Select ARSessionOrigin. Drag and drop the AR Default Plane Prefab in the PlanePrefab field of the AR Plane Manager Script component in the AR Session Origin GameObject.

Build for Android

Go to File > Build Settings > Select Android > Switch Platform. Let the switching process complete. Now click on Player Settings.

In the Player Settings window go to XR Plugin Management > Check ARCore.

Go back to the Build Settings window. Connect your testing device via USB. Click on the refresh button beside “Run Device” drop down menu. Select your device from the drop down menu and then Build and Run. When a window pops up create a new folder called “Builds”, double click it to open it, give a name for your apk and then click save. Basically all the files related to the build will be pushed inside this Builds folder now, including the APK.

Test

If you found this tutorial helpful, then kindly consider sharing it with your peers who are interested in AR/VR.

Follow me on LinkedIn or Twitter for more updates related to AR/VR