Lester Madden - Professional Augmented Reality Browsers for Smartphones: Programming for junaio, Layar and Wikitude

CONTENTS

PART I

Introduction

• CHAPTER 1: Introducing Augmented Reality (AR)
• CHAPTER 2: Natural-Feature Tracking and Visual Search
• CHAPTER 3: Introduction to AR Browsers
• CHAPTER 4: Latitude, Longitude, and Where to get POIs

Chapter 1

Introducing Augmented Reality (AR)

WHATS IN THIS CHAPTER?

• What is augmented reality?
• Types of augmented reality
• Why augmented reality is useful

This chapter introduces you to the various augmented reality technologies and provides details of some applications that you may want to try.

MY AUGMENTED REALITY JOURNEY

I first stumbled upon augmented reality (AR) in February 2009 when I was working for Symbian, the company that developed the mobile OS for S60, UIQ, and MOAP smartphones. It was a Friday afternoon and I was in need of some distractions after a particularly long meeting about the annual Smartphone Show we were planning.

Like many companies, we had a forum where employees could post interesting things they have found on the web; one of those interesting things happened to be an AR video. The video had just been posted to YouTube by Microsoft Research and showed a researcher walking through the campus with his laptop screen open and a webcam filming the hallway in front of him. Displayed on the laptop screen, superimposed on top of the video from the webcam was a trail of bubbles that were emanating from an object hidden somewhere in the building. As the researcher got closer to the object, the bubbles became thicker and thicker until the object was eventually located.

I remember being amazed at how computer graphics and the live video feed had been combined in this way. Beyond simple multimedia, it was simply amazing and I knew it was going to be the next big thing for smartphones. After much frantic searching on the web, I eventually discovered a handful of AR applications (mostly available for Nokia smartphones) and I decided to create AugmentedPlanet.com as the place to document the rise of AR across mobile, web, and the desktop.

What is AR?

I like to think of AR as being the opposite of virtual reality. Virtual reality immerses the user in a computer-generated world whereas AR combines the real world with computer graphics. In effect, AR brings the computer world to us. Unlike virtual reality, which requires specialist equipment to be experienced, AR requires only a way to capture the world around you and the means to experience the computer world (typically by overlaying computer graphics in the camera window). Because the requirements are minimal, many of todays smartphones are ideal AR devices.

Before we get started on AR and its technologies, it is worth pointing out that this book takes the popular view on what AR is rather than presenting the view of the purist. Over the past year or so, many new solutions have been released to the various mobile application stores calling themselves AR applications. In addition, companies like Google and metaio are combining technologies like visual search (discussed in the next chapter) with their AR browsers, blurring the line defining AR. Ultimately, as developers, I think the How do I build that? question is far more entertaining than a discussion on the pros and cons of what is and isnt AR. With that in mind, I am going to take the all-encompassing view of AR and define AR as a technology that:

• Combines the real world with computer graphics
• Provides interaction with objects in real-time
• Tracks objects in real-time
• Provides recognition of images or objects
• Provides real-time context or data

This broader definition allows us to include technologies that are often included under the AR umbrella but are not strictly AR in the purist sense. AR, as you will learn, means different things to different people. Through blogging about AR solutions, I have learned that perceptions of what AR is and what AR isnt vary widely. Wikipedia ( http://en.wikipedia.org/wiki/Augmented_reality ) defines AR as a term for a live direct or indirect view of a physical real-world environment whose elements are augmented by virtual computer-generated sensory input such as sound or graphics.

Simply put, AR is the combining of computer graphics with live video feed. With such a simple description, however, you can see why its difficult to agree on what is and what isnt AR. For example, if you are watching a live TV sporting event and the players stats are shown during the game, is that AR? If you have a digital camera, there is good chance that it has a small screen that provides you some additional information. That information might be the battery power, the number of photos you have taken, or perhaps even information about the lighting environment. Are these examples of AR? Well, they both augment your reality by providing you with additional contextual information. To many, they are valid example of usages of AR, but others will argue that to be considered true AR, they must perform tracking to keep track of objects in real-time.

As I suggested, lets not get too focused on what AR is and what AR isnt. Instead, lets look at what is generally accepted as being AR technology.

Gravimetric AR

Gravimetric AR is the latest trend in AR applications for mobile devices. These applications are typically called browsers and will be the focus on the applications that we build in this book. Browsers use the phones gravimeter to determine the position of the user and the orientation. The browser genre was invented and made popular by two applications that originally appeared for the Android. These applications overlay computer data about objects that appear in the camera window. If the device is pointed at a tourist attraction (such as the Statue of Liberty shown in ), relevant information about the object is overlaid for the user.

: AR Browser example

AR browsers take advantage of a smartphones hardware, so when the user pans the device around, new information is displayed to provide even more contextual information. Since no tracking of objects or image recognition is used, the application can be used indoors (even where a wall obstructs the view of the target object). In fact, the camera lens of a smartphone can be covered completely because the application neither knows nor cares about what the camera sees. This has led some to argue that AR browsers are not true examples of AR because the use of the camera is largely superficial, with no real-time tracking taking place.

You will learn more about AR browsers in Chapter 3, which covers terminology in more detail and provides insight into the main browsers for which you will be building content.

Fiduciary Markers

Fiduciary markers are the truest form of AR because they are used to track objects in the real-world. As shown in , black and white squares are used as a point of reference or to provide scale and orientation to the application.

: A fiduciary marker

When the marker is recognized by the software, an action takes place. Typically that action is the overlaying of a 3D object. The software is able to track the orientation of the marker and as the orientation changes, either because the user is moving his device or moving the marker, the view of the 3D object can be changed accordingly. Ultimately, this enables the user to move around the object and view it from 360 degrees. Similarly the application may know about scale, and as the marker is moved nearer or further from the camera, the 3D object size can be adjusted accordingly. In , the ARGirl AR application created for the iPhone by APetrus shows a 3D image drawn on a marker.