My "virtual and augmented reality" week: simply mind–blowing!, 11/06/17, Giulia's blog

June 11, 2017

My "virtual and augmented reality" week: simply mind–blowing!

Seeking out new experiences - you know, they always tell you you need to "get out of your comfort zone" and "think out of the box" - last month I enrolled in a 5-day course on Augmented Reality, organised by Fastweb Digital Academy and taught by two young guys, Luca Raho and Alessio Marzoli, of Forge Reply, a company, within the Reply Group, specialised in videogame development and virtual reality projects.

headset1

I'm not a developer nor I work for a tech company. So you might wonder how I came about taking such a course. Let's start with what we all already know: digital tools and technologies are having a profound impact on the global economy. To make the most of these opportunities and stay ahead of competitors, companies in all industries need to recruit digitally engaged professionals.

Despite this urgent demand, levels of digital skills worldwide are staggeringly low. To tackle this issue, many initiatives have been launched and the Fastweb Digital Academy is one of these, with the additional benefit that courses are free to attend thanks to the financial support of Cariplo Factory.

This was an opportunity too good to be missed: I wanted to do something very strange and unfamiliar and that's why I ended up studying Augmented Reality.

As you would expect in any good course, we started with a bit of theory to understand the difference between virtual reality (VR) and augmented reality (AR).

To put it simply, VR is a digital replica of the real world and you need some kind of headset to access it (such as google cardboard, Playstation VR, Samsung Gear VR). On the other hand, AR is about adding digital content to the real world. An example of AR I'm sure you know about, is Pokemon Go where your smartphone camera shows the real world around you but with the addition of a Pokemon.

AR software

How does AR work?

The second part of the course was very practical and I was introduced to the programming language C# and the software Unity, used to create 2D and 3D games. These lessons were very technical and, honestly, the most difficult part. In fact, it takes years of study to master a programming language. Notheless, it's always useful to be exposed to something new: now I know what a game developer does and all the knowledge and skills he needs to master!

In fact, to create a believable virtual world, you need to know about physics (if your character in the game throws a ball up in the air, this ball can't travel forever up in the sky but needs to fall back to the ground at a certain point), perspective and geometry (you need to know that objects far away from the camera must appear smaller) and then it's important to add the correct light so that objects have shadows in the proper position as they have in the real world.

While discussing these issues in class and having studied humanities at college and university, I could not stop thinking about Leonardo da Vinci and his studies on geometric perspective - just think about the perfect space he painted in the Last Supper - studies on colours when he painted mountains in blue tones knowing that this is how they appear to our eyes when looking at them far in the horizon and on lights when he added shadows to his figures to convey the idea of a 3d body.

I was introduced also to polygon mesh, a collection of faces that defines the shape of a polyhedral object in 3D computer graphics and solid modelling. With his fascination for geometry, the golden ration and the human body Leonardo da Vinci would have been a superb 3D modeller. The idea that shapes from the real world can be recreated with the use of cubes can also be found in Cezanne's paintings, whose works area at the origin of Cubism.

This brief digression into the history of art can just show how the humanities and the STEM subjects are not at all two worlds apart.

Let's go back now to AR to see how the "magic" effect is created. To put it very simply the software recognises a "marker" and then shows you the additional digital content. A marker can be anything: an image, audio, gps coordinates etc.

There's also a special type of markers, Vumarks (called like that because they were invented by a company called Vuforia. Vumarks are an evolution of bar codes. Vumarks allow the freedom for a customised and branded design while simultaneously encoding data and acting as an AR target.

my ar app

As there's no better way to learn than trying out things for yourself, the course tutor guided us in the creation of a simple AR application: choose an image and instruct the software to show a cube when it detects the target image in front of the camera. It sounds very easy to do but it's not, believe me!

As target image I chose the front cover of a book by Roald Dahl and instructed the software to show a blue cube when it recognised the target image in front of the camera. You can see the result in the picture to the right: it worked!

Area 360 - Applications of VR and AR

On the last day of the course we students headed to Lomazzo, near Como, to visit Area360 and experience first hand the latest headsets for virtual and augmented reality. Area360 is the Reply Center featuring laboratories dedicated to the development and testing of immersive virtual reality and augmented reality solutions. We definitely had fun!

In the photo collage below you can see me in a cave, a virtual reality room: it's a three-wall setting + floor and the virtual environment is projected on these walls. You need a VR headset to see it (and those cool slippers too!). The VR glasses also have "antennas"so that the computer knows when you turn your head and it projects accordingly images on the walls.

AREA360

Apart from the obvious use of these technologies in the gaming industry, virtual reality is also used for design purposes and prototyping. Let's say, you're an architect working on a new office development. Thanks to virtual reality, your client can already experience the office, virtually walk into it and give feedback on how the design could be improved.

Virtual reality supports also companies making custumized products. At Area360 we were told that one of their clients is a luxury car manufacturer which makes customized cars. The manufacturer uses VR to show the future car owner how the car will look like, inside and outside, in order to get his approval before going into production.

Another interesting use in the health sector. For example, Forge Reply built a cave for the hospital Istituto Auxologico in Milan. The cave is used for cognitive rehab therapy.

Then I tried on also Microsoft Hololens (the headset I'm wearing in the main picture opening this post).

It's a device for mixed reality. Mixed reality lets the user see the real world while adding virtual objects. These virtual objects are anchored to the real world. Let's say there's a virtual castle sitting on top of your kitchen table: in mixed reality, if you walk around, the object holds its position on the table and if you get closer, the castle appears bigger to your eyes. It's not like in AR where the virtual object sits in front of your eyes and as you move around, just the real world changes around you. The virtual object stays the same with regard to its position and its size as perceived by our eyes.

For mixed reality to work, great computational power is required. As you move around, the software needs constantly and quickly re-map the environment around you and show the virtual object accordingly. In fact, with Microsoft Hololens you're "wearing a pc" as the device runs Windows operating system.

By the way, if you wonder what I'm looking at in the picture where I'm down on my knees, wearing the Hololens, it's a 3D car, perfectly defined in all its details. I could even see inside the cabin!

accademia carrara

Thrown into this frenzy of virtual, augmented and mixed reality, now that the course is over I'm still trying any possible device I come across. While visiting the Accademia Carrara Museum in Bergamo last weekend I took the augmented reality guided tour (picture to the right). It's a set of glasses which can show additional content in front of your eyes, while still being able to see the real world around you: just imagine something to be projected on a portion of the lens, so that you can see it right in front of you.

The whole device is not so comfortable to wear, though: a piece of hardware, the size of a smartphone, hanging from your neck, the glasses and the also ear pods....Nontheless, I'm very confident as these technologies are evolving fast and constantly improving the user experience.