WORK:

Fantasia Express

The world's first Augmented Reality app for train journeys.

“Any train journey will get you from A to B – but where’s the fun in that?” That’s the question clients MeYouandUs put to us at our first meeting.

Imagine, they said, taking the train from London to Edinburgh and on the way landmarks come alive. A sea monster off the Holy Island coast; a herd of Toon zebras near Newcastle; or Viking villages in fields around York.

This sounded right up our street – creative, innovative & technically challenging – and so a beautiful relationship was born.

Fantasia Express uses AR and locative media to highlight landmarks and entertain users. A few miles ahead of each experience you’ll get a ping on your phone to tell you something is about to happen, giving you plenty of time to hold your device up or put it in a viewer if you’re feeling fancy.

Using markers on the train and the landmarks themselves, the AR kicks in and characters come to life, revealing a flock of ravens, the doorway to the home of a Yorkshire Hob or the Angel of the North heading off to a nightclub dancefloor.

Train passengers in full AR mode

Train passengers in full AR mode

LNER staff member enjoying his first AR experience

LNER staff member enjoying his first AR experience

Clients MeYouandUs, a creative technology arts company, commissioned us to develop Fantasia Express as a prototype demonstrator which they could go to potential partners with.

This Innovation Fund backed R&D project, using AR and mixed reality, had two specific, objectives which on the face of things looked straightforward, they were:

  • Create an augmented reality experience inside a moving train carriage
  • Create an augmented reality experience outside a moving train to be viewed from inside the carriage
Gyroscope visualisation (SLAM Data)

Gyroscope visualisation (SLAM Data)

The problem

Standard Spatial Localisation and Mapping (SLAM) using a device’s gyroscope and camera to describe its location, was no good in this case simply because of the fact that the app was to be used on a moving train. Try taking a corner at speed using AR and your image will just slide off the screen.

Let there be no mistake, this was groundbreaking stuff, literally nobody had done this before. With a straightforward augmented reality job we might start with an out-of-the-box AR framework and build from there but no such luck in this case, we started from scratch and built our own.

Opening AR scene model prototype

The solution

Our SLAM system had to work differently for AR inside the carriage than outside, and took inspiration from the Mars Exploration Rovers and aerospace technology.

Inside the carriage we used visual odometry, whereby the device’s camera feed is analysed frame-by-frame, identifying specific feature points, then working backwards to understand where things are compared to where they were, and how they move.

For AR outside the carriage the device’s gyroscope was employed differently to make assumptions about the movement of the train itself. Sensors then brought data together to estimate the motion and turn of the device and a fusion technique, similar to those used in aeroplanes, brought all this data together. Finally Kalman Filters were applied to cancel out data drift & noise and ensure steady, accurate AR images.

This avante-garde, state of the art tech, whilst outstanding on its own, was, of course, nothing without the stories to go with it.

AR ravens models

AR ravens models

AR raven models in motion

AR raven models in motion

Anyone that knows anything about the creative process will tell you that the journey from concept to product is multi-layered, well researched, informed and sometimes lengthy, but also that it’s well worth the effort for, as with most things in life, you get out of it what you put into it.

The initial ideas that MeYouanUs came to us with were storyboarded, mood boards were drawn up, and creative workshops were run to work out how best to visualise the stories in an AR world.

AR 'Dragon egg' scene

We worked with a BAFTA award winning writer to translate draft scripts into our AR format before taking them to the studio to be professionally recorded by a team of voice actors which we then matched up with our animation.

Our client had strong, clear ideas about the characters to be used, and their personalities, but when it came to their visual design the brief was looser. From a baby dragon to stroppy Raven our animators had pretty much creative control with this aspect and ran with it creating 3D models and artwork which practically leapt out of the screen.

The final part of the journey (last pun, honest) was field testing. This involved building contraptions in the studio which would upset standard AR and latterly sending the team out on random train journeys to test it on an actual, moving vehicle.

AR 'Hob house' scene

Results

The results, as you can see, are pretty special. We’re proud to have developed such an innovative and cutting edge technology and it’s something we hope to be able to take into other arenas.

As a prototype demonstrator Fantasia Express gave our clients, MeYouandUs, an exceptional product to approach train companies, in the UK and worldwide, with. The app offers their passengers a unique experience, through this advanced use of AR, as well as creative assets to showcase it, and in turn opens up the possibilities of commercialisation to the companies themselves.

With full support from East Coast franchisee LNER, we delivered an app where the user is entertained, they learn something new, and can even take pictures of what they see. What started as a regular, dull train journey has suddenly become an adventure.

Of course the potential opened up by the project is huge with Fantasia Express becoming a showcase for what is possible on the platform – if you’ll excuse the pun. Possibilities obviously include new experiences, such as the ones we’ve seen here, but they also extend to user targeted options and commercial opportunities.

“Corporation Pop is an incredibly strong technical company that puts creativity at the heart of everything they do.”

Alastair Eilbeck
Director, MeYouandUS