“Any train journey will get you from A to B – but where’s the fun in that?” That’s the question clients MeYouandUs put to us at our first meeting.
Imagine, they said, taking the train from London to Edinburgh and on the way landmarks come alive. A sea monster off the Holy Island coast; a herd of Toon zebras near Newcastle; or Viking villages in fields around York.
This sounded right up our street — creative, innovative & technically challenging — and so a beautiful relationship was born.
Fantasia Express uses AR and locative media to highlight landmarks and entertain users. A few miles ahead of each experience you’ll get a ping on your phone to tell you something is about to happen, giving you plenty of time to hold your device up or put it in a viewer if you’re feeling fancy.
Using markers on the train and the landmarks themselves, the AR kicks in and characters come to life. It might reveal a flock of ravens, the doorway to the home of a Yorkshire Hob or the Angel of the North heading off to a nightclub dance floor.
Train passengers in full AR mode
LNER staff member enjoying his first AR experience
Clients MeYouandUs, are a creative technology arts company. They commissioned us to develop Fantasia Express as a prototype demonstrator which they could approach potential partners with.
This Innovation Fund backed R&D project, using AR and mixed reality, had two specific, objectives. On the face of things they looked fairly straightforward, they were:
- Create an augmented reality experience inside a moving train carriage
- Create an augmented reality experience outside a moving train to be viewed from inside the carriage
Gyroscope visualisation (SLAM Data)
Challenge
Standard Spatial Localisation and Mapping (SLAM) uses a device’s gyroscope and camera to describe its location. This was no good in our case simply because the app was to be used on a moving train. Try taking a corner at speed using AR and your image will just slide off the screen.
Let there be no mistake, this was groundbreaking stuff, literally nobody had done this before. With a straightforward augmented reality project we might start with an out-of-the-box AR framework and build from there. But this was no straightforward project, so we started from scratch and built our own.
Opening AR scene model prototype
Response
Our SLAM system had to work differently for AR inside the carriage from outside. And so we took inspiration from the Mars Exploration Rovers and aerospace technology.
Inside the carriage we used visual odometry, which analyses the device’s camera feed frame-by-frame. It identifies specific feature points, then works backwards to understand where things are compared to where they were, and how they move.
For AR outside the carriage, we employed the device’s gyroscope differently making assumptions about the movement of the train itself. Sensors then estimated the motion and turn of the device and a fusion technique, similar to those used in aeroplanes, brought all this data together. Finally, we applied Kalman Filters to cancel out data drift & noise and ensure steady, accurate AR images.
This state-of-the-art tech, whilst outstanding on its own, was nothing without the stories that went alongside it.
AR ravens models
AR raven models in motion
Creative process
Anyone that knows anything about the creative process will tell you that the journey from concept to product is multi-layered. It is well researched, informed and often lengthy. But, it’s worth the effort because as with most things in life, you get out of it what you put in.
We story boarded the initial ideas MeYouandUs had come to us with. Then we drew up mood boards and ran creative workshops to work out how best to visualise the stories in an AR world.
AR 'Dragon egg' scene
Fantasia Express characters
We began by working with a BAFTA award-winning writer to translate draft scripts into our AR format. We then took them to the studio where a team of voice actors professionally recorded them and then we matched it up with our animation.
Our client had strong, defined ideas about the characters to be used, and their personalities. However, when it came to their visual design, the brief was looser. From a baby dragon to a stroppy Raven, our animators had pretty much creative control. And they ran with it, creating 3D models and artwork which practically leapt out of the screen.
The last part of the journey was field testing. This involved building contraptions in the studio which would upset standard AR and latterly sending the team out on random train journeys to test it on a moving vehicle.
AR 'Hob house' scene
Outcome
The results, as you can see, are pretty special. We’re proud to have developed such an innovative technology, and it’s something we hope to take into other arenas.
As a prototype demonstrator Fantasia Express gave our clients, MeYouandUs, an exceptional product to approach train companies, in the UK and worldwide, with. The app offers their passengers a unique experience, through this advanced use of AR, as well as creative assets to showcase it. In turn, it opens up the possibilities of commercialisation to the companies themselves.
With full support from East Coast franchisee LNER, we delivered an entertaining app. Users learn something new, and can even take pictures of what they see. What started as a regular, dull train journey has suddenly become an adventure.
Of course the potential opened up by the project is huge with Fantasia Express becoming a showcase for what is possible on the platform – if you’ll excuse the pun. Possibilities include new experiences, such as the ones we’ve seen here, but they also extend to user targeted options and commercial opportunities.
“Corporation Pop is an incredibly strong technical company that puts creativity at the heart of everything they do.”
Alastair Eilbeck
Director, MeYouandUS