360 Film Experimentation

A feature of my app concept is to use footage from within local towns. I would like to use 360 film if possible, for several reasons.

1. It will provide a more immersive user experience
2. It will provide opportunity to hide clues to the game within the virtual environment which are a little more challenging to find
3. It will generally make for a more unique and innovative feel to the app

There are several areas I need to explore and experiment with in order to test the feasability of this idea.

I would like to explore the possibility of shooting studio footage – for example of character acted scenes, using a green screen which would be integrated into the town footage. This would mean I could reuse the same studio footage in multiple towns and therefore offer the app to a greater number of users. As my theme for the app is Film Noir, I could also use other props and footage such as period vehicles and so on.

I have began to experiment with 360 footage and stock character footage using Adobe Premier and Adobe After Effects.

I started by opening a short 360 video from a local woodland in Adobe Premier. I then applied some filters to create a black and white affect.

Next I took the footage into Adobe After Effects to add rain and a more darker ambience. The rain and darkness will be a common theme throught my ‘Film Noir’ style app.

My next tasks was to test superimposing footage filmed on a green screen. For testing purposes I grabbed a free clip from Pixabay. Using the Keylight plugin provided with After Effects a transparency is created allowing the content to intergrate within the existing scene.

In order to make the 2d footage work within the 360 environment, I had to convert the layer to a 3d layer and use VR tools within After Effects to anchor down the layer within the 360 sphere so that it remains in place even when the viewer navigates around the 360 environment.

Here is the 360 version on Youtube:

Conclusions and Next Steps

I found the experiments with 360 film quite promising. There are a few areas I need to iron out though. The output from After Effects is producing an ill formatted sphere. This can be seen in the Youtube video above. I am not sure why this is happening. The view is fine within After Effects so I think it is a problem within the export settings. The VR controls in After Effects are still quite daunting to me. But I’m pleased I’ve been able to apply some effects like the ‘Noir’ filter; the rain and mist effects as well as bring in the extra character.

My next steps are to iron out the export issues and make another short clip using new 360 footage from my town which I intend to capture this coming weekend. I will also film some test acting scenes using a greenscreen and try getting together a solid example of a clip suitable for use in my app.

I would then like to test the integration of 360 film into a Flutter app. To see what controls are available.

One consideration that I am now aware of, is that the output file sizes are quite big. This could be an issue with a mobile App. I will seek to keep the clips as short as possible and try to loop and use Flutter for the audio rather than the video. This way I can keep the video files as compact as possible. It will be a good idea to look at managing downloading forthcoming videos for the gameplay and removing previous ones, so that the app is not a huge download size and doesn’t become more and more bloated either.