I have spent the previous six weeks exploring software kits, programming frameworks and 3D graphics techniques. I have been focused on developing skills which are relevant to building apps for virtual and mixed/augmented reality; otherwise collectively known as XR – eXtended (or in some cases ‘cross [x]’) reality.
To start putting these skills to good use I have decided to embark upon a real-world cross-reality app project. The project is called “Lovelock World” and is inspired by recent news articles about the phenomenon of Love Locking – the practice of attaching a household padlock to a part of a historic bridge or other structure. It is happening in cities all around the world. People do this as a souvenir of their visit to the place and in most cases the lovelock is placed by a romantic couple who also leave their initials or a short message on the lock. The reason why this phenomenon has hit the news is because the practice has become so popular that some authorities are concerned about the long-term heritage degradation and even the potential danger it poses to visitors. In some cities, including Chicago, Berlin and Paris, authorities have been removing the locks. My idea is to create an app which will allow players to place virtual lovelocks at famous landmarks or anywhere in the world without the risk of causing heritage degradation!
My app targets a relatively broad demographic of users. Generally speaking, anyone one who is in some kind of relationship [even an imaginary one!] can use the app to place a virtual lovelock at a real world location. My app is particularly appealing to those who are concerned about the preservation of historic sites and who love to visit world cities; but also, as you will find out further along this article, people who prefer the countryside, too.
The end-user of the app is the player. The player wants to create a virtual lovelock so that they can attach it to a real-world location without causing physical damage.
In addition to the simulated real-world action of adding a physical padlock, the app opens up further opportunites. For example, a player wants to add a selfie to their virtual lovelock so that it becomes more personalised.
The above two stories, are part of a broader epic which fulfills the player’s basic requirement to be able to place their virtual lovelock in the realworld. Further epics include stories for creating special premium lovelock designs, media and even the ability to visit the location to see their lovelock (and those of others) in virtual reality.
In this article I will focus on the theme of a user adding a lovelock to a location and the technical challenges I face in implementing this feature as well as my response to initial user feedback about the app concept.
As the owner of the app, my first task was to carry out some market research to see if my idea already existed in the marketplace. I was pleased to find that there are currently no similar apps using cross-reality in either of the two main app stores I wwill be targetting. This means there is plenty of scope to further develop my idea. I know from news articles that lovelocks are very popular and I also know that a virtual version would be much-welcomed by local authorities who are responsable for heritage sites being damaged by the practice.
I was able to find a number of web based lovelock services. But, these services do not allow a user to place a lovelock in-situ while at the actual location. This is a key feature and a critical requirement for my app so that the lovelock is actually ‘out there’ in the world; albeit in a virtual format. The web app versions I saw, merely place graphics on a web page. One such example is virtual-love-lock.com
As a creative developer, I want to be able to build a basic version of the app first so that I can test the concept with real users. This is so that I can get some critical feedback before the app becomes more complex and therefore less straight-forward to refactor.
The initial version of the app will use a mobile device’s AR (Augmented Reality) capabilities. At the time of writing not all phone devices have this capability and this is something I must take into consideration. However, AR is becoming more and more popular as has been proven with the success of AR games like Pokemon Go. The makers of that game have since release a Harry Potter AR game, too. This is a good sign to expect adoption of AR devices to continue an upward trend.
There are two general approaches to AR development. One is to natively develop an app using a device’s operating system’s (OS) specific software development kit (SDK). This means that the native approach requires the development and maintenance of a distinct app to target each platform. I will be targetting Apple iOS and Google Android operating systems, as these are the two most widely used platforms. This means native development would, if I chose this approach, involve building one app using Java or Kotlin programming languages for the Android app and another using Swift for iOS. Each platform also has it’s own specific AR SDK, ARcore for Android and ARkit for iOS. Furthermore, each device has it’s own compiler and integrated development environment – Android Studio or Xcode (for iOS). The advantage of this approach is that I would be able to build a truly custom app built specifically to my project’s requirements. However, the inconvenience of native development would mean that it takes much longer to build as that everything has to be done twice; once for each of my target platforms. While, I am quite confident with Android based development, I have very little experience with Xcode development. I would therefore need to improve my knowledge of Swift (for iOS). I would be happy to do this, however, this would involve alot of work, which in realworld terms equates to a longer timescale to release and a greater number of man-hours for development.
The alternative AR development approach I researched was using a cross-platform AR SDK. The key advantage with this approach is that I can create apps for both Android and iOS from single codebase. The main inconvenience that I have identified with this method is that the advanced cross-platform SDK’s are not open-source. This means that a costly commercial license is required. However, that cost could be offset against the dev time which is saved, aswell as being justified by the income generated from in-app purchases. There are a couple of fairly mature open-source AR SDK’s available. But, they only provide very basic features such as marker based triggers, which require a 2D pattern to be placed in the realworld to trigger the app. Whereas, a prerequisite for my app is that it works in a markerless way; that is to say that the app will work without the need for external assets like images, posters, or smartcodes.
Vuforia or Wikitude?
I tested Vuforia by integrating the SDK for Unity. I found the features offered by Vuforia to be quite limited. The features, at least in the mainstream version (they also offer a bespoke version), are very much aimed at a broad usage scenario and not very flexible for specific cases. Part of my research was to identify how realistic I could simulate the action of placing a lovelock to a physical structure. With Vuforia this would not be possible at all. The only potential solution offered within the SDK, would be to create a database of 3D models of cross sections of actual bridges. This would present a considerable amount of work to gather this kind of data and would restrict the app to only being usable with a selection of pre-scanned bridges and structures.
On the other hand, Vuforia offers features for ground plane and mid air positioning. This enables the app to detect flat surfaces or floating spaces in which to place objects. I tested this functionality using my Google Pixel 3, which has the latest version of ARCore. The process of creating basic AR interactivity with Vuforia was quite straightfoward.
Wikitude, the other SDK supplied I researched, offer a variety of license options. The closest match to the Vuforia plan that I discussed above, costs a whopping £2700 per year! They do offer a free version for start-ups. But, the conditions are restricted to companies which are less than two years old and the license does not include any SDK upgrades. Wikitude does include a larger feature-set than Vuforia, such as scene recognition which could be particularly useful within the context of my app for recognising structure types. But, I do not want to have to set up a new company, specifically for this project, at this early stage, or invest £2700 until I have proven the app concept to be a viable one. My priority is to get a working version of my app into the marketspace, with a critical set of features so that I can start collecting user feedback and have something tangible to show to potential investors and/or crowdfunding sites.
Initial Concept Feedback
The key user story for my initial version of the app involves the player placing a virtual lovelock into the real-world space. I shall achieve this using Augmented Reality technology. I had originally hoped to be able to simulate the process of adding a lock to a structure as closely as possible to the real thing i.e. structure detection and placing lock within a realistic context. However, there are two challenging factors which I have since had to take into consideration. The first is the technological constraint due to cost and time. I have already discussed this above. The second factor is based on feedback from people with whom I have spoken about my app concept. The most common feedback I received was the desire to be able to create lovelocks anywhere; instead of being restricted to bridges and similar structures. For example, one person mentioned they would like to be able to create lovelocks in more intimate and secret places such as a corner of a field in the countryside where they had enjoyed a romantic picnic or other places like beaches, mountains, woodland. This idea conflicts somewhat with my original aim to make realistic in-situ lovelocks. But, it would actually help me bypass the technical challenge of having to implement realistic locks which detect and attach to structures. In terms of cost and time, the most accessible AR approach I am able to use is Plain and Mid Air placement. These techniques are supported in Vuforia and provide the opportunity for players to be able to place their lovelocks just about anywhere.
To reduce costs, I will develop my app using the Vuforia SDK. I shall allow players to create two types of animated lovelock: in-air and on-a-surface. This approach will allow me to respond to initial user feedback and it also enables me to fix achievable goals in terms of the cost and complexity of the required development work.