iOS

Recording ARKit Videos & Animated GIFs Using ARVideoKit


Mobile augmented reality has been a trend recently and it’s been implemented in many applications to provide more interactive, fun experiences to share with people. Hence, major augmented reality frameworks, such as ARKit, enable developers to easily implement complex computer vision features into their apps and make AR a reality on mobile 😉.

However, ARKit and similar frameworks do not simplify the process of recording videos with augmented reality components, although users LOVE sharing their funny experiences while using augmented reality features (referring to face filters & trends like the AR hot dog), which led developers to go with alternative solutions such as screen record or screenshots.

In this tutorial, we will demonstrate how to record videos and GIFs that contain augmented reality components without the use of screen record or screenshots. To do so, we will be using a framework built specifically to simplify the process of rendering and recording ARKit videos, called ARVideoKit. This framework is built using Apple’s ARKit, AVFoundation, Metal and CoreMedia to take away the complex parts of rendering an ARSCNView or ARSKView content with the device’s camera stream.

Let’s Get Started!

To begin, we will first download the latest release of ARVideoKit by going to the repository releases page and download Framework.zip file.

Download ARVideoKit

Then we will create a new ARKit project on Xcode. For this tutorial, we are creating ARKit + SpriteKit project.

Create Xcode Project

When the project is created, Xcode will automatically generate a sample SpriteKit code to use. Although it’s good for a demo, but we will modify it a little bit to add our user interface, video recorder, GIF recorder, and display more different emojis 😇!

Adding the Framework

We will start off by adding ARVideoKit to our project! To add the framework properly, do the following steps:

  1. Create a folder in your project folder called Frameworks.
  2. Copy the downloaded ARVideoKit.framework into the Frameworks folder
  3. copy_framework
  4. Drag ARVideoKit.framework into your project target’s embedded binaries. Make sure to check “Copy items if needed”.
  5. add-arkitvideo-framework

Now let’s configure the framework in our application delegate, by taking the following steps:

  1. Import ARVideoKit in AppDelegate.swift by adding the following statement:
  2. Add the following method in AppDelegate.swift class to allow recording videos & GIFs in different orientations.

Your application delegate file should look like this:

Adding User Interface

We will build the user interface programmatically instead of using Interface Builder. To keep things simple, we will create 3 simple buttons: Record/Stop, Pause/Resume, and Capture GIFs.

To do that, declare the following properties in ViewController class:

Next, add the buttons as subviews of the View Controller. Insert the following lines of code in the viewDidLoad() method:

In order to handle the button actions, we then create three action methods:

Now, back in viewDidLoad(), add buttons’ targets and connect them to the methods created above:

Before we moving to the next section, let’s run the app and see what you’ve built so far. If you’ve followed me correctly, you will have a simple ARKit app with three buttons on screen. Remember that you have to test the app on a real device instead of the simulator.

arkit-demo-withbutton

Implementing the ARVideoKit Framework

Now it’s time to enable the recording feature! We will implement the ARVideoKit framework into ViewController.swift. The very first step is to import the framework:

Next, create a variable of the type RecordAR. RecordAR is an ARView sub-class that renders an ARSCNView or ARSKView content with the device’s camera stream to generate a video, photo, live photo or GIF.

In the viewDidLoad() method, initialize RecordAR with ARSKView and specify the supported orientations:

To prepare the recorder, insert the following statement in the viewWillAppear(_ animated: Bool) method:

Lastly, to “rest” the recorder when the view disappears, insert the following line of code in the viewWillDisappear(_ animated: Bool) method:

Developing the Record/Stop and Pause/Resume Functions

Now that the preparation of the RecordAR variable is ready, let’s move on to the implementation of record and stop features.

For the recording action method, update the method like this:

In the block of code above, we are checking if the video recorder status is ready to record. The app would start recording a video of your ARKit scene. Otherwise, if the recorder is currently recording or paused, the app would stop the video recorder and export the fully rendered video to the camera roll.

Next, we will implement the pause/resume feature into the pauseAction(sender:UIButton) method, by update the pauseAction method like this:

The block of code above is pretty straight forward. We first check if the recorder is currently in the recording state. Then the app will pause the video when the user selects the pause button. Otherwise, it resumes recording.

Now we have something to test! Before running the application on your iOS device, we need to make sure to add the usage description of the camera, microphone, and photo library in the app’s Info.plist.

To do so, add the following to the plist source code:

Alternatively, you can add the properties using the Property Editor:

infoPlistUsage

Now let’s run it!!! 📲🎊 Tap to record button and start recording your AR video.

Capture GIFs

Finally, let’s implement the GIFs function so that you can capture animated GIFs. Update the gifAction method like this:

Modifying the SpriteKit Content

In this final part we will modify the SpriteKit content to show some different emojis in the AR space 🤗🤓🧐🔥.

We will first create a variable that returns a random emoji from an array of emojis. By using the C-based arc4random_uniform() function, we will be able to retrieve a random number from 0 to the count of the array.

To do so, create the following variable as a global variable (place it after gifButton) in the ViewController class:

Next, edit the view(_ view: ARSKView, nodeFor anchor: ARAnchor) -> SKNode? method like this:

We just replace the static text of the SKLabelNode with the newly created randoMoji.

That’s it! Have Fun

You can now run the application on your device and use it to record ARKit videos and GIFs! To download the full project, you can download it on GitHub.

ARKit+SpriteKit-GIF

What do you thing about this tutorial? Please leave me comment below and share your thought.

iOS
iOS Programming 101: How To Create Circular Profile Picture and Rounded Corner Image
iOS
Introduction to the Accelerate Framework in Swift
iOS
Using AsyncDisplayKit to Develop Responsive UIs in iOS
Shares