Building an Interactive Voice App Using Custom Siri Shortcuts in iOS 12

Siri Shortcut is a new API that Apple introduced in iOS 12 built on top of SiriKit framework. It provides ways for developer to create custom shortcut to the activities that the user often used within the app. User invokes the shortcut by issuing a voice command to Siri, the actual phrases can be customized by recording custom phrase associated with the shortcut.

Since the release of SiriKit in iOS 10, Apple has been limiting the usage of Siri within several built in domains that Apple provided, such as ride sharing, payment, messaging, and etc. There is no options for developers to provide their own custom intent. But with Siri Shortcut in iOS 12, finally Apple has provided the framework for developers to build their own custom intent that can be configured with custom parameters and responses that can at least be used for Siri Shortcut.

The ability for developers to create custom intent opens many ways for developers to create interaction with Siri, for examples:

1. Order something (e.g. pizza, coffee, breakfast 😋);
2. Search information that can connects to any APIs;
3. Do any custom action that we want Siri to perform;
4. Many more!

Apple encourages developers to only create meaningful Shortcuts for user, for example provide shortcut for activities that users like to repeat when they use the app. It’s not recommended to create shortcut for an activity that users just perform once in the app. I really encourage you all to watch Apple WWDC 2018 videos below to learn more about Siri Shortcut from Siri engineers from here and here.

What We Will Build for this Siri Shortcut Demo

In this article, I would like to provide a simple example on how we can use Siri Shortcut to perform search on upcoming movies using The Movie DB API, then display the result using custom Intent UI Extension.

In order to build the demo app using Siri Shortcuts, here is what we will go through:

1. Create New Project and setup Podfile.
2. Create Shared Framework to share between main App and Extension.
3. Build Movie List View Controller
4. Create Siri Intent Definition File
5. Create Intent Extension
6. Create Intent UI Extension
7. Donate Intent from View Controller.
8. Test Shortcuts

Create Project and Setup

Create a new project using Xcode and initialize Podfile using pod init command from terminal. For the dependencies, we are going to use Kingfisher image download library to handle image download and cache when displaying thumbnails of movie posters.

After pod install, open the project from the generated .xcworkspace file.

Create Shared Framework to Share Code Between Apps & Extension

Click the project within the Project Navigator. In editor area inside the targets on the left, click the + button to add new Target. Select Cocoa Touch Framework from the options, and set the name of the framework to MovieKit. This framework will contain all the Model, Remote Repository, and UI Components for the Movies that can be shared between the App Target and App Intent Extensions. Remember that the Extension in iOS runs on different processes separated from the main app.

Create Movie Model with Codable

First, within the MovieKit folder in the project navigator, create new File with the name of Movie. This file will contain all the Models from Movie, Video, Cast, Crew that conforms to Codable protocol to make it easier to decode the JSON response from the The Movie DB API.

Create Movie Repository

Second, create a new File called MovieRepository. This is the class that will provide interface for the client to fetch movies from Movie DB API. It uses enum Endpoint so the client can pass the endpoint such as popular, now playing, upcoming, top rated movies to access the resources they want to consume. It also provides method to fetch single movie by passing the id of the Movie. Make sure to create your own API key from The Movie DB website.

Thanks to Codable protocol that our Model uses, JSONDecoder object can decode the response data automatically. We just need to configure the key decoding strategy to convert from snake cases because the API uses snake cases for the naming convention of the properties. For the date decoding strategy, we specify YYYY-MM-DD as the date format convention from the API.

Create Movie Collection View Cell

Next, create a new File called MovieCollectionViewCell, with the subclass of UICollectionViewCell and also create the XIB file. This cell will display the poster and the average vote rating of the movie.

Inside the XIB, set the size of the Cell to 160x240. Then, drag a UIImageView and set the constraint for leading, trailing, top, bottom to the superview with the constant of 0. Next, drag a UILabel for the rating, set the font to Caption 1, and the trailing and top constraint constant to 0 to the superview. At last, drag a UILabel for the title, set the font to Headline, and set the bottom, trailing, leading constant to 8, and the lines to 3.

Inside the MovieCollectionViewCell, connect all the outlets from the XIB. We also create a property for the Movie and create a didSet observer to setup the UI based on the property from the Movie object. To download the image, we use Kingfisher as the Image library to handle download and caching.

We finally created all the components that we will share between the Main app and the extensions later. Now, let’s move back to the main App Target. Make sure that you add the new MovieKit target inside the Podfile and add the Kingfisher pod inside like so. Then, run pod update to make sure the project compiles. You can also remove the Kingfisher dependency from the main App Target because it won’t be used there.

Build The Main App UI with Movie List View Controller

As we won’t be using storyboard for the app, delete the main.storyboard file from Project Navigator and the Main Interface in General tab inside the project.

Create a new File called MovieListViewController with the subclass of UICollectionViewController. To display grid interface with 3 items on each row, we will be using UICollectionViewFlowLayout.

We will use a custom initializer that accepts 2 parameters, enum Endpoint and MovieRepository. The enum Endpoint determines the resources of movies that we want to fetch, while the MovieRepository is the remote data source that provides the interface for the client to fetch the data. We keep array of Movie as an instance property that will be reload the collection view every time the value of the array change.

In viewDidLoad, we setup the collection view layout and set the item size of each cells by dividing the width of the screen by 3. We also register the XIB for the MovieCollectionViewCell and assign it reusable identifier. At last, we utilize UIRefreshControl with a target selector that will fetch the data every time the user do pull to refresh on the collection view and assign the response to Movie array to reload the UI with new data.

Inside the AppDelegate we will set up our UIWindow root view controller using a UITabBarController containing MovieListViewController for all the cases within the enum Endpoint.

We finally finished our main app! Before we try to open the app’s info plist and allow the App Transport Security Settings to allow arbitrary load from all domains, finally build and run the App to see it in action!.

Create SiriKit Intent Definition File

Next, create a new File and use SiriKit Intent Definition File as the template. Click on the File to open the Intent Editor. Within this editor, we can create our custom intents and response. It will also generate the classes for our targets, in this case the app and the extensions we will create later.

Click on the + button at the bottom to create new Intent we call Movies. Set the category to View, title to View Movies, and description to View list of movies. Then, add a single parameter named endpoint with the type of String. At last, create one shortcut types with endpoint as the parameter combination. Set the title to use the endpoint as the input like the screenshot below. Make sure you check the Supports background execution checkbox.

For the response, add one property named type as String. Then, set the response failure and success templates phrases like the screenshot below.

Create Intent Extension

Next, create a new Target using Intents Extension as the template. Set the product name to MoviesIntent and make sure to check include UI Extension checkbox. Xcode will also ask to activate the build scheme. Make sure that you confirm to activate the scheme for the extensions.

Go back to the Intent Definition file and include the created extensions as the target for the Intent classes.

Open IntentHandler file, here we need to handle the intent using our MoviesIntentHandling protocol that is generated from the Sirikit intent definition file. Check if the intent is the MoviesIntent. Then, we return a MovieIntentHandler class that implements the MovieIntentHandling to handle the intent.

Inside the handler, we make sure that the endpoint parameter is existed within the intent. Then, complete the intent by passing success as response code and passing the endpoint for the response type property.

Create Intent UI Extension

Open the MainInterface.storyboard file inside the MoviesIntentUI extension, then drag a collection view as the root view for the view controller.

Inside the IntentViewController, create an outlet property for the collection view and reference it within the storyboard. In viewDidLoad, we set the view controller as the UICollectionViewDataSource. We also register the MovieCollectionViewCell nib to the collection view.

Inside the configureViewmethod, we check the intent response parameter is available. Then, initialize the enum endpoint using the response.

After that, we use the endpoint to fetch the data from the repository. Inside the success handler, we set the size of collection view layout item size and assign the result to the movies instance property that will reload the collection view.

Donate Intent from Movie List View Controller

At last, to add the intent as the shortcut, we need to donate the intent inside our MovieListViewController. Also, make sure that you add Siri privacy usage description authorization inside the app info.plist to authorize Siri usage within the app.

We will create a function called donateIntent. Inside we request authorization for Siri. Once it is authorized, we create the MoviesIntent and assign the view controller endpoint description as the parameter and the suggested invocation phrase for the shortcut. At last, we create INInteraction object passing our intent and donate it. We will invoke this function in the view controller viewDidLoad.

Test Shortcuts

Build and run the app. Navigate to all the tabs to donate the intent for each endpoint. After that, navigate to setting -> Siri and search to see the recommended shortcut. Our app shortcut should be available to add. Add the shortcut and record the custom phrase to trigger the shortcut.

At last, try to speak hey Siri to trigger Siri then speak your custom recorded phrase to trigger the shortcut.

Conclusion

Building voice interactive app using Siri Shortcut is really amazing and pretty straightforward using the SiriKit Framework. With this, we can provide users new simplified experience for the action they like to use just using a customize voice command. I really believe that voice interaction between human and machine are going to become more and more advance in upcoming years. This kind of combination between technology and arts really makes my heart singing as a developer that can use this technology as building blocks to build solutions for the people and businesses.

This is a guest post by Alfian Losari. The article was first published on Medium.