This intermediate book (over 950 pages) uses a problem-solution approach to teach you Swift programming and covers some of the common APIs provided by the iOS SDK including Core ML and ARKit. The book and the source code bundled are now compatible with iOS 15 and Xcode 13.

This book is written for developers with some experience on the Swift programming language and with an interest in developing iOS apps. It is not a book for beginners. But if you have some experience in Swift, you will definitely benefit from this book.

While you can start reading from chapter 1 of the book, but this is not a must. Each chapter stands on its own, so you can also treat this book as a reference. Simply pick the chapter that interests you and dives into it.

What You’ll Learn in this book

Here are what you will learn in this Intermediate Swift book:

Chapter 1 – Adaptive UI Using Universal Storyboards and Size Classes

Chapter 2 – Creating Table View Sections and Index list with Diffable Data Source

Chapter 3 – Animating Table View Cell

Chapter 4 – Reading and Parsing JSON

Chapter 5 – How to Integrate Twitter and Facebook Sharing

Chapter 6 – How to Create Email with Attachment

Chapter 7 – Sending SMS and MMS

Chapter 8 – How to Get Direction and Draw Route on Map

Chapter 9 – How to Search Points of Interest Using Local Search

Chapter 10 – Audio Recording and Playback

Chapter 11 – Scan QR code using AVFoundation Framework

Chapter 12 – Working with URL Schemes

Chapter 13 – Working with Camera

Chapter 14 – Video Capturing and Playback using AVKit Framework

Chapter 15 – Display Banner Ads using iAd

Chapter 16 – Using Custom Fonts

Chapter 17 – AirDrop

Chapter 18 – Building Grid Layouts Using Collection Views and Diffable Data Source

Chapter 19 – Interacting with Collection View

Chapter 20 – Adaptive Collection View

Chapter 21 – Building a Weather Widget Using WidgetKit

Chapter 22 – Building Slide Out Sidebar Menus Using Objective-C Libraries

Chapter 23 – View Controller Transitions and Animations

Chapter 24 – Building a Slide Down Menu Like Medium App

Chapter 25 – Self Sizing Cells and Dynamic Type

Chapter 26 – XML Parsing and RSS

Chapter 27 – Apply a Blurred Background Using UIVisualEffect

Chapter 28 – Using Touch ID for Authentication

Chapter 29 – Building a Carousel-like User Interface

Chapter 30 – Working with Parse

Chapter 31 – How to Preload Existing Data into SQLite Database

Chapter 32 – Connecting Multiple Annotations with Polylines and Routes

Chapter 33 – Using CocoaPods in Swift Projects

Chapter 34 – Building a Simple Sticker App

Chapter 35 – Building iMessage Apps Using the Messages Framework

Chapter 36 – Building Custom UI Components Using IBDesignable and IBInspectable

Chapter 37 – Using Firebase for User Authentication

Chapter 38 – Google and Facebook Authentication Using Firebase

Chapter 39 – Using Firebase Database and Storage to Build an Instagram-like App

Chapter 40 – Introduction to CoreML (Available in late March)

Chapter 41 – Building AR Apps with ARKit and SpriteKit

Chapter 42 – Working with 3D Objects in Augmented Reality Using ARKit and SceneKit

Chapter 43 – Use Create ML to Train Your Own Machine Learning Model for Image Recognition

Chapter 44 – Building a Sentiment Classifier Using Create ML to Classify User Reviews

Chapter 45 – Working with Image Tracking in ARKit

Don’t Miss the Launch Discount

If you want to learn more about the book and download a sample, please head over to this link. From now till Mar 6 (23:59 PST), you can use the discount code “NEXTLEVEL” at checkout to receive an extra 20% off.

For beginners, if you want to learn more about Swift, you can check out our beginner book at https://www.appcoda.com/swift.

For SwiftUI, you can check out our new Mastering SwiftUI book (https://www.appcoda.com/swiftui).

It is very common that you need to display web content in your apps. The iOS SDK provides three options for developers to show web content: Mobile Safari, WKWebView, and SFSafariViewController. In iOS 14 (or later), the SwiftUI framework provides a view called Link for you to open a web link in mobile Safari. The usage is very simple. You just need to specify the text of the link and the destination URL like this:

Link(destination: URL(string: "https://www.appcoda.com")!, label: {
    Text("AppCoda")
        .foregroundColor(.orange)
})

This presents a text link in orange. When a user taps the text, the app opens the link in the Safari browser. You are not limited to use a text link. In the label closure, you can change the code to present an image link using an Image view or other custom views.

However, the current version of SwiftUI doesn’t come with an embedded web view. To display web content within your applications, you will need to tap into the UIKit framework. In this tutorial, I will walk you through the procedures to adopt WKWebView in SwiftUI projects.

Adopting WKWebView Using UIViewRepresentable

Assuming you have some experience with the integration of SwiftUI and UIKit, you know we need to adopt the UIViewRepresentable protocol to use the components from UIKit.

For this demo, I will create a new file called WebView.swift to implement a custom web view for SwiftUI. In the project navigator, right click the project folder and choose New File…. Select the Swift File template and name the file WebView.swift. Replace the file content like this:

import SwiftUI
import WebKit

struct WebView: UIViewRepresentable {

    var url: URL

    func makeUIView(context: Context) -> WKWebView {
        return WKWebView()
    }

    func updateUIView(_ webView: WKWebView, context: Context) {
        let request = URLRequest(url: url)
        webView.load(request)
    }
}

To use a UIKit view in SwiftUI, you wrap the view with the UIViewRepresentable protocol. Basically, you just need to create a struct in SwiftUI that adopts the protocol to create and manage a UIView object. In the code above, we create a WebView struct adopts the UIViewRepresentable protocol and implement the required methods.

In the makeUIView method, we return an instance of WKWebView. This is how you wrap a UIKit view and make the web view available to SwiftUI.

While the makeUIView method is responsible for creating and initializing the view object, the updateUIView method is responsible for updating the state of the UIKit view. So, we load the given URL in the updateUIView method.

Now it is ready to use WebView in our SwiftUI project. Switch over to ContentView.swift and add a state variable to store the current link:

@State private var showWebView = false

To bring up the web view, attach the .sheet modifier to the Button view:

Button {
    showWebView.toggle()
} label: {
    Text("AppCoda")
}
.sheet(isPresented: $showWebView) {
    WebView(url: URL(string: "https://www.appcoda.com")!)
}

We present the custom web view using a sheet. You can run the app on a simulator to have a test. When you tap the button, the app brings up a web view to load the website in a modal view.

By using the same technique, you can integrate SFSafariViewController in your SwiftUI projects. I will leave it to you as an exercise.

If you want to learn more about SwiftUI, you can check out our updated Swift course.

A launch screen is the very first screen presented to users when your app starts up. When you create a new project with UIKit, Xcode automatically generates a storyboard file named LaunchScreen.storyboard for developers to design the launch screen of the app. However, if you are developing an app with the SwiftUI framework, the launch screen file is no longer there. So, how can you prepare a launch screen in SwiftUI projects?

In Xcode 12, the tool provides a new way to implement a launch screen in SwiftUI. I will walk you through the procedures in this tutorial.

Editor’s note: If you want to learn SwiftUI, check out our Mastering SwiftUI book.

Preparing the Launch Image and Color Set

The figure below shows the launch screen we are going to build. It’s very simple screen that displays an image at the center of the view. The launch screen also adapts to different background color for Light and Dark mode.

To implement the launch screen above, you should first prepare the image and import it into the asset catalog.

Since the app supports both Light and Dark modes, we need to create a new color asset for the background color.

Name the color set LaunchScreenBackground and set the color code to the following:

• Any appearance – #EEEEEE
• Dark appearance – #111111

Adding a Launch Screen

In SwiftUI projects, the launch screen is not generated by default. You need to add it manually in the Info.plist file. After opening the file, you should see an entry named Launch Screen. Click the arrow on its left to change the direction. Then click the + button to add a new entry.

There are a number of options for you to choose from. To configure the background color, set the key to Background color and value to LaunchScreenBackground, which is the color set created in the previous section.

To include an image in the launch screen, click the + button to add another key. Set it to Image Name. For its value, set it to the name of the image (i.e. ramen) we imported.

If we want to ensure the image stays within the safe area, add the Image respects safe area insets and set its value to YES. The rest of the options are used for configuring the appearance of the navigation bar, tab bar, and toolbar. It’s up to your preference to enable them or not.

Once you complete the configuration, you can hit the Play button to run the app on a simulator. When the app is launched, it should display the launch screen. The background color adapts itself to the underlying interface style.

In case if your simulator couldn’t bring up the launch screen, go up to the simulator menu and click Device > Restart to clear its cache.

Using LaunchScreen.storyboard

Can you still use the old way to create a launch screen in SwiftUI projects? Yes, Xcode 12 still allows developers to do so. However, Xcode no longer generates the LaunchScreen.storyboard file for you. You have to add it manually by creating a new file using the Launch Screen template.

And then, you can set the launch screen option in the project configuration.

Importing photos and videos as media items is one of the most common features that the majority of iOS applications include. In general lines, there are two ways to do that; either by using a system-provided controller, or by implementing a custom picker manually. Obviously, going with the second approach is a much harder and complicated process.

In this post we are going to focus on the first option, and particularly on a brand new photo picker controller that was first introduced in iOS 14. That is the PHPicker API! Let’s take a look at a few interesting and useful details, but let’s do that starting from a different point.

Up until iOS 13, the available photo picker to use had been the UIImagePickerController; a familiar view controller that made its debut many years back. UIImagePickerController has been providing double functionality; the first is photo and videos importing to the app, and the other capture of new media using a camera from the iPhone or iPad.

This controller, however, carries along two disadvantages; the first is that it provides a relatively limited environment to pick media items from. Users can select one photo or video at each time only, while there’s no option to search for specific items when there are many of those stored in user’s library. The second downside is that developers are required to ask from users for their consent in order to present it, and therefore add new entries in the app’s Info.plist file with a description of the intended use.

PHPicker changes all that, and makes importing photos and videos an easier and faster task. First of all, its greatest feature is that it allows both single and multiple item selection, a feature that is definitely much appreciated by users. In addition, it offers a search feature in a picker similar to the Photos app, so users meet a familiar place; they already know how to move and what to do when such a picker is presented to them.

Also, and contrarily to the UIImagePickerController, PHPicker provides a better privacy and does not require to ask for user consent so it can be used. Even though an app can trigger its presentation, the picker runs in a separate process where users can select only type of media necessary for their tasks, and do nothing beyond that; not even to get a snapshot of the picker. So, with PHPicker, simply forget about usage descriptions and rely on the built-in iOS privacy.

To get the photos and videos that users have picked and handle them in our apps, PHPicker transfers them over temporarily and offers a few ways to fetch them. We are going to see all the related how-to in the upcoming parts of this post. However, before we continue, keep something important in mind; PHPicker is not a class, nor a framework. It’s just a name that hides classes and other types behind it, where all of them belong to the PhotosUI framework, also announced in WWDC 2020.

What PHPicker does not do is to offer capture capabilities. If an app wants to do so, then the UIImagePickerController is still the way to go. Also, apps running on iOS 13 and below cannot use PHPicker APIs; it’s meant for iOS 14 and above only.

That being said, in this tutorial we are going to meet PHPicker and all necessary actions that will let us import photos, videos and live photos to a demo application. We will have the chance to talk about some not so much known techniques, and we’ll see various aspects of PHPicker, such as how to configure it or how to handle picked items. In the mix we will also add pinches of SwiftUI, so get ready for a really interesting post!

An Overview Of The Demo App

In order to get a good, in-depth taste of PHPicker, we are going to be working on a simple demo application. It’s about a single-screen SwiftUI app, and there is a starter project to download that I’ve prepared for you.

Once you download and open it in Xcode, you’ll find a few files in the Project navigator:

• ItemsView.swift: The SwiftUI view that we’ll be working on.
• PhotoPicker.swift: This is where we’ll do the PHPicker related work.
• PhotoPickerModel.swift: A place where we’ll implement a custom model to keep data about selected media items.
• LivePhotoView.swift: It will become handy in order to preview live photos in the demo app.

I’m not going to provide any details about the contents of these files, as we are just about to see thoroughly next.

In the ItemsView there’s some initial code already implemented. Even though we won’t perform any navigation, there is a NavigationView added in order to create a navigation bar and therefore to have two bar buttons; one for adding new media items to the app (a plus icon), and one for deleting them (a trash icon).

The main content of the view is practically empty. There is an EmptyView that we are going to replace soon with a List view, and inside that List we are going to display the various media that we will be importing. There is also a sheet ready to be used; with it we’ll be presenting the photo picker.

Take your time to look around in the starter project, and once you’re done, then just keep reading. A quite interesting and detailed tutorial is just about to unfold.

Presenting The PHPicker View Controller

Let’s start working in order to present the PHPicker controller. We are going to show it modally, and for that job we are going to employ the sheet in the ItemsView; presenting content modally is what a sheet is meant for.

In the demo application doing so has already been taken care of:

.sheet(isPresented: $showSheet, content: {

})

In the ItemsView.swift file you will find the sheet view modifier right after the closing of the NavigationView. This is what presents the actual sheet, but the flag that controls whether the sheet should be presented or not is the showSheet property that has been marked with the @State property wrapper:

@State private var showSheet = false

The sheet modifier gets two arguments:

1. The binding value of the @State property that indicates whether sheet should be shown or not. That binding value is the $showSheet argument in code, and it means that content inside the sheet can change the value of showSheet, which belongs to ItemsView.
2. A closure where we’ll provide the sheet’s content; soon this will be the picker controller.

The PHPicker controller is a UIKit view controller, represented by the PHPickerViewController class. Since it’s not a native SwiftUI component, unavoidably we will mix SwiftUI and UIKit together.

In order to make a UIKit view controller available to SwiftUI and accessible in it, it’s necessary to create a new custom type, a structure, that will be conforming to a specific protocol called UIViewControllerRepresentable. That structure will actually be implementing and dealing with the UIKit view controller, and it will be the bridge between the controller and the SwiftUI view that is meant to be making use of it.

Let’s see that happening in action, but for keeping our code as clean and readable as possible, let’s work on a different file. That is the PhotoPicker.swift in the starter project, so just click on it to open and start editing it.

The first step is to replace the Foundation framework imported by default with the following two:

import SwiftUI
import PhotosUI

The SwiftUI framework is needed so we can use the UIViewControllerRepresentable protocol; the PhotosUI for accessing the PHPickerViewController class.

Next, let’s define a custom type, which is going to be a structure and we’ll call PhotoPicker. Don’t forget that it has to be conforming to the UIViewControllerRepresentable protocol:

struct PhotoPicker: UIViewControllerRepresentable {

}

Every UIViewControllerRepresentable type must be necessarily doing the following three things:

1. To specify the view controller type that is dealing with.
2. To implement a method where it creates, configures and returns the view controller instance.
3. To implement one more method that is used in order to perform updates to the view controller when those originate from the SwiftUI environment. Even though it’s mandatory to define it, this method can remain empty if there is no reason to update the view controller. And that’s what we’ll do here; we’ll just leave it empty.

Having said all that, let’s see how they are all implemented in the PhotoPicker structure:

struct PhotoPicker: UIViewControllerRepresentable {
    typealias UIViewControllerType = PHPickerViewController

    func makeUIViewController(context: Context) -> PHPickerViewController {

    }

    func updateUIViewController(_ uiViewController: PHPickerViewController, context: Context) {

    }
}

We’ll now focus on the makeUIViewController(context:) method, as that’s the place where we’ll initialize the picker view controller. At the initilization time of a PHPickerViewController object, we must provide the desired configuration that should apply to the picker when it will be presented. That configuration is given as a PHPickerConfiguration object, which is another PHPicker API.

Initially we’ll start simple, by allowing to pick only photos. Also, at first we’ll limit the number of allowed picked items to one (1), but that’s something we are going to change later. In order to do all that, at first we have to create a new PHPickerConfiguration object:

func makeUIViewController(context: Context) -> PHPickerViewController {
    var config = PHPickerConfiguration()
    config.filter = .images
    config.selectionLimit = 1
}

It’s also necessary to specify the value of one more property of the config object:

config.preferredAssetRepresentationMode = .current

Even though the documentation of that property is actually empty on Apple docs, the import process is going to take significantly too much time in order to complete without setting current to preferredAssetRepresentationMode. So, add the above along with the other two configurable properties.

With the configuration object ready, we can move on and initialize a PHPickerViewController instance, and eventually return it from the method:

func makeUIViewController(context: Context) -> PHPickerViewController {
    ...

    let controller = PHPickerViewController(configuration: config)

    // Set the delegate here...

    return controller
}

See that we are passing the config in the initializer of the controller object at the time of its creation. Also, right before the return statement, there is a comment I added there about setting the delegate. And here is why:

A user will have two choices once the picker controller has been presented; either to pick a photo to import, or to cancel and dismiss the picker. In any case, the selected action will be communicated back through a delegate method, which we obviously have to implement and handle the results. However, at this point we still don’t have the object that we will set as the picker’s delegate. And it cannot be the PhotoPicker instance; it has to be a class instance. In particular for UIViewControllerRepresentable types, a coordinator class is playing that role.

Implementing The Coordinator Class

A coordinator class is a custom type that has to be implemented inside a UIViewControllerRepresentable type, in our case inside the PhotoPicker structure, and it has a quite specific purpose; to act as the delegate for view controllers that send messages through delegate methods, and pass received data to SwiftUI. If we could phrase that in simpler words, just think that this is the way for a UIKit view controller to send messages to the SwiftUI view that is using it.

It all starts by defining the following class in the PhotoPicker body:

struct PhotoPicker: UIViewControllerRepresentable {
    ...

    class Coordinator {

    }
}

In the class body, we will declare the following property which will be keeping the PhotoPicker instance:

class Coordinator {
    var photoPicker: PhotoPicker
}

Along with that, we’ll also create a custom initializer method:

class Coordinator {
    var photoPicker: PhotoPicker

    init(with photoPicker: PhotoPicker) {
        self.photoPicker = photoPicker
    }
}

Now, and before we go any further, it’s the best time to create a Coordinator instance in the PhotoPicker structure by using an optional method of the UIViewControllerRepresentable protocol. That method will simply initialize and return a Coordinator object, passing the PhotoPicker instance (self) as an argument:

struct PhotoPicker: UIViewControllerRepresentable {
    ...

    func makeCoordinator() -> Coordinator {
        Coordinator(with: self)
    }

    class Coordinator { ... }
}

Back to the makeUIViewController(context:) method, we are now able to replace the following comment:

// Set the delegate here...

with this:

controller.delegate = context.coordinator

Notice that the Coordinator instance is accessible through the context parameter value of the method. After having added the above, here’s the entire method:

func makeUIViewController(context: Context) -> PHPickerViewController {
    var config = PHPickerConfiguration()
    config.filter = .images
    config.selectionLimit = 1
    config.preferredAssetRepresentationMode = .current

    let controller = PHPickerViewController(configuration: config)
    controller.delegate = context.coordinator
    return controller
}

The PHPicker Delegate Method

We can now pay our full attention to the implementation of the Coordinator class. At first, it’s necessary to make it conform to the PHPickerViewControllerDelegate protocol; the one that will provide us with the delegate method that has to be implemented:

class Coordinator: PHPickerViewControllerDelegate {
    ...
}

Even though two different kind of actions can take place in the picker view controller after it has been presented (pick photos or cancel), there is one delegate method to implement only. This is called both when user selects media items, and when just cancels and dismisses the picker. This is the following:

class Coordinator: PHPickerViewControllerDelegate {
    ...

    func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {

    }
}

See that the selected items are contained in the results parameter; it’s a collection of PHPickerResult objects, which we’ll deal with in a while.

At this point, there is a question rising; how are we supposed to let the SwiftUI view know that the picker has been dismissed, and whether the user selected items or just cancelled?

This question does not include the actual selected media items. Those will be handled asynchronously and we’ll provide them to the SwiftUI view in a different way which we’ll implement next. But to answer and come up to a solution, first we have to consider something else.

The time that the picker will be dismissed is undefined and depends totally on the user. We can’t predict it and the best approach in order to react to that is by defining and using an action handler. That action handler will be called in the delegate method, but it will be implemented as a closure in the SwiftUI view, provided to the PhotoPicker instance right at the time of the initialization. Then, each app will be able to trigger any necessary further actions once it has the information that the picker has been dismissed.

So, in the PhotoPicker structure, add the following declaration at the top:

struct PhotoPicker: UIViewControllerRepresentable {
    var didFinishPicking: (_ didSelectItems: Bool) -> Void

    ...
}

The argument that we’ll provide to the above callback will be indicating whether the user just dismissed the picker, or that happened after having selected media items.

Returning to the picker(_:didFinishPicking:) delegate method now, we can add the first line of code which will be calling the above callback, passing the proper value as argument:

func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {
    photoPicker.didFinishPicking(!results.isEmpty)
}

If the results array is not empty and the user has selected media items, then the argument above will be true. Otherwise, it’s going to be false. Notice also that we are accessing it through the photoPicker property of the Coordinator class; photoPicker has been declared for accessing PhotoPicker properties like what we just did above.

If the user simply dismissed the picker, then there’s nothing to do besides than just returning from the method. We will proceed if only the results array is not empty, but we’ll work on that in just a while. For now, let’s append the following to the delegate method:

func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {
    ...

    guard !results.isEmpty else {
        return
    }
}

We can now go back to the ItemsView.swift file, and create a PhotoPicker instance in the sheet’s content body:

.sheet(isPresented: $showSheet, content: {
    PhotoPicker() { didSelectItems in
        // Handle didSelectItems value here...
        showSheet = false
    }
})

Here we won’t deal with the value of the didSelectItems parameter of the closure, but see the comment I’ve put in the above snippet; that’s the place to check whether it’s true or false, meaning to determine if the user has selected items or not, and then act accordingly.

The only thing we’re doing here is to toggle the value of the showSheet property and set it to false. Doing so will cause the dismissal of the sheet, and eventually of the photo picker.

You can give the app a first try at this point, however you’ll get the same result no matter if you select a photo or just cancel. The picker will simply get dismissed.

Implementing A Model To Store Picked Items Data

We managed to present the PHPicker view controller without much effort, but it’s practically not useful at all right now. We haven’t done any results handling yet, but even if we proceed to that now, where exactly are we going to store the fetched photos?

We have to come up with a solution to that, and that’s why in this part we’ll focus on the implementation of a basic data model. It will let us keep photos, videos and live photos once we have extracted them from the results in the picker delegate method.

So, to work our way towards that, we’ll open the PhotoPickerModel.swift file to add some new code there.

As before, the first thing we’ll do in this file is to replace the Foundation framework with the next two:

import SwiftUI
import Photos

After that, let’s define the following structure:

struct PhotoPickerModel {

}

Inside the PhotoPickerModel we will define another custom type, an enumeration with cases that will be indicating the type of the media that an instance of this structure is holding:

struct PhotoPickerModel {
    enum MediaType {
        case photo, video, livePhoto
    }
}

One of the tasks that we’ll perform later, will be to present a collection of PhotoPickerModel items in a SwiftUI List view. A List, as well as some other views, need each item in the collection to be uniquely identified by some value if possible. We can avoid do that and ask SwiftUI to create unique values automatically. Indeed, it will do that by calculating the hash value of each item, but it’s always a better option to provide our own.

In fact, that can be done really easily, as long as we make the PhotoPickerModel conform to a specific protocol called Identifiable. That protocol has one requirement only; the custom type that adopts it to have a property called id. It can be of any type we want, but we must ensure that it’s going to have a unique value for each instance of the PhotoPickerModel.

There are various ways to generate unique values, but in order to keep things simple in this tutorial we are going to use UUID strings as id values. According to Apple docs, a UUID is:

A universally unique value that can be used to identify types, interfaces, and other items.

So, with all that in mind, let’s update the PhotoPickerModel as shown next:

struct PhotoPickerModel: Identifiable {
    ...

    var id: String
}

Now we can continue on declaring properties that will be storing the various media items we can get from the photo picker.

Let’s start with normal photos. In that case we are talking about images, so we need to declare a UIImage property:

var photo: UIImage?

Things change when it’s about videos. As we will see in details next, a video cannot be fetched as a single object. When dealing with videos, we’ll unavoidably be dealing with files, so what we actually need to store here is the URL to the file containing a picked video:

var url: URL?

Apart from the above two media types, PHPicker also allows to import live photos. A live photo can be taken as a single object, just like normal images can, but there is a specific data type to use here; the PHLivePhoto:

var livePhoto: PHLivePhoto?

Finally, there’s one last property that has to be declared. That is the type of the media:

var mediaType: MediaType = .photo

Providing a default initial value (photo here) helps avoid to declare the above as an optional, and therefore make some future tasks easier.

That’s all we need, so here’s how the PhotoPickerModel has become after having added all the above:

struct PhotoPickerModel: Identifiable {
    enum MediaType {
        case photo, video, livePhoto
    }

    var id: String
    var photo: UIImage?
    var url: URL?
    var livePhoto: PHLivePhoto?
    var mediaType: MediaType = .photo
}

See that the photo, url and livePhoto have been declared as optional properties, meaning that their default value is nil. That makes sense, if you think that when a PhotoPickerModel item stores the data of a specific media type, just one property will get an actual value; there will be no values to assign to the other two properties.

In addition to all the above, we will also implement a few custom initializer methods in order to provide a convenient way to store each media type. Let’s start with the case of photos:

init(with photo: UIImage) {
    id = UUID().uuidString
    self.photo = photo
    mediaType = .photo
}

Three distinct things take place here:

1. We create the unique identifier for the current item.
2. We keep the photo to the photo property.
3. We provide the proper MediaType value to the mediaType property. In this case, doing that might looks redundant as photo is the default value for mediaType, however it makes no harm to be specific.

The implementation of the initializers for the other two media types is going to be done in an exact similar fashion, so let’s see them together:

init(with videoURL: URL) {
    id = UUID().uuidString
    url = videoURL
    mediaType = .video
}

init(with livePhoto: PHLivePhoto) {
    id = UUID().uuidString
    self.livePhoto = livePhoto
    mediaType = .livePhoto
}

In both of them we assign the parameter object to the matching property, and we set the correct media type value. Of course, we create a unique id value as well.

For now the PhotoPickerModel is finished, but we’ll come back to it one more time in order to add a handy method we’ll need later.

The PickedMediaItems Class

Even though we just implemented the PhotoPickerModel structure, we can’t just use it as is. We need one more custom type that will actually be keeping an array of such objects. Besides from just being the place where we’ll be storing media items once they have been extracted from the picker results, it will also be the datasource for the List view we’ll show in the ItemsView.

While still being in the PhotoPickerModel.swift file, go after the closing curly bracket of the PhotoPickerModel structure and add the next class:

class PickedMediaItems: ObservableObject {

}

It conforms to ObservableObject protocol for one reason; to make it possible to inform SwiftUI views about changes happening to the array we are declaring right next simply by annotating it with the @Published property wrapper.

class PickedMediaItems: ObservableObject {
    @Published var items = [PhotoPickerModel]()
}

items array is a collection of PhotoPickerModel objects. When items will be added to that array, or removed from it, the ItemsView will be notified about that right before the change actually takes place. As a result, it will re-render the entire view, and the displayed content will be in harmony with the contents of the items collection.

Of course, marking items with the @Published property wrapper is not enough for what I just described to happen; there’s a counterpart action we have to take in the ItemsView as well, but before we get to that, let’s add the following method to the PickedMediaItems class just for our own convenience:

class PickedMediaItems: ObservableObject {
    ...

    func append(item: PhotoPickerModel) {
        items.append(item)
    }
}

The only thing the above method does, is to append the given argument to the items array. As said, it’s here just to make storing new items faster. Later we’ll make one more addition to the PickedMediaItems class, but for now we are good to go.

Let’s go to the ItemsView.swift file now, and at the beginning of the ItemsView structure, right where the declaration of the showSheet @State property is, add the following:

struct ItemsView: View {
    @ObservedObject var mediaItems = PickedMediaItems()

    ...
}

mediaItems property is an instance of the PickedMediaItems class. By marking it with the @ObservedObject property wrapper, we enable the view to observe for and react to changes that happen to the items array that was annotated as @Published in the PickedMediaItems class. Now, whenever a change occur to that array, view will be re-rendered in order to show the proper content.

Notice that ItemsView structure is the place where we declare and at the same time initialize the mediaItems object. Even though we need it here because it will act as the datasource for a List view that we’ll add next, there’s also another type where it’s necessary as well; the PhotoPicker structure!

Open the PhotoPicker.swift file, and at the beginning of the PhotoPicker structure declare the next property:

struct PhotoPicker: UIViewControllerRepresentable {
    @ObservedObject var mediaItems: PickedMediaItems

    ...
}

Here we declare the exact same property (we could have used a different name if we wanted to) marked with the @ObservedObject property wrapper too, but there’s a difference; we do not initialize it!

Instead, we will pass the mediaItems object we created in the ItemsView right upon the initialization of the PhotoPicker object. To do that, go back to the ItemsView.swift and update it according to the next snippet, inside the sheet’s content closure:

.sheet(isPresented: $showSheet, content: {
    PhotoPicker(mediaItems: mediaItems) { didSelectItem in
        // Handle didSelectItems value here...
        showSheet = false
    }
})

From now on, whenever a media item will be added to the items array of the mediaItems property, ItemsView will be notified about that and it will redraw its contents in order to display the new item. Of course, we have to make a few additions to the ItemsView in order for that to happen, but we’ll get to that soon.

Getting A Photo From Picker Results

In the picker(_:didFinishPicking:) delegate method of the PHPickerViewControllerDelegate we saw that selected items in the picker are represented as a collection of PHPickerResult objects. Each one of them contains a property called itemProvider, which is an instance of the NSItemProvider class. Citing the definition from Apple docs:

NSItemProvider: An item provider for conveying data or a file between processes during drag and drop or copy/paste activities, or from a host app to an app extension.

In the introduction of this post I said that PHPicker runs on a different process out of the app. When a user selects an item to import, that item is transferred from that process to our own app wrapped into an item provider object.

Having that in mind, let’s go to the PhotoPicker.swift file and in the Coordinator class, where we are going to implement the following method:

private func getPhoto(from itemProvider: NSItemProvider) {

}

This one will be extracting the photo from the given item provider, and on success, it will be storing it to a new PhotoPickerModel object, which in turn will be appended to the items array of the PhotoPicker’s mediaItems property. Notice that we declare it as private; it’s meaningful for the Coordinator class only, and there is no reason to make it visible out of it.

Since we want to get a photo, the first step in the method is to check if we can actually load an image object from the item provider. This is done as follows:

if itemProvider.canLoadObject(ofClass: UIImage.self) {

}

The above method returns true or false, depending on whether the item provider can load an image object. Notice the UIImage.self argument; with that we tell to the method that we are interested in the UIImage type.

Supposing that the above returns true, our next move is to load the image from the item provider. This is possible in a similar way as above, but using a different method this time:

itemProvider.loadObject(ofClass: UIImage.self) { object, error in

}

This method works asynchronously, and on completion it will bring back either the loaded object (a UIImage object in this case), or an error if occurred, as parameter values in the completion handler.

Before doing anything else, we have to check if the error is nil or not. In this tutorial we won’t do any specific error handling, we’ll simply print the error description. But in a real app, you’d probably want to perform a more appropriate action in such case.

if let error = error {
    print(error.localizedDescription)
}

If there’s not an error, then most probably the object contains the image we are looking for. But in order to get it, we have to cast to the UIImage type; object is a NSItemProviderReading value:

if let image = object as? UIImage {

}

If casting and unwrapping at the same time the optional object to the image constant succeeds, then we can eventually store the loaded image to a new PhotoPickerModel instance. This is what we’ll append to the items array of the PhotoPicker’s mediaItem property afterwards. Note that this is a task that has to take place on the main thread of the app! We are just about to trigger changes to a @Published property (the items array), and that should always happen to the main thread.

DispatchQueue.main.async {
    self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: image))
}

The new PhotoPickerModel is created with the PhotoPickerModel(with: image) part using the first custom initializer we implemented, and it’s given as an argument to the append(item:) method. This one appends it to the items array of the mediaItems property.

Our method is ready, so let’s see how it looks complete:

private func getPhoto(from itemProvider: NSItemProvider) {
    if itemProvider.canLoadObject(ofClass: UIImage.self) {
        itemProvider.loadObject(ofClass: UIImage.self) { object, error in
            if let error = error {
                print(error.localizedDescription)
            }

            if let image = object as? UIImage {
                DispatchQueue.main.async {
                    self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: image))
                }
            }
        }
    }
}

However, we are not done yet; we still don’t make any use of the above method. The place to call it is the picker(_:didFinishPicking:) delegate method in the Coordinator class. Before adding a few bits of new code there, remember that we configured our picker so it’s possible to select one photo only. That means that in the picker(_:didFinishPicking:) we’ll take the item provider of the first result, and then we’ll call the getPhoto(from:) providing the item provider as argument.

Here’s how the picker(_:didFinishPicking:) must be updated with what I just said:

func picker(_ picker: PHPickerViewController, didFinishPicking results: [PHPickerResult]) {
    ...

    let itemProvider = results[0].itemProvider
    self.getPhoto(from: itemProvider)
}

Displaying Photos

With the PhotoPicker type capable of fetching selected photos, let’s head to the ItemsView.swift file. Here we’ll do the necessary preparations, so photos to be displayed after they have been selected in the picker.

I mentioned a few times up until now that we are going to use a List view in order to show multiple selected media items. That List does not exist yet, and that’s what we’ll do as a first step here; to create it.

Right now in the ItemsView body you will find an empty view (EmptyView), which acts as a placeholder until we have something to show. That time has come, so replace the following line:

EmptyView()

with the following List initialization:

List(mediaItems.items, id: \.id) { item in

}

The first argument in the List is a collection of items that will be used as the datasource by the views that we’ll add to the List. The second argument is the keypath to the id value of each item in the collection (\.id). That is the id property declared in the PhotoPickerModel structure. The third argument is the closure where we’ll add all necessary views in order to display photos, videos, and live photos. The List works by making iterations, and the item value in the closure represents the currently accessed item object in the items collection.

Since we want to display photos, we are going to use an Image view for that purpose. We’ll put that inside the List’s closure:

List(mediaItems.items, id: \.id) { item in
    Image(uiImage: item.photo ?? UIImage())
}

Two things to notice here; first, we are initializing an Image view passing a UIImage as an argument. That’s how we keep photos stored in the PhotoPickerModel. Second, in case the photo property of the item is nil, then we just provide an empty UIImage object.

Even though the above will show the photo that’s saved in the item, it won’t do that properly; if you would run the app now, you would have seen that the photo does not fit the Image view size. To change that, we’ll set a specific aspect ratio value with the aspectRatio(contentMode:) view modifier. However, in order for that to have a real effect, it’s necessary to use prior to it the resizable() modifier, so the Image view can resize its contents:

List(mediaItems.items, id: \.id) { item in
    Image(uiImage: item.photo ?? UIImage())
        .resizable()
        .aspectRatio(contentMode: .fit)
}

The snippet above is the only modification we had to do in ItemsView. All the rest will happen automatically, as the view will redraw its contents once a photo has been stored to the items collection of the PickedMediaItems instance.

Run the app again now, and tap on the plus button to select and import a photo. You can do that multiple times; all of them will be displayed in the List view we just defined.

Picking Multiple Photos

Now that we have managed to pick and present a single photo, let’s continue to changing that and let’s allow multiple photo picking. The place to start working for that is the PhotoPicker type, so open the PhotoPicker.swift file to edit it.

In the makeUIViewController(context:) method there is a line where we set the maximum selection limit:

config.selectionLimit = 1

To allow picking unlimited items, set the selectionLimit to zero:

config.selectionLimit = 0

You can also set a specific number if you want to have a limit higher than 1 but not unlimited.

Next stop is the picker(_:didFinishPicking:) delegate method inside the same file. Currently we are accessing the item provider of the first result only, and then we provide it to the getPhoto(from:) method.

Having changed the number of items items that can be selected, that implementation will not give us back all selected photos. To change that, first spot the following two lines:

let itemProvider = results[0].itemProvider
self.getPhoto(from: itemProvider)

Once you find them, move them inside a loop as shown here:

for result in results {
    let itemProvider = results[0].itemProvider
    self.getPhoto(from: itemProvider)
}

That for loop will go through all returned results, and process all photos. There is one last change to do; replace the first line in the loop that contains the results[0] with this:

let itemProvider = result.itemProvider

By running the app now you will find out that it’s possible to select and import multiple photos at once.

Importing Videos

The solution we implemented in the getPhoto(from:) method inside the Coordinator class of the PhotoPicker type works for photos, and with slight modifications is going to work for live photos too. But when it comes to videos, the approach we must follow is totally different.

Unlike photos, there is not a single object we can load from the item provider; and that makes sense if you think about it. It’s not possible to load hundreds of megabytes of data to memory just for a single video. Videos have to be loaded from files.

And that’s exactly what we’ll do; we are going to deal with files.

Thankfully, NSItemProvider class has more APIs available to use, and one of them allows to get the data of a selected item as a file. However, there is something important to know here; the item provider creates a temporary file for every video we want to get to the “tmp” folder of the app, and then deletes it automatically after a while. It’s our duty to copy that file to a location where it can live for a longer time, such as the documents or the caches directory of the app.

So, here we are going to implement another method in the Coordinator class, which will be responsible for getting a video from an item provider, and copying it to the documents directory of our application. Let’s define it:

private func getVideo(from itemProvider: NSItemProvider, typeIdentifier: String) {

}

The first argument is the item provider object which will provide us with the video file. The second argument is something that we meet for first time in this tutorial, but meant to play an important role here. It’s the uniform type identifier (UTI) of the file we want to fetch.

Note: In a nutshell, a UTI is a string value with a special syntax that allows to uniquely identify the kind of data in files, data types, etc, and it was defined by Apple. I would recommend to read here and here for more information from the official documentation.

To get an idea, the string “public.png” is the type identifier for PNG images. A type identifier can inherit from other types as well; for example, “public.png” inherits from “public.image”. Keep that information in mind, we’ll need it soon.

How we’ll get the type identifier that we need to provide in this new method is something that we will bother with later. Now, let’s start adding some code to it.

The first and quite important move is to actually request from the item provider to give us the file representation of the selected video. This is done as shown right below, and see that we provide the type identifier as an argument:

itemProvider.loadFileRepresentation(forTypeIdentifier: typeIdentifier) { url, error in

}

The method is executed asynchronously, and in its completion handler we are going to deal with the video file. The url parameter is pointing to the temporary file that the item provider created for us. Of course, chances for something to go wrong always exist, so the error object is there to let us know what error occurred, if any.

Just like we did in the getPhoto(from:) method, we’ll continue by examining whether error is nil or not. If it’s not, we’ll simply print the error message, but do something more useful than that in your apps:

if let error = error {
    print(error.localizedDescription)
}

In case where error is nil and we can proceed, then the first we’ll do is to unwrap the url object; it’s an optional value, and we must be sure that it’s not nil:

guard let url = url else { return }

If for some reason it’s nil, and at the same time the error is nil, then there’s nothing else we can do. We simply return from the method.

Supposing that url is unwrapped properly, then our next task is to copy the video file from the temporary URL to the documents directory of the app. Note that this the directory of my choice for the purposes of the tutorial, you can use caches directory instead.

Before we perform the actual copy operation, it’s necessary to construct the target URL. This will be composed by two parts:

1. The URL to the documents directory.
2. The last path component of the source directory, meaning the file name and any potential extension it might have.

Let’s get the URL to the documents directory first:

let documentsDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first

Getting the last path component from the source URL is as easy as that (don’t copy the following, it’s just a demo):

url.lastPathComponent

To add it to the documents directory URL, we’ll use the appendingPathComponent() method of the URL class, and eventually we’ll build the target URL:

guard let targetURL = documentsDirectory?.appendingPathComponent(url.lastPathComponent) else { return }

If the target URL has been constructed properly, and once we make sure that it’s not nil by unwrapping it, then we can do the actual copy. First though, we’ll check if a file with the similar path already exists or not to the target URL, and if so we’ll delete it and then we’ll copy.

Deleting and copying files are operations that can throw exceptions, and the proper way to treat them is by including them in a do-catch statement:

do {

} catch {
    print(error.localizedDescription)
}

Once again, if something bad happens, we just print the error description.

Inside the do clause now, let’s do the real work. We’ll use the FileManager class to perform all file-related tasks. We’ll start by checking if a similarly named file exists, and we’ll delete it, if it does:

if FileManager.default.fileExists(atPath: targetURL.path) {
    try FileManager.default.removeItem(at: targetURL)
}

fileExists(atPath:) and removeItem(at:) are the FileManager methods that do the job. Notice that FileManager is a Singleton class, and we access everything through the default shared object. Also, two more noteworthy things:

1. removeItem(at:) method can throw an exception, so it’s marked with the try keyword.
2. The parameter of the fileExists(atPath:) should be a string value, but the parameter of the removeItem(at:) should be a URL object. That’s why in the first case we access the path property of the target URL; it returns it as a string value.

Next, we can perform the actual copy:

try FileManager.default.copyItem(at: url, to: targetURL)

We mark this method with the try keyword too, as if the file copy fails it will throw an exception. The first argument we provide it with is the source URL, and the second is the destination URL.

If things go well and we get no exception up to this point, then we can go ahead and create a new PhotoPickerModel object in order to store the target URL; the place where we are going to find the selected video in order to display it in the app. Rememeber that we do that on the main thread of the app:

DispatchQueue.main.async {
    self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: targetURL))
}

The method is now complete:

private func getVideo(from itemProvider: NSItemProvider, typeIdentifier: String) {
    itemProvider.loadFileRepresentation(forTypeIdentifier: typeIdentifier) { url, error in
        if let error = error {
            print(error.localizedDescription)
        }

        guard let url = url else { return }

        let documentsDirectory = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first
        guard let targetURL = documentsDirectory?.appendingPathComponent(url.lastPathComponent) else { return }

        do {
            if FileManager.default.fileExists(atPath: targetURL.path) {
                try FileManager.default.removeItem(at: targetURL)
            }

            try FileManager.default.copyItem(at: url, to: targetURL)

            DispatchQueue.main.async {
                self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: targetURL))
            }
        } catch {
            print(error.localizedDescription)
        }
    }
}

Updating The Picker Delegate Method

As we did with the getPhoto(from:) method, we are going to call getVideo(from:typeIdentifier:) in the picker’s delegate method too. But in order to do that, we must come up with a solution about two things:

• How to get the type identifier necessary for the method we just implemented.
• How to distinguish the media type so we know which method to call; the one for getting the photo, or the one for getting the video.

In fact, taking care of the first will help answer the second bullet above as well! We will be based on the type identifier of the data that each item provider carries over; depending on the parent type that conforms to, we’ll decide whether it’s a photo or a video. Photo-related types conform to “public.image” type, and video-related types conform to “public.movie” type.

But first things first. To get the type identifier of the data that an item provider contains, we have to access a property called registeredTypeIdentifiers. Its value is an array of strings representing all types that data relates to. The first object of that array is what we actually need:

let typeIdentifier = itemProvider.registeredTypeIdentifiers.first

To manage to determine what parent type the identifier conforms to, it’s necessary to create an object of the UTType type, also new in iOS 14:

let utType = UTType(typeIdentifier)

Let’s combine the above two in one single guard statement, as both of them can give us a nil value:

guard let typeIdentifier = itemProvider.registeredTypeIdentifiers.first,
      let utType = UTType(typeIdentifier)
else { continue }

The above must go inside the for loop in the picker(_:didFinishPicking:) method:

for result in results {
    let itemProvider = result.itemProvider

    guard let typeIdentifier = itemProvider.registeredTypeIdentifiers.first,
          let utType = UTType(typeIdentifier)
    else { continue }

}

Note that the call to the getPhoto(from:) method does not exist now in the above snippet. That’s because we have to determine first if data on each item provider is an image or a video.

To achieve that, we will use a method of the UTType type, called conforms(to:), in an if statement (still inside the for loop) as shown next:

if utType.conforms(to: .image) {

} else if utType.conforms(to: .movie) {

}

The argument provided to the conforms(to:) is a UTType value indicating the parent type that we want to know if our type inherits from.

Now, depending on whether we are talking about an image or a video, we can call the appropriate custom method:

if utType.conforms(to: .image) {
    self.getPhoto(from: itemProvider)
} else if utType.conforms(to: .movie) {
    self.getVideo(from: itemProvider, typeIdentifier: typeIdentifier)
}

Before we switch to the ItemsView in order to make it possible to playback videos, let’s go to the makeUIViewController(context:) method of the PhotoPicker structure in order to allow picking videos as well. Find the next line:

config.filter = .images

Now, replace it with this:

config.filter = .any(of: [.images, .videos])

The any(of:) method is a PHPickerFilter API that let us provide an array of PHPickerFilter values matching to media items we want to make available in the picker; exactly what is shown above.

Playing Back Videos

At the moment, the ItemsView is capable of presenting only photos. But now that we are able to import videos as well, that’s something we are going to change. To determine what kind of media should be displayed, we are going to be based on the mediaType property of each PhotoPickerModel item in the List view.

To get started with the view updates, first open the ItemsView.swift file. Then, inside the List view add the following (you can remove for a while the Image view with its modifiers):

List(mediaItems.items, id: \.id) { item in
    if item.mediaType == .photo {

    } else if item.mediaType == .video {

    }
}

The conditional display of views is the first reason for the existence of the mediaType property. There is one more, but we’ll talk about that in a while.

Now, in the first case where the media type indicates a photo, we can put back the Image view and its modifiers:

List(mediaItems.items, id: \.id) { item in
    if item.mediaType == .photo {
        Image(uiImage: item.photo ?? UIImage())
            .resizable()
            .aspectRatio(contentMode: .fit)
    } else if item.mediaType == .video {

    }
}

In the else if clause where the media type regards a video, we are going to add a view that is capable of playing back videos. As you will see right next, video playback is an extremely simple task in iOS 14.

But first, we need to import another framework to the view; go to the beginning of the file, and right after the SwiftUI import statement add the following:

import AVKit

The AVKit framework will allow us to access the VideoPlayer structure; a SwiftUI view that allows to playback videos.

The argument we have to provide with a VideoPlayer view upon initialization is an AVPlayer instance:

An object that provides the interface to control the player’s transport behavior.

An AVPlayer object in turn is initialized with a URL pointing to the location of a video file; exactly what we keep stored in our model for videos!

So, after all that, let’s see how everything is done in code. At first we’ll make sure that the URL to a video file is not nil; in case it is, we’ll add an EmptyView:

if let url = item.url {

} else { EmptyView() }

In the if let clause, we’ll create a new VideoPlayer view, passing an AVPlayer object as argument:

VideoPlayer(player: AVPlayer(url: url))

Lastly, to avoid strange effects when media will be displayed in the List, let’s set a minimum height to the video player:

VideoPlayer(player: AVPlayer(url: url))
    .frame(minHeight: 200)

Here’s the List view now:

List(mediaItems.items, id: \.id) { item in
    if item.mediaType == .photo {
        Image(uiImage: item.photo ?? UIImage())
            .resizable()
            .aspectRatio(contentMode: .fit)
    } else if item.mediaType == .video {
        if let url = item.url {
            VideoPlayer(player: AVPlayer(url: url))
                .frame(minHeight: 200)
        } else { EmptyView() }
    }
}

Now we can go ahead and pick both photos and videos. When you add a video, you’ll see that you can start and stop its playback using default controls provided by iOS.

Importing Live Photos

Having dealt with photos and videos, it’s about time to discuss about how to import live photos to the app using the PHPicker view controller. As you will see in this part, there are many similarities to handling photos, and that will allow us to use code we have already implemented.

To get started, open the PhotoPicker.swift file and go to the getPhoto(from:) method in the Coordinator class. Unlike videos but like photos, live photos can be loaded as objects from an item providers. Therefore, the loadObject(ofClass:) method can be used in this case too, with the only difference being the type of object we want to load.

In fact, everything we’ve done in getPhoto(from:) method will be kept almost as is. What needs to be added as a first step, is a way to determine whether the object that is about to be loaded should be a normal photo or a live photo. And to do that, we’ll start with the method’s signature. We will add a new parameter value, a Boolean value, which will be indicating whether the item provider contains a live photo or not. Update the method as follows:

private func getPhoto(from itemProvider: NSItemProvider, isLivePhoto: Bool) {
    ...
}

When isLivePhoto is false, then the UIImage type is what we will provide to both canLoadObject(ofClass:) and loadObject(ofClass:) methods. But when it’s true, then we have to specify a different type. For live photos, that type is PHLivePhoto.

Based on that thinking, let’s add the following statement as the first line in the getPhoto(from:isLivePhoto:) method. It declares a constant called objectType as a NSItemProviderReading type; the data type that both item provider methods expect. In case of a normal photo, it gets the type of the UIImage (UIImage.self), and in case of a live photo, it gets the type of the PHLivePhoto (PHLivePhoto.self):

private func getPhoto(from itemProvider: NSItemProvider, isLivePhoto: Bool) {
    let objectType: NSItemProviderReading.Type = !isLivePhoto ? UIImage.self : PHLivePhoto.self

    ...
}

Now we can update the arguments we provide to both canLoadObject(ofClass:) and loadObject(ofClass:) methods:

if itemProvider.canLoadObject(ofClass: objectType) {
    itemProvider.loadObject(ofClass: objectType) { object, error in
        ...
    }
}    

A few lines below, we have the code that converts the loaded object to a UIImage and then stores it to a new PhotoPickerModel object. We have to move that part of code inside an if-else condition which will be checking the value of the isLivePhoto parameter value:

if !isLivePhoto {
    if let image = object as? UIImage {
        DispatchQueue.main.async {
            self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: image))
        }
    }
} else {

}

In the else clause we’ll do exactly what we do in the case of the image object; except that this time we’ll cast to a PHLivePhoto object:

if let livePhoto = object as? PHLivePhoto {
    DispatchQueue.main.async {
        self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: livePhoto))
    }
}

These are the necessary changes to this method, and here’s how it finally looks:

private func getPhoto(from itemProvider: NSItemProvider, isLivePhoto: Bool) {
    let objectType: NSItemProviderReading.Type = !isLivePhoto ? UIImage.self : PHLivePhoto.self

    if itemProvider.canLoadObject(ofClass: objectType) {
        itemProvider.loadObject(ofClass: objectType) { object, error in
            if let error = error {
                print(error.localizedDescription)
            }

            if !isLivePhoto {
                if let image = object as? UIImage {
                    DispatchQueue.main.async {
                        self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: image))
                    }
                }
            } else {
                if let livePhoto = object as? PHLivePhoto {
                    DispatchQueue.main.async {
                        self.photoPicker.mediaItems.append(item: PhotoPickerModel(with: livePhoto))
                    }
                }
            }

        }
    }
}

Now, let’s update the picker(_:didFinishPicking:) delegate method for one last time. From a previous part, we have a condition there that checks the parent type of the type identifier taken from the item provider:

if utType.conforms(to: .image) {
    self.getPhoto(from: itemProvider)
} else if utType.conforms(to: .movie) {
    self.getVideo(from: itemProvider, typeIdentifier: typeIdentifier)
}

We are going to append an else case to the condition, which will match to live photos. In it we’ll call the getPhoto(from:isLivePhoto:) method, passing true as the second argument:

if utType.conforms(to: .image) {
    ...
} else if utType.conforms(to: .movie) {
    ...
} else {
    self.getPhoto(from: itemProvider, isLivePhoto: true)
}

Then, we are going to fix the call to getPhoto(from:) method in the if clause. That method now expects for one more parameter value, which in that case will be false, indicating a normal photo:

if utType.conforms(to: .image) {
    self.getPhoto(from: itemProvider, isLivePhoto: false)
} else if utType.conforms(to: .movie) {
    self.getVideo(from: itemProvider, typeIdentifier: typeIdentifier)
} else {
    self.getPhoto(from: itemProvider, isLivePhoto: true)
}

Changes have finished in that method too. Lastly, let’s pay one more visit to the makeUIViewController(context:) method of the PhotoPicker structure, and allow to select live photos in the picker along with the other media types. Replace the following line:

config.filter = .any(of: [.images, .videos])

with this one:

config.filter = .any(of: [.images, .videos, .livePhotos])

And with that, our work in the PhotoPicker.swift file is now finished.

Previewing Live Photos

A live photo is a special category of media, and in order to preview such a photo properly, PhotosUI framework provides a specific view for that; it’s called PHLivePhotoView. That’s the good news, because there’s bad news as well when working in SwiftUI. PHLivePhotoView is a UIView, meaning a UIKit view, and it cannot be used natively in SwiftUI.

To workaround this issue, it’s necessary to create a new custom type, a structure, that will be conforming to the UIViewRepresentable protocol. In analogy to UIViewControllerRepresentable and view controllers, UIViewRepresentable allows to bring UIKit views to SwiftUI.

The steps required to be followed in order to implement a UIViewRepresentable type are quite similar to those we already met in the PhotoPicker structure. We’ll go through them by implementing a new type which we’ll call LivePhotoView.

We’ll do that in a separate file, the one that remains in the starter project and we haven’t added code to yet; The LivePhotoView.swift, so open to edit it.

Start by replacing the Foundation framework with the usual two we added in other files too:

import SwiftUI
import PhotosUI

Now, let’s define our new type, the LivePhotoView, which will be conforming to the UIViewRepresentable protocol:

struct LivePhotoView: UIViewRepresentable {

}

UIViewRepresentable has three requirements, similar to the UIViewControllerRepresentable:

• To specify the type of view that we are importing to SwiftUI.
• To implement a method where we’ll initialize, configure and return the view.
• To implement one more method for updating the view.

Let’s add all those at once and specify the PHLivePhotoView as the one that our custom type will be handling:

struct LivePhotoView: UIViewRepresentable {
    typealias UIViewType = PHLivePhotoView

    func makeUIView(context: Context) -> PHLivePhotoView {

    }

    func updateUIView(_ uiView: PHLivePhotoView, context: Context) {

    }
}

The updateUIView(_:context:) method is required to exist, but adding code to it is optional. We are going to leave it empty, as there’s nothing to update.

Before initializing a brand new PHLivePhotoView, let’s declare the following property to LivePhotoView structure:

struct LivePhotoView: UIViewRepresentable {
    var livePhoto: PHLivePhoto

    ...
}

This is going to be the actual live photo that we will be previewing.

Initializing a PHLivePhotoView instance hides no surprises; at first we create the instance, then we assign the live photo object, and finally we return it from the makeUIView(context:) method:

func makeUIView(context: Context) -> PHLivePhotoView {
    let livePhotoView = PHLivePhotoView()
    livePhotoView.livePhoto = livePhoto
    return livePhotoView
}

It’s possible in a PHLivePhotoView to playback a part of the live photo’s video, as a hint that this is a live and not a normal photo. If you want to see that happening, add the following line right before returning from the method above:

livePhotoView.startPlayback(with: .hint)

There is also a delegate protocol called PHLivePhotoViewDelegate, and provides two delegate methods:

livePhotoView(_:willBeginPlaybackWith:)

livePhotoView(_:didEndPlaybackWith:)

You can use them to be notified when the playback of a live photo is about to start, or it’s finished respectively. We don’t need them, so we won’t implement them here. But be advised; in order to do that, you have to add a Coordinator class in LivePhotoView, exactly as we did in the PhotoPicker structure. Also, the coordinator will have to inherit from the NSObject class and adopt the PHLivePhotoViewDelegate protocol.

With the LivePhotoView type implemented at this point, let’s head to the ItemsView.swift file to make use of it. Inside the List view we have a conditional presentation of views; we display an Image view for normal photos, and a VideoPlayer view for videos.

Now we are going to add an else clause to that condition, where we’ll create a LivePhotoView instance. There, we’ll pass the fetched live photo as argument, but after we’ve made sure that it’s not nil. If it is, once again we’ll have an EmptyView:

List(mediaItems.items, id: \.id) { item in
    if item.mediaType == .photo {
        ...
    } else if item.mediaType == .video {
        ...
    } else {
        if let livePhoto = item.livePhoto {
            LivePhotoView(livePhoto: livePhoto)
                .frame(minHeight: 200)
        } else { EmptyView() }
    }
}

See that once again we set a minimum height for the live photo view, like we did for the VideoPlayer view.

To test importing live photos, you will have to run the app in a real device and pick one or more of those from the library. If you have enabled the hint playback with the method I demonstrated previously, then you’ll see the live photo playing back automatically. Regardless, just long press on any imported live photo and you’ll preview it just like you do in Photos app!

Deleting Media Items

Before we reach to the end of our work in this post, it would be a good idea to make it possible to delete media items that have been imported to the app. By doing that, we’ll enable the trash button that already exists in the starter project and it’s not usable up until now. Also, this will make the demo app more complete.

We’ll manage to delete items by performing three distinct steps. The first one is going to take place in the PhotoPickerModel.swift file, in the PhotoPickerModel structure. It’s necessary to add a new method which:

• Will be setting nil to photo and livePhoto properties.
• Will be deleting the video file using the stored URL, and then making the URL nil as well.

All that will happen conditionally in a switch statement:

mutating func delete() {
    switch mediaType {
        case .photo: photo = nil
        case .livePhoto: livePhoto = nil
        case .video:
            guard let url = url else { return }
            try? FileManager.default.removeItem(at: url)
            self.url = nil
    }
}

Notice that we mark the delete() method as mutating because it’s causing changes to the properties of the structure. Also, to speed up the work, we use the try? keyword with the removeItem(at:) method without a do-catch statement.

Our next stop is the PickedMediaItems class; we’ll add a method which will be calling the one that we just implemented for each item existing in the items array, and then will be emptying it:

func deleteAll() {
    for (index, _) in items.enumerated() {
        items[index].delete()
    }

    items.removeAll()
}

Finally, the last step is to go to ItemsView.swift file, and update the action of the trash button in the navigationBarItems modifier:

.navigationBarItems(leading: Button(action: {
    mediaItems.deleteAll()
}, ...

If you run the app now, all items you have imported to the app will be deleted by tapping on the trash button.

Bonus Part – Visually Distinguish Media Items

This part is not a mandatory step in order to work with PHPicker and handle media items. The essential part of the work has already finished, but this is an additional step that will make the demo application a bit better, and might give you ideas for your own apps. So, reading it is optional.

The way the demo app is right now, makes it difficult to say what the type of each imported media item is simply by looking at it. There is a difference in videos, as there is the play button on top them, but what about normal and live photos? Unless we tap on them, we don’t know their type.

To make it easy to distinguish each media item, we are going to add a small image on top of each displayed media item. Each media type will have a different image, but what makes it interesting is that we are not going to import any custom images to the project. We will use SF Symbols for that.

Note: In case you haven’t done that yet, I really recommend to download the SF Symbols app from Apple.

To display an SF Symbol to an Image view as an image, the Image(systemName:) initializer must be used. The argument we need to pass to it, it’s the name of the SF Symbol we want to use as a string value.

Since we have three different media types, we are going to create a small assistive method in the ItemsView structure which will be performing a simple task; it will return the name of the SF Symbol that should be used, depending on the mediaItem value of each item in the List view.

In the ItemsView.swift file, go right after the ItemsView’s body, and add the following method:

fileprivate func getMediaImageName(using item: PhotoPickerModel) -> String {
    switch item.mediaType {
        case .photo: return "photo"
        case .video: return "video"
        case .livePhoto: return "livephoto"
    }
}

We are using a switch statement in order to return the proper SF Symbol name for each mediaItem possible value. The returned string values are not random; they have been taken from the SF Symbols app. Also, the above method is the second reason why mediaType property was necessary in the PhotoPickerModel type.

Now, in order to display an Image with an SF Symbol on top of each view that displays a media item, it’s necessary to use a ZStack; such a stack piles up contained views on top of each other.

Inside the List view, add the following as the first thing in the closure:

List(mediaItems.items, id: \.id) { item in
    ZStack(alignment: .topLeading) {

    }
}

The argument we provide to the alignment parameter of the ZStack is the desired position of the top-most view. That view will be the Image view with the SF Symbol, and it will be placed on the top-leading edge of the view that exists underneath.

Note: Top-leading edge is the top-left side for LTR localizations, and top-right side for RTL localizations.

Next, inside the ZStack’s closure we will add the conditional presentation of the views as is:

ZStack(alignment: .topLeading) {
    if item.mediaType == .photo {
        Image(uiImage: item.photo ?? UIImage())
            .resizable()
            .aspectRatio(contentMode: .fit)
    } else if item.mediaType == .video {
        if let url = item.url {
            VideoPlayer(player: AVPlayer(url: url))
                .frame(minHeight: 200)
        } else { EmptyView() }
    } else {
        if let livePhoto = item.livePhoto {
            LivePhotoView(livePhoto: livePhoto)
                .frame(minHeight: 200)
        } else { EmptyView() }
    }

    // Image view will go here...
}

Even though we have several lines of code in the condition above, the final result will be the display of one view only. Right below that bunch of lines, exactly where I’ve put the // Image view will go here... comment, we’ll add the Image view:

Image(systemName: getMediaImageName(using: item))

This is where the getMediaImageName(using:) method we implemented above is meant to be used. Even though that line will show the SF Symbol as an image, it would be nice to configure it a bit with view modifiers and make it look nice.

At first we’ll make sure that the SF Symbol image will fit to the Image view size, so we’ll add the next two modifiers:

.resizable()
.aspectRatio(contentMode: .fit)

Then, we’ll set a specific size for the Image view; the resizable() modifier we added above will help on that:

.frame(width: 24, height: 24)

Also, we’ll apply some padding around the image inside the Image view, and we’ll set a semi-transparent background color; with that we ensure that the image will be visible on any background underneath the Image view:

.padding(4)
.background(Color.black.opacity(0.5))

Finally, we’ll set a foreground color to the displayed image in order to make sure that it will be looking properly in both light and dark color scheme:

.foregroundColor(.white)

Eventually, this is how the List view looks after all the above:

List(mediaItems.items, id: \.id) { item in
    ZStack(alignment: .topLeading) {
        if item.mediaType == .photo {
            Image(uiImage: item.photo ?? UIImage())
                .resizable()
                .aspectRatio(contentMode: .fit)
        } else if item.mediaType == .video {
            if let url = item.url {
                VideoPlayer(player: AVPlayer(url: url))
                    .frame(minHeight: 200)
            } else { EmptyView() }
        } else {
            if let livePhoto = item.livePhoto {
                LivePhotoView(livePhoto: livePhoto)
                    .frame(minHeight: 200)
            } else { EmptyView() }
        }

        Image(systemName: getMediaImageName(using: item))
            .resizable()
            .aspectRatio(contentMode: .fit)
            .frame(width: 24, height: 24)
            .padding(4)
            .background(Color.black.opacity(0.5))
            .foregroundColor(.white)
    }
}

If you run the app now you’ll see the SF Symbols presented above each media item, indicating their type. If you run on a device, you’ll see how live photos look too.

Summary

In an undeniably long post, we met the brand new PHPicker API in iOS 14, and how to use it in order to import photos, videos and live photos to your apps. You can now take what you learnt and did here, and use it straight in your projects. Or, feel free to modify whatever you find necessary and make it fit your needs.

Before closing, I’d like to give you a word of caution. Imported normal and live photos are loaded to memory at full size, and you can easily run out of memory if you import many of them. That will lead to unexpected terminations of your app, which is a really bad thing to happen on users’ hands. You have two options to counterattack that; either to resize imported images before displaying them (as an intermediate step after fetching them), or, even better, to use the method we followed for videos; to copy files instead of loading objects. That might not be that easy for live photos, as they require a lot of additional work in order to break and put back together the image and the video, but keep it in mind anyway.

For a small number of imported photos, you’ll have no problem at all. With that said, I hope you enjoyed this tutorial, and that you learnt something new here! Take care!

You can download the complete project on GitHub.

WWDC20 finished almost two months ago, but still we are all talking about the new frameworks, APIs, and improvements announced this year. Among all those there’s something that is going to have a strong impact to the way we work when implementing in-app purchases in our apps. That is the brand new capability of testing StoreKit locally in Xcode 12.

Up until now one had to stop the development workflow and visit the App Store Connect in order to create the necessary in-app purchase records and at least one sandbox user so testing is possible. After having gone through all required steps that we had described in a previous tutorial, developer could continue writing the in-app purchases related code. Obviously a disturbing yet unavoidable change of mindset like that is always enough to slow the development process down. In addition to that, subsequent visits to the App Store Connect would also be necessary most of the times in order to create additional test users.

Thankfully, that slow situation might belong to the past starting from Xcode 12 with the StoreKit local testing! Not only it’s not necessary to stop the development workflow in order to visit the App Store Connect, but implementing and debugging IAPs can now be done at a blazing speed locally, simply using the Xcode and the Simulator. Developers can focus more on code implementation and make in-app purchases work perfectly, without any concern about in-app purchases records or test users until all implementation is finished. An additional benefit of StoreKit local testing is that it works offline, so in-app purchases can be implemented and tested even when there’s no Internet access available.

During the next parts of this post you’ll probably find various reasons why testing StoreKit locally is great. However, what makes it really awesome is the fact that it will eventually make in-app purchases integration a less painful task, and testing implemented logic and any related user interface a much, much faster process.

Before we move on to the next part, there’s one more noteworthy fact; there’s a brand new framework called StoreKitTest which allows to write unit and UI tests, and therefore automate in-app purchases testing. Isn’t that really amazing?

About The Demo App

Like it happens in the most of our tutorials, there’s a project for you to download and start with. However, and contrarily to other times, here we’ll write almost no code at all; just a few bits at the end of the post when we’ll talk about unit testing. What we’ll focus on is the necessary configuration files that allow to test StoreKit locally, and the available options Xcode provides in order to test various states and conditions of in-app purchases.

The demo project we’ll be working on in the upcoming parts of the tutorial is a SwiftUI based application. It’s really simple, as it shows a list with three recipes only which are meant to be purchased and therefore to be unlocked. Nothing will happen after they’re purchased, but we don’t really care about that; what we care about is the behaviour of the app when purchasing under various conditions.

The three recipes (in-app purchase products) are described in a JSON file called Recipes.json and you can find it in the project navigator along with the rest of the project files. Programmatically, a recipe is represented by an instance of the RecipeInfo class which is implemented in the RecipeInfo.swift file. Recipes are handled by the RecipesModel class, which among others contains the recipes array (a collection of RecipeInfo objects), and it is also responsible for reading the JSON file and decoding the recipe data.

RecipesModel class is also the one that triggers the in-app purchase actions, such as loading the in-app purchase products and initiating purchases. It’s important to highlight here that all the in-app purchases related code is implemented in the IAPManager class that you can find in the IAPManager.swift file. This class had been presented and implemented step-by-step in this older tutorial where we had talked about in-app purchases thoroughly, and you might want to take a look there too.

A requirement of the IAPManager class is that all product identifiers available for in-app purchases should exist in a property list (.plist) file, called IAP_ProductIDs.plist. You will find it in the project navigator too, and you can open it to see the identifiers matching to the three products we’ll be testing today.

Before we get to the end of this part, there’s one more important statement I’d like to make:

In order to test StoreKit properly in Xcode and get the maximum out of it, it’s recommended to perform local receipt validation. However, we won’t do that here; local receipt validation is a long discussion, probably the topic of one or more new tutorials, so we’ll just skip it. We’ll do the best we can without validating the fake receipt that Xcode will create for us, but in certain topics, such as subscriptions, we won’t go into much depth. We’ll see all necessary steps to configure a local test, but we won’t actually test anything. In case you have implemented your own local receipt validation mechanism, feel free to move further by following the guides from a link I give you later. To help the UI being in accordance to the purchased state of our recipes in this project, the User Defaults dictionary is used to mark whether recipes have been purchased or not.

Other than that, and after that long description, take your time to explore the demo project, and even visit the previous tutorial and read about the IAPManager class that is used in this project too. Once you’re ready, keep reading to unveil the most important aspects of StoreKit testing locally in Xcode.

Preparing To Test In-App Purchases Locally

There are some standard actions that must be taken in order to test in-app purchases locally in Xcode. The first one is to create a configuration file that simulates App Store by containing in-app purchase records. Depending on the kind of the in-app purchases an app offers, multiple configuration files can be created into the same app; for example, there can be one for testing non-consumable in-app purchases and one for testing subscriptions. However, note that only one can be active and used at any given time.

To create a configuration file, open the File > New > File… menu in Xcode, or just press Cmd+N in your keyboard. In the window that appears with all the available file templates, scroll to bottom under the Other section where you’ll find the StoreKit Configuration File. To spot it easier, you can also type “storekit” in the search bar.

Click next in order to give a name to the file that’s going to be created and added to the project. We are going to work with non-consumable products here, so give the name NonConsumables to it and then click Create. In fact, you can give any name you want, however that cleary indicates what that configuration file is all about.

Once the file appears in the project navigator, click to open it in case it won’t open automatically. Currently it’s empty, and it’s waiting for us to add either in-app purchases or subscriptions. We’ll go for the first, and the way to do that is by clicking on the Plus (+) button to the bottom left side of the window.

There’s a context menu appearing here containing options to add:

1. A consumable in-app purchase.
2. A non-consumable in-app purchase.
3. An auto-renewable subscription.

We want to try out non-consumable in-app purchases here, so click on the second menu item. Here’s what you’ll see:

The Reference field should contain a short descriptive name for the in-app purchase. In our demo project we’re going to have sample recipes for a salad, a spaghetti and a cake, so let’s suppose that this record is about the first one; type Salad as the reference name.

The Product ID is the product identifier as we would also provide it in the App Store Connect. As I mentioned earlier, you can find all product identifiers that will be used in the app in the IAP_ProductIDs.plist file. The one that we’ll use here is the com.appcoda.storekitlocaldemo.salad. This will replace the default value “com.temporary.id”.

The next field is the Price of the in-app purchase. Even though there are price tiers in App Store Connect, here we can provide any value we want as a free text. The price is just for testing reasons, it won’t apply to the real in-app purchases and of course, there will be no charging. So feel free to set any price you’d like. For this specific in-app purchase that we’re configuring here, 0.99 value is just fine. The currency of the price will be the currency matching to the Simulator’s locale, or any locale that has been manually selected (we’ll talk about that later). So, 0.99 can be dollars, euros, yen, and so on.

Right below the three fields described we just met, there’s a checkbox which you can turn on to test the family sharing feature. We won’t do that here, so simply leave it unchecked.

Lastly, there’s the Localizations section. Here is the place where we must provide a display name and a description for our in-app purchase, as we would normally do in the App Store Connect too. Click on the default localization row and a window will be presented as a sheet. Fill the following values in:

• Display name: Ceasars Salad
• Description: The most popular salad everybody loves!

Here’s how the in-app purchase should look now configured:

The steps described above concern one in-app purchase only. However, in our demo application there are three recipes that we’ll offer as in-app purchases, so we have to add and configure the other two in a similar way.

Following the steps as described previously, add two more non-consumable in-app purchase entries and configure them as follows:

1. For the spaghetti recipe in-app purchase:

• Reference name: Spaghetti
• Product ID: com.appcoda.storekitlocaldemo.spaghetti
• Price: 1.19
• Localizations > Display Name: Spaghetti & Meatballs
• Localizations > Description: A tasty meal that makes you happy!

1. For the cake recipe in-app purchase:

• Reference name: Cake
• Product ID: com.appcoda.storekitlocaldemo.cake
• Price: 1.49
• Localizations > Display Name: Chocolate Cake
• Localizations > Description: The most delicious dessert ever!

At the end, you should be able to see the next three in-app purchases in the configuration file:

In case you add in-app purchase records that you don’t want to use, select them by clicking on them (keep Ctrl key pressed for more than one record), and then click on the Minus (-) button at the bottom left side of the window.

Finally, when you finish configuring the in-app purchases press Cmd+S to save your changes.

Using The StoreKit Configuration File

With the NonConsumables.storekit file created and configured, the next step is to tell our app that we want to use that instead of the App Store. To do that start by clicking on the StoreKitLocalDemo scheme in Xcode toolbar, and select the Edit Scheme option.

Make sure that the Run action is selected on the left column, and then choose the Options tab on the main window. You’ll find an option there called StoreKit Configuration, with its current value being to None. Click to open the popup button, and you will find the NonConsumables.storekit file we created right there. Select it and close the scheme editor.

Note that additional .storekit files can be listed along with the NonConsumables.storekit file, but as I have already mentioned and as you just saw, only one can be active at a time.

Testing In-App Purchases

At this point we are able to test for first time in-app purchases locally. Make sure that you have followed all steps as described in the previous two parts, and then run the app in the Simulator. You’ll see the three demo recipes listed, where a lock icon exists next to each one to indicate that they have not been purchased yet.

Click on the first (or any other) recipe in the list, and you’ll see an alert showing app suggesting to buy it. The interesting thing here is that the alert contains the product display name, description and price as they were defined in the NonConsumables.storekit file. If you make any change on any of those three values in the storekit file, you’ll see it being reflected on this alert.

In the presented alert, select to buy the recipe by clicking on the Get recipe for XXX button. The system UI that would appear in a normal purchase from the App Store appears here too in order to simulate the purchase experience, and all you have to do to continue with the transaction is to click on the Confirm button. Of course, no charge will happen, and that’s something that is also clarified in the details section of the presented UI.

After confirming the purchase you’ll see a system alert notifying that the transaction was successful. The lock icon next to the selected recipe will disappear right after you dismiss that alert, which is the indication in our demo project that the recipe has been purchased.

The Transactions Manager

All transactions that take place locally using a storekit file like the one we created previously are visible in a special window in Xcode. This is the transactions manager, and you can present it either by going to the Debug > StoreKit > Manage Transactions… menu, or by clicking on the Manage StoreKit Transactions button in Xcode’s status bar while the app is running:

When transactions manager window is presented, you’ll find the transaction that you made in the previous part listed in it.

What is really great with local testing is that we can delete any transaction that was made so far and test our in-app purchases from start immediately. Using the App Store in order to do the exact same thing we would need to create new test users every time we would like to start testing fresh.

To see that in action, select the transaction you see in the manager window, and either click on the Trash button, or right click on it and select Delete Transaction.

Run the app again, but before you click to buy the same recipe again, please make sure to click on the Refresh button so the recipe is marked as non-purchased in the User Defaults dictionary and its visual state to be initialized. After you do so and you see the lock icon again, click to purchase the exact same recipe that you purchased before. You’ll see that you’ll go through the exact same steps you met previously. With a couple of clicks we managed to revert the app to its original state and to test in-app purchases from scratch, like nothing had already happened before.

Back to the transactions manager again, you’ll see that the new transaction is present, this time with a different identifier. Besides just deleting the purchase, you can also refund it through the manager window by selecting the respective option either in the toolbar or by right clicking on it. You can do so if you want now, however there will be no visual update in the UI of our demo app; as declared at the beginning, we won’t do any receipt validation here, and refund is something that can be checked only in the receipt. However, if you have your own local receipt validation mechanism, you can pretty easily check if the the transaction is marked as cancelled in the receipt, and update the UI of this demo or any other app accordingly.

Testing Interrupted Purchases

The scenario we tested above is the one everybody wishes for their apps. However, things are not going always as smoothly as we would want, so handling cases out of the ordinary is something that we must take care of as well. Such a case is an interrupted purchase; a purchase that cannot be completed because an action out of the app must be taken by the user. For example, the user might need to update the payment information to the App Store in order to be able to make purchases.

To test interrupted purchases, first select the NonConsumables.storekit file in the project navigator. Then go to menu Editor > Enable Interrupted Purchases. By doing so, any purchase you’ll be making from now on will be handled as interrupted, so don’t forget to go to the Editor > Disable Interrupted Purchases menu to switch back to normal later.

The IAPManager class provided in the starter project that contains the in-app purchases related implementation will return an error in case of an interrupted purchase. That error is a SKErrorDomain with value 0, meaning an unknown error. Most of the times you’ll want just to inform users that the transaction cannot be completed, and this is what happens in this demo project too. Of course, you can follow a different approach in your own apps and treat that error in a way that you find more proper and suitable.

After having turned interrupted transactions on, run the app and try to purchase a recipe that you haven’t purchased already. If you have purchased all of them, then use the transactions manager window to delete them, and don’t forget to use the Refresh button in the app to initialize the UI too. Then, start the process of purchasing a recipe, and see what happens right after you confirm it through the system UI; an alert informing that the transaction could not be completed is presented, and you can see the error printed in the console at the same time too.

In the testing environment that we’re working on we can resolve the issue that caused the interrupted transaction and see how our app will behave. Go back to Xcode and open the transactions manager window. You’ll find the transaction that was made right there, marked as failed. To proceed, select it and use either the Resolve Issues button in the toolbar, or right click on it and choose the Resolve Issue option from the context menu.

The fake issue will be resolved and the transaction will continue normally. If you go back to the app, you’ll see that the purchase is now complete. At the same time, a new entry will show up in the transactions manager where the selected recipe is marked as purchased.

Testing Ask To Buy

Another option that can be found in the Editor menu when the NonConsumables.storekit file is selected in Xcode is the Enable Ask to Buy. By enabling it you can test what will happen if an underaged user initiates the purchase process through your app and the Ask to Buy feature is turned on on their device.

To see it in action make sure to open NonConsumables.storekit by selecting it in the project navigator, and then go to menu Editor > Enable Ask to Buy.

Note: Make sure that you’ve disabled interrupted purchases first.

Run the app again and start the purchase process of a recipe. After confirming the purchase you’ll get the following system alert:

By selecting the Ask option you’ll see a detailed error logged in the console saying that the payment was complete but with errors. Until the parent or guardian of the underaged user who initiated the purchase will approve or reject it, there’s no need to change our UI. However, we must make sure that the recipe will become available in case of approval.

Once again, testing can continue through the transactions manager in Xcode. Open it, and you’ll find the transaction we just made being listed there, marked as Pending Approval.

Select the transaction and use the toolbar or right click on it and select the Approve Transaction to approve it. Switch back to the running application in the Simulator, and you’ll see that the recipe has being unlocked.

Read more about the Ask to Buy feature here.

Changing The Storefront

While testing in-app purchases locally in Xcode 12, it’s possible to change the default storefront and make sure that your app presents the price of the selected product properly formatted using the correct currency. Responsible for formatting properly the price in our demo app is the getPriceFormatted(for:) method of the IAPManager class. It’s already used as needed in the RecipesView, so all we have to do is to test if it’s working as expected. Before we continue, remember that during the configuration of the in-app purchases in the storekit file we set just a numerical value as the price; we did not set any currency. The displayed currency depends on the selected storefront.

Back in Xcode, open the NonConsumables.storekit file, and then go to Editor > Default Storefront > Greece menu. This will simulate the use of the Greek App store, and the euro sign will be displayed instead of the dollar sign next to the price when we’ll purchase a recipe.

After you make the above change, run the app and initiate a new purchase. You’ll see that this time the Euro currency sign is displayed next to the price value.

Changing The Default Localization

In conjunction to the default storefront, you can also change the default localization that will affect the displayed product metadata. To do so, make sure that the NonConsumables.storekit file is open, and then go to Editor > Default Localization > Greek menu (or choose any other localization you want).

However, simply changing the default localization is not enough. We must provide the respective display name and description that will be used for that localization in the configuration file. Let’s do that for the Salad in-app purchase.

Note: If you run the app without providing localized display name and description for a product, then no title and description will appear in the alert that prompts to buy the recipe.

In the NonConsumables.storekit file select the Salad in app purchase. Then, click on the Plus (+) button in the Localizations section. In the localization configuration window, first select the Greek locale (or any other locale you prefer). Then provide a localized display name and description in the language you prefer, or use the example given here. For the Greek locale, that would be:

• Display name: Σαλάτα του Καίσαρα
• Description: Η διάσημη σαλάτα που όλοι λατρεύουν!

Click on the Done button, and the new localization will appear in the Localizations list.

Run the app now, and initiate a purchase process. You’ll see that the message appearing in the alert is now translated to the selected localization, and the currency of the price is in accordance with that localization.

Of course, in a real app the UI should be translated too, however what we focus on now is the in-app purchases only; so just don’t mind about the rest of the UI that is still in English.

Creating A Configuration File To Test Subscriptions

Even though we won’t test subscriptions as they require receipt validation in order to properly work, we’ll go briefly through the steps of creating a storekit configuration file which includes subscription records.

Note: Don’t forget to edit the app scheme and select any new storekit configuration file in the StoreKit Configuration popup (Run action, Options tab) that you’d like to use for testing.

To start, go to File > New > File… menu, and then search for the “storekit” file template. Select it and proceed by naming it as Subscriptions. Once you see the Subscriptions.storekit file in the project navigator, click to open it.

The first move here is to add a new auto-renewable subscription record to the configuration file. Go to the Plus (+) button to the bottom left side of the configuration window, click on it and select the Add Auto-Renewable Subscription option from the context menu.

Since this is the first subscription added, you’ll see a window asking to create a subscription group. Such a group can contain multiple subscriptions, but users can subscribe to one of them only at a time, having however the ability to switch subscriptions that belong to the same group whenever they want.

Even if you are going to have one subscription only a group is necessary to be created here, so just type a name in, such as MySubscriptions or anything else you want. Click on the Done button to proceed.

As you can see in the subscription configuration, there are several fields that we can provide values for, depending always on what we’d like to test.

The first section under the Auto-Renewable Subscription section is pretty similar to the set of fields we filled in for each in-app purchase in the NonConsumables.storekit file. Each subscription should have a reference name, a product identifier, and a price that users should pay when the subscription is renewed. Additionally, here we have the Subscription Duration field for specifying how long the subscription we’re about to test will last.

Right below there’s the Introductory Offer section. Use it if you’re planning to provide a one-time introductory offer to the users and you want to test whether your app works as expected or not. To configure the introductory offer start by clicking on the Offer Type popup button. Select one of the three options (besides None) depending on the subscription kind:

• Pay as you go
• Pay up front
• Free

For any selected option you have to specify the offer duration in a new field that appears, and for the first two options there’s a price field to fill in as well that contains the offer’s lower price.

Upon testing the introductory offer, the offer price (or free) specified right above should be displayed in the system sheet presented to confirm the payment. Remember that any transaction is listed in the transactions manager window, so make sure to delete them from there if you want to start fresh.

To test if your subscription will be renewed properly after the period of the introductory offer, you can speed things up by going back to Xcode and the Subscriptions.storekit file. Then go to Editor > Time Rate menu, and choose a time rate that you find fast enough. Usually, “1 second is 1 day” is the option to go if you want to test quickly. Once the time period has expired, switch back to a slower time rate, and test if your app renews the subscription properly by reading the proper fields in the receipt.

Besides the introductory offer that you can test, there’s also the Promotional Offers section. Here you can define offers that you’d like to test for each subscription, which you can display either when the subscription expires or when you decide it’s the best time depending on certain conditions.

To create a promotional offer click on the Plus button in that section and fill the required values in. Type in custom value for the reference name and the product code, and then select the offer type, its duration, and a price if it’s not a free offer. Once again, use the Editor > Time Rate menu to speed up or slow down the time rate while testing.

Lastly, don’t forget to set a display name and a description for the default localization that will be displayed to users upon purchasing the subscription. This is similar to the localization values we also met previously in the in-app purchases storekit configuration file.

Note that for any new subscription record you’ll add to this configuration file, you’ll be given the option either to append it to the current subscriptions group, or create a new group to contain it.

You are recommended to take a look at the guides you’ll find here about testing subscriptions.

Writing Unit Tests

Besides all the manual work that can be done to test in-app purchases locally as shown in the previous parts, it’s also possible to create unit and UI tests thanks to a new framework called StoreKitTest. This framework provides the SKTestSession class, which in turn offers several methods and properties in order to configure test conditions and perform fake transactions.

We’re going to write a couple of unit tests here and get a small, first taste of the StoreKitTest framework. Begin by opening the StoreKitLocalDemoTests.swift file; the place to write unit tests in our project. Then, go to the top of the file, and right after the import XCTest line, import the StoreKitTest framework:

import StoreKitTest

In our first test we’re going to do something really simple; we’ll make a purchase and we’ll ensure that it’s completed successfully. Since it’s a test environment things can’t go wrong, but it’s a good chance to meet a few bits of the new API and validate that our in-app purchases code works properly.

Let’s define the following new method:

func testSaladPurchase() throws {

}

Initially, we’ll create a SKTestSession instance through which we’ll test the in-app purchase:

func testSaladPurchase() throws {
    let session = try SKTestSession(configurationFileNamed: "NonConsumables")
}

See that upon initialization we provide the name of the storekit configuration file we want to use.

Then, in order to have everything run automatically from start to finish, let’s disable the appearance of any system dialogs and UI with the next line:

func testSaladPurchase() throws {
    ...
    session.disableDialogs = true
}

Lastly, we’ll make the purchase using the buyProduct(productIdentifier:) method of the SKTestSession class through the session object. This is a method that can throw an exception, and since this is the last line of our method, we’ll call it as the argument of a XCTAssertNoThrow(_:) method:

func testSaladPurchase() throws {
    ...
    XCTAssertNoThrow(try session.buyProduct(productIdentifier: "com.appcoda.storekitlocaldemo.salad"))
}

Notice that it’s necessary to specify the product identifier matching to the in-app purchase we want to test. Here’s the full method:

func testSaladPurchase() throws {
    let session = try SKTestSession(configurationFileNamed: "NonConsumables")
    session.disableDialogs = true
    XCTAssertNoThrow(try session.buyProduct(productIdentifier: "com.appcoda.storekitlocaldemo.salad"))
}

Let’s run the test now. Click on the small play icon on the left side of the method’s name, and wait until the test is finished. It should be successful, if it isn’t, please make sure that you followed all steps here as described.

To see the purchase that was just made, first run the app by pressing Cmd+R; then open the transactions manager; the transaction will be listed there.

It’s possible to delete all previous transactions each time a test is running. Right after the session initialization add the next line:

session.clearTransactions()

Let’s pass to the second test now. This time we’ll test the Ask to Buy scenario in a new test method:

func testAskToBuy() throws {

}

As previously, we’ll start by initializing a test session object and by performing some basic configuration:

let session = try SKTestSession(configurationFileNamed: "NonConsumables")
session.clearTransactions()
session.disableDialogs = true

To enable the Ask to Buy feature we just need to set true to the following flag:

session.askToBuyEnabled = true

Now we can buy the same in-app purchase as before:

XCTAssertNoThrow(try session.buyProduct(productIdentifier: "com.appcoda.storekitlocaldemo.salad"))

If the above assertion is successful, then a new transaction will be created with the purchase pending for approval. In this case, the transaction that was made remains in the deferred state and it’s completed only once it gets approved or rejected. To test that, let’s add the next two assertions:

XCTAssertTrue(session.allTransactions().count == 1)
XCTAssertTrue(session.allTransactions()[0].state == .deferred)

Here’s the entire method so far:

func testAskToBuy() throws {
    let session = try SKTestSession(configurationFileNamed: "NonConsumables")
    session.clearTransactions()
    session.disableDialogs = true
    session.askToBuyEnabled = true

    XCTAssertNoThrow(try session.buyProduct(productIdentifier: "com.appcoda.storekitlocaldemo.salad"))

    XCTAssertTrue(session.allTransactions().count == 1)
    XCTAssertTrue(session.allTransactions()[0].state == .deferred)
}

Run the test, and once it’s marked as successful, run the app and open the transactions manager. You will find the in-app purchase in the pending approval state:

Let’s add one last assertion in the above test method where we’ll be approving the purchase:

func testAskToBuy() throws {
    ...

    XCTAssertNoThrow(try session.approveAskToBuyTransaction(identifier: session.allTransactions()[0].identifier))
}

The identifier that the approveAskToBuyTransaction(identifier:) method requires is the transaction identifier as shown in the transactions manager.

Run the test again, and then run the app and open the transactions manager; this time the transaction is marked as purchased because it was approved with the last line we just added.

In the above two tests we met a few methods and properties of the SKTestSession class. I prompt you to read more about this class and all the available API to use, so you can create tests that can cover all possible cases when dealing with in-app purchases.

Summary

Testing StoreKit locally is a game-changer when the implementation of an app comes to in-app purchases. It can really save a significant amount of time and allows to focus on the important things; how to make an app work properly when it offers in-app purchases. Of course, this doesn’t mean that testing with real in-app purchase records in the App Store Connect shouldn’t be done at all.

On the contrary, that’s the next step once we ensure that our app works properly after testing locally. In any case, I hope that you found some valuable and interesting stuff among the lines of this tutorial, and please make sure to watch the Introducing StoreKit Testing in Xcode session of WWDC20. Enjoy testing StoreKit!

For reference, you can download the complete project here.

Hey everyone and welcome back to the second and final part of this tutorial series where we explore the intricacies of Apple’s MusicKit by building our very own music player in SwiftUI that can stream songs from our Apple Music account. If you haven’t read Part 1, you can do that here.

In the last tutorial, we looked at how to create a MusicKit Identifier for our Apple Developer account, created a JSON Web Token private key, and we were able to successfully make web requests to the Apple Music API. At the end of the tutorial, we also built the UI of our music player and were able to make it look like below.

In this tutorial, we’ll start making calls to the Apple Music API to populate our app with real data. We’ll also look at how to control media playback with the MediaPlayer framework and enhance the app to this.

Cool, right? Let’s get started.

Note: This tutorial was made using Xcode 11.4 and Swift 5.1. An active Apple Music subscription will be needed in order to test the app out. You will also need to run your app on a real device because as of this time, the Simulator does not support Apple Music playback.

Apple Music API vs iTunes Search API

If you’ve worked with podcasts, tv shows, or any other Apple generated media content, you must have come across the iTunes Search API. The iTunes Search API lets you search for content in the iTunes Store, App Store, and iBooks Store. This will give you access to information about apps, books, movies, podcasts, music, music videos, audiobooks, and TV shows. The best part about using this API is that it’s completely free so you can get updated information, on the fly, for no additional cost.

But why do we use the Apple Music API instead of the iTunes Search API? This is because of how much information and power the Apple Music API can deliver. While the Apple Music API provides the same functionality as the iTunes Search API (fetching information about music and music videos), the Apple Music API takes it to the next level by accessing a user’s personal library and recommendations.

This means your app can leverage a user’s playlists, songs, or ratings for their content. This is why it’s required for your user to have an Apple Music account if you will be using the API. Another important reason why the Apple Music API is better over the iTunes Search API is because it is built to work harmoniously with the MediaPlayer framework. As you’ll see, streaming audio is a breeze with the Apple Music API. Let’s get started!

Creating our Classes

Due to the nature of SwiftUI, making URL calls within our structs can get really complicated. Furthermore, it can get pretty complicated to populate our UI with different information gathered from different songs. This is why we’ll create an AppleMusicAPI class that will contain the functions we’ll frequently use in our app and a Song structure to make it easier to populate our UI.

Before we begin, download SwiftyJSON.swift and add the file to your project. This file contains the code for the popular library SwiftyJSON. This is a tremendous tool when making HTTP requests and receiving and parsing JSON responses. You can find more information about SwiftyJSON here.

To begin, choose to File > New > File. From the popup that appears, select Swift file. Name this file AppleMusicAPI.swift and save this file in the default location. You should now have a template file as shown below.

Similarly, let’s create a struct for our songs. Go to File > New > File and select Swift file. Name this file as Song.swift and save this file. We’ll begin creating our Song structure first.

Add the following code below import Foundation:

struct Song {
    var id: String
    var name: String
    var artistName: String
    var artworkURL: String

    init(id: String, name: String, artistName: String, artworkURL: String) {
        self.id = id
        self.name = name
        self.artworkURL = artworkURL
        self.artistName = artistName
    }
}

The code above creates a simple structure for Song. In case if you’re not familiar with structures, don’t worry. A structure is a type of class that helps make your code more concise, especially when dealing with structural data. Structures, just like classes, can define properties, methods, and initializers to set up their initial state. This is what we are doing with our init method. We are providing a default initializer method for the Song structure that lets a user input the ID, name, artist’s name, and album artwork’s URL.

Now switch over to the AppleMusicAPI.swift file. Here, we’ll be implementing a class called AppleMusicAPI that stores our developer token and a bunch of methods that will help when playing our song through our app.

Remove import Foundation and Add the following code to AppleMusicAPI.swift:

// 1
import StoreKit

// 2
class AppleMusicAPI {
    // 3
    let developerToken = "YOUR DEVELOPER TOKEN FROM PART 1"

    // 4
    func getUserToken() -> String {
        var userToken = String()

        return userToken
    }
}

The above code is pretty self-explanatory but don’t worry if this is a little unclear. Here’s what the above function does.

1. First, we import the StoreKit framework. This helps us access a lot of built-in methods that can communicate with the Apple Music API.
2. Here, you’ll notice we’re defining a class called AppleMusicAPI. This makes it easier to manage multiple instances of this class and reference the methods from other views in our app.
3. Here we’re defining a constant developerToken containing the developer token we created in Part 1 on this tutorial series. This makes it easier to call the token when communicating with the API.
4. Finally, we define our first method in this class: getUserToken(). As mentioned earlier, the Apple Music API can get a user’s library and playlists. This is only possible if we receive a token that is identifiable to a particular user.

Let’s finish implementing the rest of the getUserToken method.

func getUserToken() -> String {
    var userToken = String()

    // 1
    let lock = DispatchSemaphore(value: 0)

    // 2
    SKCloudServiceController().requestUserToken(forDeveloperToken: developerToken) { (receivedToken, error) in
        // 3
        guard error == nil else { return }
        if let token = receivedToken {
            userToken = token
            lock.signal()
        }
    }

    // 4
    lock.wait()
    return userToken
}

You might face some new code in the above snippet. Don’t worry though as most of it will be explained.

1. First, we define a lock that is of type DispatchSemaphore. What is a DispatchSemaphore? A dispatch semaphore is an efficient implementation of halting a thread until a particular message has been passed. This “locks” a thread from executing more code until a signal has been given.
2. We access the SKCloudServiceController().requestUserToken() method to get a token that authenticates the user in personalized Apple Music API requests. Notice how we use our constant developerToken in this method. It’s definitely easier than typing the long string again and again.
3. Here, we write some code to error check what the requestUserToken() function returns. If receivedToken is not empty, we set it equal to our userToken variable from above. Notice, afterwards, we call lock.signal(). This lets the dispatch semaphore we create earlier know that it’s ok to start executing remaining code.
4. Since this method executes asynchronously, it’s possible that when getUserToken() executes, it can skip to the line return userToken, even before a token is received from the SKCloudServiceController. By adding the lock.wait() line of code, this tells the code to halt executing any further code until a signal is given (as we implemented in Step 3).

Note: Dispatch sophomores must be used with caution. This can transform code to execute synchronously which can slow down an app or not update UI in time. In fact, if lock.signal is not called, the app will forever remain stuck until the user restarts the app. Thus, it’s not advised to use this in big production apps. However, for our purposes, it’s completely acceptable.

Now, let’s implement our next method: fetchStorefront().

A storefront is an object that represents the iTunes Store territory that the content is available in. When we perform a search using the Apple Music API, we’d like to show results relevant to our user’s location. Beneath, getUserToken(), add the following method:

func fetchStorefrontID() -> String {
    // 1
    let lock = DispatchSemaphore(value: 0)
    var storefrontID: String!

    // 2
    let musicURL = URL(string: "https://api.music.apple.com/v1/me/storefront")!
    var musicRequest = URLRequest(url: musicURL)
    musicRequest.httpMethod = "GET"
    musicRequest.addValue("Bearer \(developerToken)", forHTTPHeaderField: "Authorization")
    musicRequest.addValue(getUserToken(), forHTTPHeaderField: "Music-User-Token")

    // 3
    URLSession.shared.dataTask(with: musicRequest) { (data, response, error) in
        guard error == nil else { return }

        // 4
        if let json = try? JSON(data: data!) {
            print(json.rawString())
        }
    }.resume()

    // 5
    lock.wait()
    return storefrontID
}

1. Just like earlier, we create a dispatch semaphore called lock to make sure that the function returns a storefrontID only after the data has been received from our URL request.
2. We create a URLRequest using the Apple Music API url. A lot of the details about handling requests and responses from the Apple Music API can be found here, but here’s the gist. To compose a request to the API, first specify the root path, https://api.music.apple.com/v1/  . Here, we specify the storefront component. Using a GET Request, we create a request, adding our developer token in the header.
3. Next, we use URLSession to send musicRequest. This is very similar to performing any networking requests for either obtaining images or data. As the .dataTask method is built, we are returned with three constants: data (the data the network response sends back), response (details about the response), and error(this may be nil if there is no error). After checking to make sure there is no error, we move to Step 4.
4. The data we get is a JSON payload. A payload is the part of transmitted data that is the actual intended message. Using the SwiftyJSON library we added earlier, we’ll print the raw JSON output before parsing it. Notice that we’re not calling lock.signal() here because we’re not setting anything to the storefrontID. If we did call lock.signal(), our app would crash when testing it. We’ll add the signal when we parse our JSON result.
5. Finally, we ask the dispatch semaphore to wait before returning the storefront’s ID.

This was a lot of code so now is a good time to make sure everything is working properly. We’ll switch to ContentView.swift and add an onAppear function to our TabView. Whatever code we place inside this method will be executed when TabView is displayed to the user. The code we’ll place inside will ask the user for permission to access their media library and upon authorization, will call the fetchStorefront() method which will print the JSON payload.

Let’s do it! At the top of the file, add import StoreKit and then make your TabView view look as such

TabView(selection: $selection) {
// Previous Code
    ...
}
.accentColor(.pink)
.onAppear() {
    SKCloudServiceController.requestAuthorization { (status) in
        if status == .authorized {
            print(AppleMusicAPI().fetchStorefrontID())
        }
    }
}

These few lines of code are easily comprehendible. We ask the SKCloudServiceController (remember, this is an object provided by StoreKit that determines the current capabilities of the user’s music library) for authorization to access a user’s media library. If the user authorizes this access, we run print(AppleMusicAPI().fetchStorefrontID()) which will print the JSON payload of the request.

Notice how we call out AppleMusicAPI class. Similar to ContentView in SwiftUI or ViewController in storyboard-based Swift, by adding the pair of parentheses after the class’s name, we are initializing it which gives us access to all its constants and methods, such as the fetchStorefrontID() method.

Now before we run the app to make sure everything is functioning as expected, we need to make a slight addition to our Info.plist. We need to add the following property that gives gives a description to the user what we need access to their music library for. The key is Privacy - Media Library Usage Description and the value can be any message you wish it to be that adequately informs users what the data will be used for.

Now we can run out app!

You’ll need to run the app on a physical device with an active Apple Music subscription. Unfortunately, the simulator does not have the Music app so accessing a user’s Apple Music account is near impossible. Having an active Apple Music subscription lets you test the features of your app on your account.

After authorizing the app to access your Apple Music account, you should wait for some time and expect an output that looks like the one below.

Don’t worry about the error messages stating something along the liens of “Unable to get the local account”. Beneath that, you’ll see a JSON string printed out. This goes to show that our code is working. Let’s go back to AppleMusicAPI.swift and finish the rest of the fetchStorefrontID() method. Delete the line print(json.rawString()) and replace it with the following

// 1
let result = (json["data"]).array!
let id = (result[0].dictionaryValue)["id"]!

// 2
storefrontID = id.stringValue

// 3
lock.signal()

1. We set result to be the array the JSON payload provides underneath the data key. id is simply value provided under the id key in the dictionary provided in result. This can sound a little confusing. The names of keys and values can be derived from the JSON string we printed earlier. To learn more about reading and parsing JSON, this [article][4] can help you.
2. We set the storefrontID to the string value of the id constant from Step 1.
3. Just like earlier, we signal the dispatch semaphore that it’s okay to execute remaining code and free up the thread.

Your entire fetchStorefrontID() method should look like this.

Run the app again and you’ll see how instead of printing a JSON file, only two characters are printed, signifying the storefront ID of your Apple Music account. For me, in the United States, a simple “us” is printed out.

If everything works, give yourself a pat on the back! We have one last method to implement in our AppleMusicAPI class and this one is pretty big. This method will allow us to search Apple Music’s entire library of 40 million+ songs based on the keywords given. Now’s a good time to take a break!

Music Search

Underneath our fetchStorefrontID() method, let’s add a new method called searchAppleMusic(searchTerm:) that will let us search Apple Music’s library based on the term. Here’s a boilerplate to get you started. You’ll notice it’s very similar to our fetchStorefrontID() function.

func searchAppleMusic(_ searchTerm: String!) -> [Song] {
    let lock = DispatchSemaphore(value: 0)
    var songs = [Song]()

    let musicURL = URL(string: "https://api.music.apple.com/v1/catalog/\(fetchStorefrontID())/search?term=\(searchTerm.replacingOccurrences(of: " ", with: "+"))&types=songs&limit=25")!
    var musicRequest = URLRequest(url: musicURL)
    musicRequest.httpMethod = "GET"
    musicRequest.addValue("Bearer \(developerToken)", forHTTPHeaderField: "Authorization")
        musicRequest.addValue(getUserToken(), forHTTPHeaderField: "Music-User-Token")

    URLSession.shared.dataTask(with: musicRequest) { (data, response, error) in
        guard error == nil else { return }

    }.resume()

    lock.wait()
    return songs
}

In terms of variables, we have our routine dispatch semaphore defined and a new songs array of type Song that we defined earlier. We’ll be storing the songs in this array and returning this array to populate our table. You’ll notice that searchTerm is a String provided as an input to this function. We also perform string manipulation on searchTerm to replace any instances of whitespaces with the + symbol as URL’s can’t contain any whitespaces.

We also create the URLRequest using the Apple Music API url. Notice that like before, it uses the root path https://api.music.apple.com/v1/. However, the URL component is catalog and we pass the search term as a parameter of the URL. Apart from these changes, we again use the GET request, create the request, and add our developer token to the header.

Now comes the important part which is parsing the data returned from our URLSession.shared.dataTask. Let’s think about what we need. We need to parse the data for an array of songs. We then need to isolate the different components of the each song in the array such as title, artist, album artwork, etc. After isolating these components, we can create an object of type Song and add this to our songs array. This is how it can be accomplished.

Type the following code and place it underneath guard error == nil else { return } inside the URLSession.shared.dataTask() method:

// 1
if let json = try? JSON(data: data!) {
    // 2
    let result = (json["results"]["songs"]["data"]).array!
    // 3
    for song in result {
        // 4
        let attributes = song["attributes"]
        let currentSong = Song(id: attributes["playParams"]["id"].string!, name: attributes["name"].string!, artistName: attributes["artistName"].string!, artworkURL: attributes["artwork"]["url"].string!)
        songs.append(currentSong)
    }
    // 5
    lock.signal()
} else {
    // 6
    lock.signal()
}

This is very similar to the code we wrote above to fetch a user’s storefront ID. The only difference it there’s a lot more JSON parsing and we are creating a Song object to add to our array.

1. First, we type check to see is the data returned is in a valid JSON format. If it is, we create a constant called json containing all this data in the JSON file format.
2. We parse json for the songs that is nested within results -> songs -> data in an array format. Now, I don’t want to burden you with more JSON formatting but this order was derived by printing the raw string of json and noticing how each component was ordered. If you want, you can print json to see how result is nested within the JSON data. result will now contain an array of our songs.
3. Now, we parse through each song in result using our trusty for loop.
4. First, we define a constant called attributes that is a dictionary of all the attributes of our current song. These attributes contain lots of information about the current song but we’re interested in the id, name, artist’s name, and artwork URL. Then we create a currentSong constant of type Song that is populated with the values from the attributes dictionary. Finally, we append the currentSong to our songs array.
5. Last but not least, we signal our dispatch semaphore throughlock so the rest of the thread can be freed up to run the remaining code.
6. This is more of a check but if the data returned is not in a valid JSON format, we don’t want our app to stay stuck forever. This is why we add an else clause to signal the dispatch semaphore to continue the rest of the code.

Now we need to make sure our function works. Head over to ContentView.swift. In the .onAppear() declaration, we need to make a very minor change to the statement where we print AppleMusicAPI().fetchStorefrontID(). All we need to do is replace .fetchStorefrontID() with .searchAppleMusic("Taylor Swift"). If all goes well, when we run the app, this will print an array of Song objects containing all the songs Apple Music returns when its catalog is searched for Taylor Swift. (Of course, you can replace this with any artist, album, song, or word of your choice).

Run the app and observe the console. After a few seconds, you should see an output similar to the following:

As you can see, a whole array containing the songs based on your search term. We can look at the artist name, id, name of the song, and a URL to the album artwork. Since we know our code works, let’s delete the entirety of the .onAppear() declaration. That way we don’t necessarily make calls to the API when our app loads.

Now comes the second part of this section which is populating our search table view with the results.

Populating the Table

Go to SearchView.swift. We’ll be making all the changes here to populate our search table. There are many changes we have to make so follow carefully.

First, we need to replace the songs array containing some temporary strings. Delete that line and replace it with the following line:

@State private var searchResults = [Song]()

This creates a new empty array called searchResults that conforms to our Song object. We also link it to the @State property so anytime any changes are made to this array, any UI using this variable will be updated.

Now, as expected, you’ll get some errors since we removed our old songs array. Let’s start at the ForEach within our List SwiftUI component. The error displayed here is that we’re still trying to iterate over a variable songs that doesn’t exist. Replace the ForEach line with the following:

ForEach(searchResults, id:\.id) { song in
    ...

There are three changes we’ve made here.

1. The first change is we’ve replaced songs which searchResults.
2. Now since our Song object doesn’t conform to the Hashable property, we can’t use .self to use it as an id. Good news though! Our Song objects have an id property to identify each element. This is why we replace \.self with \.id.
3. Finally, and this is really minor, we replace the songTitle variable to song. songTitle referred to the string from our old array. This isn’t very representative of the current array for which we are iterating which is why we rename it to song.

While this error is removed, we now have errors wherever we used the variable songTitle. This is a simple fix as we replace songTitle with song.name. Modify the code after the Image object within your HStack to look like the following:

VStack(alignment: .leading) {
    // 1
    Text(song.name)
        .font(.headline)
    // 2
    Text(song.artistName)
        .font(.caption)
        .foregroundColor(.secondary)
}
Spacer()
Button(action: {
    // 3
    print("Playing \(song.name)")
}) {
    Image(systemName: "play.fill")
        .foregroundColor(.pink)
}

1. first change we made was to replace songTitle with song.name. This will use the name of the song as the string value for this Text object.
2. The second change we made is to replace the ”Artist Name” placeholder with the actual artists name. Just like song.name, we use song.artistName to populate this Text object with the value in song.artistName.
3. Finally, the last change we made was a slight modification to our Button object which is to replace songTitle with song.name just like Step 1.

Now, you can build and run the app. Everything should compile and work as normal but there’s a catch. You’ll notice that when you search for a song and press enter, nothing happens. This is because we still haven’t called the searchAppleMusic function from our AppleMusicAPI() class. To do this, we need to make some changes to our TextField. Before this, at the top of the file underneath import SwiftUI, add the line import StoreKit.

Now, make the following changes to your TextField:

TextField("Search Songs", text: $searchText, onCommit: {
    // 1
    UIApplication.shared.resignFirstResponder()
    if self.searchText.isEmpty {
        // 2
        self.searchResults = []
    } else {
        // 3
        SKCloudServiceController.requestAuthorization { (status) in
            if status == .authorized {
                // 4
                self.searchResults = AppleMusicAPI().searchAppleMusic(self.searchText)
            }
        }
    }
})
.textFieldStyle(RoundedBorderTextFieldStyle())
.padding(.horizontal, 16)
.accentColor(.pink)

We’re at the last stretch here but this code shouldn’t be too complicated for you to read. Here’s a gist of what it compiles:

1. First, we’d like to dismiss the keyboard when the user presses Search on the keyboard. This is done by calling the line UIApplication.shared.resignFirstResponder() .
2. Next, we’d like to make sure that if the user doesn’t enter anything, our searchResults are empty. This is why we set it to an empty array if the searchText is empty.
3. If the searchText variable is not empty, we’d like to call our searchAppleMusic function. Before we do this, we make sure that SKCloudServiceController has the necessary authorization to access a user’s media library. If the user authorizes this access, we can continue to the next line of code.
4. Finally, we set out searchResults array equal to the results of AppleMusicAPI().searchAppleMusic(self.searchText). This will pass the search term to our searchAppleMusic function and update our searchResults array with whatever Apple Music returns!

We’re finally ready to run our app. Build and run the app and head over to the Search page. Enter any term, wait for a couple seconds as your app makes network requests you’ve implemented, and watch with delight as the table view is populated with the results of your search term.

Congratulations! You’ve mastered the basics of the Apple Music API! As you can see, it’s very repetitive in that the steps can be summed down to the following:

1. Make a network call to the Apple Music API
2. Parse the JSON to see where the data you need can be found
3. Create an object and populate it with the data you’ve parsed

For the remainder of the tutorial, we’ll be focusing on the under appreciated MediaPlayer framework and its very important object, MPMusicPlayerController . Also, don’t worry about the album artwork not displaying as of now. I’ll show you how we can use Swift Package Manager to successfully load and display images from a URL.

Implementing Music Play Back

Get excited because now we’ll be looking at the MediaPlayer framework and how to access and control your device’s music player. Essentially, the MediaPlayer framework is a part of MusicKit and lets you control playback of the user’s media from your app.

To play content using this framework, we have to instantiate one of the built-in MPMusicPlayerController objects. There are two types. Here is a brief description of what they are:

1. System Player: This media player is directly linked to the Music app on your device. If you choose to use this player, then when a user click on a song, it will open up Music and start playing from that app.
2. Application Player: This will be a media player that is built-in directly to your app. That way when a user clicks on a song, they won’t be transported to the Music app, but rather all the playback functionality will occur in your app. Of course, this means that you will have to add play/pause and skip/rewind functionality. We’ll be using this type of player for our app.

Let’s add the application player to our app. Now we want to modify this object within both screens of our app. In our Player View, we want to control the playback of the currently playing item in the song and in our Search View, we want to be able to play songs directly from that page. As such, we’ll be implementing @State and @Binding protocols to help with data flow.

Head over to ContentView.swift. At the top of the file, type import MediaPlayer underneath import StoreKit. Next, add the following line where you declare your selection variable:

@State private var musicPlayer = MPMusicPlayerController.applicationMusicPlayer

We’re creating a variable called musicPlayer that is of the type applicationMusicPlayer as we discussed above. We’re also adding the @State property to it so we can use it to pass it to other views. Let’s do that right now!

Go to PlayerView.swift and at the top of the file, add import the MediaPlayer framework with:

import MediaPlayer

Next, at the top of thestruct, add the following line:

@Binding var musicPlayer: MPMusicPlayerController

This creates a Binding variable for our Player View. Since we will be binding this variable to the musicPlayer we created in ContentView.swift, anytime we make a change in Player View to this object, it will be reflected in Content View.

Since we can’t pass this binding variable to our PlayerView_Previews and SwiftUI previews have no support for MusicKit, there’s no point in having this in our file anymore. Delete the entire PlayerView_Previews structure from this file.

Let’s do the same thing in SearchView.swift. You should already have the MediaPlayer framework implemented. Just like before, add the following line to the top of your SearchView structure below the initialization of the searchResults array.

@Binding var musicPlayer: MPMusicPlayerController

Similar to PlayerView.swift, delete the entire SearchView_Previews structure from this file.

Now, head back to ContentView.swift. You should see some errors for PlayerView() and SearchView(). This is simply because we haven’t added the argument for the parameter musicPlayer when we call it. You can fix this by changing PlayerView() to PlayerView(musicPlayer: self.$musicPlayer) and SearchView() to SearchView(musicPlayer: self.$musicPlayer).

Build the app! It should compile without errors. We now have a built-in media player that is accessible to all the views in our app! Now when we implement the play and pause methods, we can easily update our player and reflect those changes across all the views in our app!

Playing Music

When our user clicks on a song from the search table, we’d like our app to play the song selected. This means that we have to add a song to our musicPlayer’s queue and tell the queue to play. Watch how we implement this in two line!

Go to SearchView.swift. Find the Button structure that should be inside the List structure. As of right now, it only prints the name of the song it is playing to the console. Delete that line and replace it with the following:

self.musicPlayer.setQueue(with: [song.id])
self.musicPlayer.play()

That’s it! In the first line, we set the queue of our musicPlayer. The queue in this case is an array of strings containing the ID’s of the song. This is why the MediaPlayer and Apple Music API work harmoniously together. The framework can automatically detect which song is to be played based on the ID given. The final line simply instructs our musicPlayer to play whatever is in the queue!

Build and run the app! You should be able to search for a song, and when you tap on it, your music player should start playing the song!

If you’ve got everything working, congratulations! This is one of the most under appreciated, unknown functionalities you can build using Swift! We’re not done yet though! While we can play music, we’d also like to control the playback state of the song! We also want to make sure that the Player View can show us the name of the song and tag artist’s name. This can be achieved relatively simply as well!

Go to PlayerView.swift. Let’s start at the top of the file and make the respective changes as we work our way down!

Find the VStack structure where we define our Text objects that will contain the name of the song and the artist’s name. Replace it with the following:

VStack(spacing: 8) {
    Text(self.musicPlayer.nowPlayingItem?.title ?? "Not Playing")
        .font(Font.system(.title).bold())
    Text(self.musicPlayer.nowPlayingItem?.artist ?? "")
        .font(.system(.headline))
}

The changes we’ve made here is to replace “Song Title” and “Artist Name“ with the actual name of the currently playing song and the artist by whom it’s sung. If no song is playing, then we leave the artist’s name empty and set the name of the song to ”Not Playing”.

Before we go further, I’d like to explain a little bit about this .nowPlayingItem object we’re calling. This object is of a type called MPMediaItem. This object is provided by the MediaPlayer framework and is an object which contains a collection of properties that represents what you could call a “song” in a media library. More specifically, an MPMediaItem can be any object that can be played and has a metadata that is similar to any audio-based object such as a song or podcast. The properties we’re using here are title, which is the name of the song, and artist, which is the name of the artist. There are many more properties that you can use if you choose to that can be found here.

Build and run your app. You can see that when you start to play a song, the metadata is displayed where it’s supposed to be!

Now, let’s add some play and pause functionality. Scroll down in PlayerView.swift and go to the Button object which currently prints ”Pause” to the console. Delete this line and replace it with the following:

if self.musicPlayer.playbackState == .paused || self.musicPlayer.playbackState == .stopped {
    self.musicPlayer.play()
} else {
    self.musicPlayer.pause()
}

Our musicPlayer has a property called playbackState that is of type MPMusicPlaybackState. This property keeps track of whether the music player is playing a song, paused, or stopped. If the playbackState is equal to.paused or .stopped, then nothing is playing and when the user clicks on this button, we’d like for it to start playing. This is why we call self.musicPlayer.play(). Similarly, in the else clause, we call self.musicPlayer.pause() because we’d like to stop the currently playing song.

You can build and run the app on your device. You should see that when you click on the pause button, the music pauses and when you press again, the music resumes. However, the icon of the button is not updating. This can be done by tracking the playback state of the currently playing item!

At the top of PlayerView.swift, underneath where you declared musicPlayer, type the following line:

@State private var isPlaying = false

This is a Boolean variable that is attached to the @State property that will be updated based on the playback state of the currently playing song. First, go to the Button object we modified above. Modify the action of the Button to look like the following:

if self.musicPlayer.playbackState == .paused || self.musicPlayer.playbackState == .stopped {
    self.musicPlayer.play()
    // 1
    self.isPlaying = true
} else {
    self.musicPlayer.pause()
    // 2
    self.isPlaying = false
}

We’ve added two lines that are pretty self explanatory. When we signal the musicPlayer to play, we set the isPlaying variable to true. When we pause the musicPlayer, we once again set the isPlaying variable to false.

Next, we need to use the isPlaying variable to update the image used as the logo for the button. This is quite simple. Underneath the button’s action where we declare the UI of the button, modify the ZStack object to look as such:

ZStack {
    Circle()
        .frame(width: 80, height: 80)
        .accentColor(.pink)
        .shadow(radius: 10)
    Image(systemName: self.isPlaying ? "pause.fill" : "play.fill")
        .foregroundColor(.white)
        .font(.system(.title))
}

The change we made here is to change the systemNameof the Image object used based on whether self.isPlaying is true or false. If it is true, we’ll use the pause.fill SF Symbol. If not, we’ll use the play.fill symbol.

Last but not least, we need a way to update self.isPlaying if there is a change made in Search View. For example, if our player is paused but we play a song from Search View, we’d like to update self.isPlaying by setting it to true. Now, we can use the @Binding protocol but I’d like to show you an easier alternative: the onAppear() function!

At the very bottom of PlayerView.swift, find the second to last } closing curly bracket. This is the bracket that will close our GeometryReader. Attach the .onAppear() function below it as shown:

.onAppear() {
    if self.musicPlayer.playbackState == .playing {
        self.isPlaying = true
    } else {
        self.isPlaying = false
    }
}

What are we doing in this function? Well, remember that our musicPlayer is already bound to all the views in our app. So any updates to this object is immediately reflected in every view. When we play a song from Search View, the musicPlayer is updated to set its playbackState to .playing. This is why in the .onAppear() function, we check to see that if the playbackState is in fact playing, then we set isPlaying to true. If not, we set isPlaying to false!

That’s all for playing and pausing! Build and run your app! You should see that the play/pause button accurately changes based on whether the song is currently playing or is paused!

Implementing Skip & Rewind

Now let’s implement the logic for skip and rewind buttons. This won’t take too long. Let’s start with the skip button.

Navigate to PlayerView.swift and locate the Button object that we’re using as a temporary skip button. It should be printing ”Skip” to the console when tapped upon.

Replace the line print("Skip") with self.musicPlayer.skipToNextItem() and that’s all! Build and run your app. When you press the skip button, it will jump to the next song in the queue, which is nothing, so the player will stop!

You can see how MediaPlayer comes with a list of functions that make it easy for us to control playback! With one line of code, we were able to implement a “skip song” functionality into our music player app. The “rewind” button is a little more challenging.

Think about your favorite music player. When you press the rewind button, what happens? If the song is more than 5 seconds through its playback, it skips to the beginning of the song. If the current playback is still within the first 5 seconds, then it goes to the previous song. With a simple if-else statement, watch how we can add the same functionality to our app.

Scroll back up to the Button object that currently prints ”Rewind” to the console. Delete that line and replace it with the following:

if self.musicPlayer.currentPlaybackTime < 5 {
    self.musicPlayer.skipToPreviousItem()
} else {
    self.musicPlayer.skipToBeginning()
}

You can see that we use a new property here: currentPlaybackTime. This is the length of the song, in seconds, that has been played by our player. If the number of seconds played so far is less than 5, we call upon a new method: skipToPreviousItem(). This method, provided by the MediaPlayer framework, we jump back in the queue and start playing the song before this song. Currently, this will continue playing the beginning of this song.

However, what we can test is that if currentPlaybackTime is greater than or equal to 5, we can call skipToBeginning() in our musicPlayer and that will ask our musicPlayer to start playing from the beginning of our current song.

Build and run your app! It should be able to handle the skip and rewind buttons as you implemented! As you can see from this section, a lot of the methods we use to control playback are already implemented in the MediaPlayer framework. With a little research, you can continue to build upon all these features.

Implementing Album Artwork

Finally, let’s replace the ugly “A” SF Symbol with the album artwork of the current song playing. If you remember from earlier, our Song object has a property called artworkURL that contains the string of a URL containing the image of the album.

Normally, we’d have to write a lot of code to process the URL, gather the image data, and cache the image so we wouldn’t make redundant calls. I’d like to show you a nifty Swift package called [SDWebImageSwiftUI][6]. This package contains a framework built for SwiftUI that can help with image loading and contains features like async image loading, memory/disk caching, animated image playback and more. If you ever need to load images from a URL in your apps, I highly recommend this framework since it automatically handles image loading and caching, thereby speeding up your app.

Here’s how we can install it. We’ll be using Swift Package Manager to link this framework with our project. In Xcode, go to the Title Bar, and click on File > Swift Packages > Add Package Dependancy.

A popup will show up and ask you to enter a package repository URL. The URL we will be using is https://github.com/SDWebImage/SDWebImageSwiftUI. Enter this URL and press Next.

After some loading, Xcode will ask you which version to use. Don’t make any changes and simple press Next. Xcode will take some time to fetch the repository and create a package to add to your project. After a minute, Xcode will show you a popup confirming you to choose the packages and target. Make sure it looks like something below.

The important point to follow here is that MusicPlayer is selected as the target. Click Finish and Xcode will take you to the main project page that shows all your Swift Packages. This means that the package has successfully been added to your project and you can start using the framework!

Go to SearchView.swift and at the top of the file, type import SDWebImageSwiftUI. This will gives us access to all the objects and methods in this framework.

In our List object, locate the Image object that is currently coded as such:

Image(systemName: "rectangle.stack.fill")
    .resizable()
    .frame(width: 40, height: 40)
    .cornerRadius(5)
    .shadow(radius: 2)

Making sure that all its modifiers are still in place, replace it with the following

WebImage(url: URL(string: song.artworkURL.replacingOccurrences(of: "{w}", with: "80").replacingOccurrences(of: "{h}", with: "80")))
    .resizable()
    .frame(width: 40, height: 40)
    .cornerRadius(5)
    .shadow(radius: 2)

SDWebImageSwiftUI provides an object called WebImage. This is very similar to our Image object but has some really neat additions. For one, it takes an argument called url which when provided, automatically loads the image from this URL.

We create a URL object and pass the string song.artworkURL. Notice that once again, we perform string manipulation here. If you notice the URL, you’ll notice that it has two parameters: {h} and {w}. These parameters stand for height and width, respectively. This is provided by the Apple Music API on purpose because we can specify the height and width that is best suited for us. I chose a height and width of 80, since this is 2x the size of our WebImage so the image loaded will have a high quality. Therefore, we call replaceOccurences on both {w} and {h} and replace it with 80.

Everything else should remain the same! Make sure your code looks like the screenshot below:

Build and run your app! Now, when you search for a song, you can actually see the album cover in the search results. The best part too is that SDWebImageSwiftUI caches the image so if you quit the app and search for the same term again, you can notice that there is a huge speed increase in loading the image.

Our last step is to change the album artwork in PlayerView.swift. Unfortunately, this is not as simple. See, in SearchView.swift we had a list of Song objects, each containing a URL to the album cover. Since we didn’t pass this list along, our Player View can’t access this album artwork URL. Furthermore, while MediaPlayer is great at providing many built-in objects and methods, it does not contain an object with a link to a particular song’s album artwork.

In order to fix this, we need to rely on State and Binding, once again. Head to ContentView.swift and at the top of the file, underneath where you created your musicPlayer variable, add the following file:

@State private var currentSong = Song(id: "", name: "", artistName: "", artworkURL: "")

We are creating a new shared object called currentSong of type Song that will be updated whenever we click on a new song from our Search View.

On that note, go to SearchView.swift and add the following Binding property at the top of the file, below where you declared your musicPlayer variable.

@Binding var currentSong: Song

Scroll to the bottom of the file and you’ll find the Button object that currently sets the queue of the musicPlayer and tells it to start playing. Over here, we want to set currentSong to the song that was tapped upon.

Modify the action of the Button as shown.

Button(action: {
    self.currentSong = song
    self.musicPlayer.setQueue(with: [song.id])
    self.musicPlayer.play()
})

All we’re adding here is setting the currentSong to the song that was tapped on.

Let’s do the same for PlayerView.swift. Navigate to that file and at the very top, add import SDWebImageSwiftUI. This will let us use the WebImage object as we did above.

Next, add the Binding variable currentSong to the structure, right below where we declared the isPlaying variable.

@Binding var currentSong: Song

Last but not least, let’s replace our current Image object which holds the SF Symbol a.square with the following code. Notice that all the modifiers remain the same.

WebImage(url: URL(string: self.currentSong.artworkURL.replacingOccurrences(of: "{w}", with: "\(Int(geometry.size.width - 24) * 2)").replacingOccurrences(of: "{h}", with: "\(Int(geometry.size.width - 24) * 2)")))
    .resizable()
    .frame(width: geometry.size.width - 24, height: geometry.size.width - 24)
    .cornerRadius(20)
    .shadow(radius: 10)

Now just like before, we pass a URL to WebImage, only this URL is very lengthy. Just like before, we are replacing {w} and {h} with the width and height we want. It’s not as simple as before however, since our height and width is reliant on the width of the device. This is why we replace {w} and {h} with geometry.size.width - 24. The number is actually of type Floatso to convert it to an integer, we wrap it around in Int. Finally, we multiply both lengths by 2 to get a sharper image quality.

Last but not least, we still need to pass the currentSong from our Content View to both the Player View and Search View. Head over to ContentView.swift and make the following changes to where you declare both PlayerView() and SearchView().

PlayerView(musicPlayer: self.$musicPlayer, currentSong: self.$currentSong)
...
SearchView(musicPlayer: self.$musicPlayer, currentSong: self.$currentSong)

In these changes, we passing the currentSong that was created in ContentView.swift to the remaining views.

Congratulations! This is the end of this lengthy tutorial! Build and run your app and watch with delight as your app can make calls to the Apple Music API, parse JSON and display a list of songs, utilize a devices media player capabilities to play songs and control playback, and harness external libraries to load images from remote sources- all using SwiftUI, a technology that is only a year old!

Conclusion

If you’ve reached the end successfully, congratulations! This two part tutorial was not your average, intermediate tutorial. Despite its lengthy content, I hope you learned something new from all the technologies we’ve utilized. I’ve compiled a list of resources below that can help you if you choose to continue building upon this app! You can download the completed project on GitHub.

For reference, you can check out the following resource related to this tutorial:

Apple Documentation and Videos

• Apple Music API Documentation
• StoreKit Documentation
• MediaPlayer Documentation
• MusicKit on Android and the Web
• WWDC 2017 Video- Introduction MusicKit
• WWDC 2018 Video- MusicKit on the Web

More on the Technology

• Introducing JSON
• Working with JSON and Codable in Swift 5
• More on @State and @Binding in SwiftUI

As always, if you have any doubts, feel free to leave a comment below or reach me on Twitter @HeySaiK. I can’t wait to see what you’ll make with MusicKit!

While I’ve introduced you quite a number of new features announced in WWDC 2020. Let’s step back a bit and check out a useful framework introduced in WWDC 2019. At first glance, it does not look as significant or important as other frameworks, however it consists of a really useful tool when it is needed. That is the LinkPresentation framework, and it provides a handful of built-in functionalities that makes presenting rich links in apps a really simple and straightforward process.

LinkPresentation framework contains mechanisms that parse the website behind a link and fetch metadata necessary in order to display the link in a visually formatted, beautiful manner. In particular, it fetches the title, icon, images, and video metadata, when any of those is provided by the website through the Open Graph protocol.

Fetched metadata can be saved locally for future use, so it’s possible to avoid downloading the exact same data repeatedly. Making the local storage of metadata (caching) a reality is assisted by the custom type that represents metadata programmatically; the LPLinkMetadata class, which is serializable, so it’s easy to create Data objects out of it that can be written locally.

But the above are not the only features available. LinkPresentation also handles the appearance of a visually formatted link by providing a custom view configured based on fetched metadata, and that view also supports user interaction through single tap and long press gestures.

If there’s just one negative point we can spot in the LinkPresentation framework, then that is the fact that it’s built for UIKit (and AppKit on macOS), so integrating it in SwiftUI projects requires some additional work. But no need to worry, as in this post we’ll do exactly this; we’ll build a SwiftUI project where we’ll embed and use the LinkPresentation framework!

LinkPresentation framework works in both iOS and macOS, starting from iOS 13 and macOS 10.15. Besides than fetching metadata for remote links, it’s also possible to deal with links pointing to local files, however that’s not something we’ll discuss in this tutorial. We’ll see all the main features that were presented shortly right above, and on top of that we’ll see how LinkPresentation framework can be combined with the activity view controller in order to share links easily, taking advantage of already cached metadata.

So, keep on reading to find out some cool features of the LinkPresentation framework, but before that, let’s meet the demo application we’re going to build in the next parts of this tutorial!

The Demo Application

Our journey to LinkPresentation framework will take place using a SwiftUI based iOS application. Parts of it have already been built and you can download it here as a starter project. Our work in this tutorial will begin by keep building on that starter project, so once you get it open it in Xcode.

In the demo application we’ll go through all features of the LinkPresentation framework that one would need to incorporate in their own apps. The first thing we’ll meet is how to fetch remote link metadata using the LPMetadataProvider class, and this will happen after having typed or pasted a URL in the InputLinkView view of the starter project.

Next, we’ll see how to display a rich link after having fetched its metadata, and we’ll get to know the LPLinkView class; the one that’s responsible for presenting a link visually formatted. We’ll display our first link view in the InputLinkView as the result of the link metadata fetching. In this part we’ll have the opportunity to see how to bring a UIKit view to SwiftUI with the UIViewRepresentable protocol.

After that, we’ll add the capability to save downloaded link metadata as part of a collection of links, and then load them. We’ll talk about serialization, the NSSecureCoding protocol, and we’ll archive and unarchive using the NSKeyedArchiver and NSKeyedUnarchiver classes respectively.

By making our app capable of caching link metadata locally and loading back we’ll move on and we’ll display all stored links in a list. This will happen in the LinksListView view, and once again, we’ll make use of the link view.

Finally, we’ll disable the default interactions with the link view and we’ll implement our own activity view controller for sharing links in order to feed it with cached metadata; we’ll prevent it that way from fetching that metadata again, which is its default behaviour. We’ll meet a specific method of the UIActivityItemSource protocol that actually provides link metadata to activity view controller, and besides that some other intersting stuff, such as how to use a UIKit view controller in SwiftUI with the UIViewControllerRepresentable protocol.

In addition to all the above we’ll make use of two additional custom types: The Link class which will represent a link programmatically, and the LinksModel which already contains a collection of Link objects and it consists of the place where we’ll add all the implementation about fetching link metadata, saving and loading.

Navigate through the current implementation of the starter project, and when you’re done, prepare to fetch link data for first time!

Getting Started: Fetching Link Data

By making clear what we’re going to talk about in this post and what our course of actions is going to be, it’s time to start building on the project and see the various aspects of the LinkPresentation framework.

Our starting point cannot be other than fetching the metadata for a link that is provided in the InputLinkView view. We’ll be triggering the actual link data fetching when the Return key is tapped on the keyboard in order to be dismissed.

First, let’s focus on the mechanism responsible for getting the link metadata. Open the LinksModel.swift file where we’ll add the following class method in the LinksModel class:

class LinksModel {
    class func fetchMetadata(for link: String, completion: @escaping (Result<LPLinkMetadata, Error>) -> Void) {

    }
}

The first parameter value is the URL of the target link as a String value; the text given to the textfield in the InputLinkView.

Since fetching data from Internet is an asynchronous process, the second parameter is a completion handler, or in other words a callback function that will be called when the download is finished. The completion handler’s argument is a Result type that will contain either the actual fetched data, or an error object in case the operation fails.

Note: Read more about the Result type here.

In case of success, the data of the Result type above is going to be a LPLinkMetadata object. LPLinkMetadata class belongs to LinkPresentation framework, and it contains properties that describe a link:

• URL
• Title
• Icon
• Image
• Video

We’re going to use the returned LPLinkMetadata instance to show the rich link preview later, as well as to store fetched metadata locally.

Before we continue, add the following import statement right before the opening of the LinksModel class so we import the LinkPresentation framework and eliminate the error that is currently displayed in Xcode:

import LinkPresentation

We’ll begin implementing the above method by creating a URL object based on the provided URL string:

guard let url = URL(string: link) else { return }

Now, we’re going to meet another class of the LinkPresentation framework called LPMetadataProvider. This is the one that will perform the actual fetching of the link’s metadata and it will return either a LPLinkMetadata object, or an error object if something turns out bad.

Using it is very simple. First we have to initialize an object of it:

let metadataProvider = LPMetadataProvider()

Then, we need to call the method that will initiate the asynchronous fetching operation:

metadataProvider.startFetchingMetadata(for: url) { (metadata, error) in

}

The completion handler of the method we just called contains two parameters: The link’s metadata and an error object . We’ll check if any of them is nil or not, and we’ll pass the respective unwrapped value to the Result type that we’ll feed to the completion handler of our method.

metadataProvider.startFetchingMetadata(for: url) { (metadata, error) in
    if let error = error {
        completion(.failure(error))
        return
    }

    if let metadata = metadata {
        completion(.success(metadata))
    }
}

If error is not nil, then we pass the .failure case of the Result type as an argument to the completion handler with the error object as an associated value. Otherwise, we check if metadata is not nil and we pass the .success case of the Result type to the completion handler along with the unwrapped metadata object.

Here is the method we just implemented in one piece:

class func fetchMetadata(for link: String, completion: @escaping (Result<LPLinkMetadata, Error>) -> Void) {
    guard let url = URL(string: link) else { return }

    let metadataProvider = LPMetadataProvider()
    metadataProvider.startFetchingMetadata(for: url) { (metadata, error) in
        if let error = error {
            completion(.failure(error))
            return
        }

        if let metadata = metadata {
            completion(.success(metadata))
        }
    }
}

Let’s open now the InputLinkView.swift file, where we’ll make use of the above method. Go to the textfield’s implementation into the view’s body and spot the comment: // Fetch link metadata..

The place where you find that comment is the action handler that gets called when the Return key in the keyboard is tapped while editing the textfield’s text. Replace the comment with the following code:

LinksModel.fetchMetadata(for: self.link) { (result) in

}

We’re going to handle the above method’s result right next. What really matters at this point is that the first important step has been now done! With just a few bits of code the demo application is capable of fetching the metadata for any given link!

Handling Fetched Link Metadata

We will handle the results of the fetchMetadata(for:completion:) in a new method that we’ll implement here. But before that, go to the beginning of the InputLinkView structure and above the body implementation in order to add the following property which will hold the fetched metadata:

@State private var metadata: LPLinkMetadata?

Additionally, import the LinkPresentation framework at the beginning of the file:

import LinkPresentation

With the metadata property handy now, let’s move on to the definition of a new method. Still inside the InputLinkView structure and right after the body implementation add the following:

private func handleLinkFetchResult(_ result: Result<LPLinkMetadata, Error>) {

}

This method will accept the Result value coming from the completion handler of the fetchMetadata(for:completion:) and will handle both the metadata, if exists, and the error.

In case where link’s metadata has been fetched successfully, then we’ll get it from the associated value of the success case of the Result type and we’ll assign it to the property we declared right before. In case of error, then we’ll simply print it on the console; that’s a fast solution good enough for this tutorial, however in real applications more appropriate actions should be taken, such as informing the user with an alert, or anything else suitable for the application.

All the above will take place in a switch statement, since it consists of the easiest way to extract associated values from a Result value. In addition to that, don’t forget that metadata fetching is an asynchronous process and that our actions here will affect the user interface (UI), therefore it is mandatory everything to be done on the main thread.

Here’s the full implementation of the handleLinkFetchResult(_:) method:

private func handleLinkFetchResult(_ result: Result<LPLinkMetadata, Error>) {
    DispatchQueue.main.async {
        switch result {
            case .success(let metadata): self.metadata = metadata
            case .failure(let error): print(error.localizedDescription)
        }
    }
}

A couple of more tweaks in the original code of the starter project are necessary and we’ll be good to go. First find the comment saying: // Clear previous metadata. in the view’s body implementation.

The closure that this comment is written into is the textfield’s handler that is being called when the textfield gets edited. For simplicity and in order to avoid small collateral complications that we don’t need in this tutorial, we’ll be scratching any metadata previously fetched whenever users edit the link’s URL. So, replace that comment with:

if self.metadata != nil {
    self.metadata = nil
}

After doing that, move a bit down in the existing code where you will find the if false condition right below a comment saying: // Link preview.

This condition has no meaning and it works as a temporary placeholder for the actual condition that we’ll specify right now. Replace if false with this:

if metadata != nil {
    ...
}

If link’s metadata exists and the metadata property we declared earlier is not nil, then we’ll display the link preview (coming next). Otherwise, a placeholder text is displayed instead. We’ll return at this specific point in a while and we’ll replace the EmptyView() with an actual view that displays the link preview.

Finally, update the fetchMetadata(for:completion:) method call and in the completion closure call the handleLinkFetchResult(_:) method:

LinksModel.fetchMetadata(for: self.link) { (result) in
    self.handleLinkFetchResult(result)
}

The Link View

One of the greatest things in LinkPresentation framework is that it provides a view that displays the link preview formatted properly, as long as it’s fed with the link’s metadata as a LPLinkMetadata object. On top of that, that view allows interaction when single tapping and long pressing on it; when the first happens it opens the selected link in the browser. In the second case, it shows a list of options including copying the link’s URL, sharing, and more. However, that behaviour is not always desirable and you’ll find out the reason later in this post.

The responsible class for creating rich link representations is called LPLinkView. Even though it offers important functionality out of the box, it has a downside when working in SwiftUI; it is a UIView view subclass (a UIKit view) and not a SwiftUI view! So, in order to use it we must mix UIKit and SwiftUI.

Thankfully, it’s quite easy to wrap UIKit views in SwiftUI using the UIViewRepresentable protocol. The plan is to create a structure that conforms to UIViewRepresentable protocol, to implement a couple of required methods, and inside those methods to initialize and configure the UIView object.

To get started, open the LinkView.swift file, and right after the current import statement add the next two as well:

import UIKit
import LinkPresentation

Now, let’s create a new structure which will be used like any other SwiftUI view when we finish with it. We’ll call it LinkView:

struct LinkView: UIViewRepresentable {

}

UIViewRepresentable protocol has a required associated type called UIViewType and it indicates the type of view that the structure will present. Here is how we fulfil that requirement in our case:

typealias UIViewType = LPLinkView

After that, it’s necessary to define the next two methods:

func makeUIView(context: Context) -> LPLinkView {

}

func updateUIView(_ uiView: LPLinkView, context: Context) {

}

makeUIView(context:) must return an initialized view of the type specified in the UIViewType value. Here we are going to return a LPLinkView object.

The updateUIView(_:context:) method is not required to be mandatorily used, but it has to be defined. Use it when the view content must be updated when new information from SwiftUI exists, or the view cannot be fully configured in the makeUIView(context:) only.

A LPLinkView needs link metadata in order to display content. Therefore, add the next property in the LinkView structure; we’ll provide metadata upon initialization of LinkView instances:

var metadata: LPLinkMetadata?

Time to add content to makeUIView(context:) method. First, we’ll make sure that the metadata property isn’t nil. Then, we’ll initialize a new LPLinkView view using the metadata, and we’ll return it from the method:

func makeUIView(context: Context) -> LPLinkView {
    guard let metadata = metadata else { return LPLinkView() }
    let linkView = LPLinkView(metadata: metadata)
    return linkView
}

That’s all we need! Here’s the entire implementation of the LinkView structure:

struct LinkView: UIViewRepresentable {

    typealias UIViewType = LPLinkView

    var metadata: LPLinkMetadata?

    func makeUIView(context: Context) -> LPLinkView {
        guard let metadata = metadata else { return LPLinkView() }
        let linkView = LPLinkView(metadata: metadata)
        return linkView
    }

    func updateUIView(_ uiView: LPLinkView, context: Context) {

    }
}

The above makes it possible to use an UIView object in SwiftUI like any other view, and you’re going to see that right next!

Previewing Rich Links

Back in the InputLinkView.swift file now where we’ll make the first use of the LinkView structure we just implemented. Find the following condition:

if metadata != nil {
    EmptyView()
}

We’re going to replace the EmptyView() with a LinkView instance. Here it is:

if metadata != nil {
    LinkView(metadata: metadata)
        .aspectRatio(contentMode: .fit)
} else {
    ...
}

Appending the aspectRatio modifier is done just for displaying properly the content of the link view, but it’s not a requirement; we could have omitted it.

With the above condition now fixed it’s possible to show a rich link preview when its metadata has been fetched, or a placeholder text when there’s no metadata or the textfield is being edited.

We can run the app for first time at this point and try out what we’ve managed so far! Click on the Plus button to add a link, type or paste any link in the textfield and tap on Return key (or hit Return on your keyboard if you’re using Simulator). Wait for a few moments and the rich link will show up!

The Custom Link Type

If a link’s metadata is not fetched for one-time use only, then the best practice that Apple recommends too is to save it locally (cache it) and load it when it’s about to be needed again. That way users will have a better experience since they won’t have to wait again and again until fetch is complete, and if they get connected through a cellular network they’ll avoid having unnecessary costs in their mobile data plans for fetching link metadata that they have already fetched in the past.

The good news for us is that metadata represented by a LPLinkMetadata instance is serializable, which in plain words means that we can archive and unarchive using the NSKeyedArchiver and NSKeyedUnarchiver classes respectively.

However, we won’t save the metadata alone straight to the disk in this post, and here is the reason:

Our next big goal is to display all stored links in a list, and in order to do that properly we need to have a type that conforms to Identifiable protocol and contains an id property.

LPLinkMetadata class does not satisfy that requirement, so we are going to use a custom type that will contain an id property along with the metadata, and it will adopt the Identifiable protocol so it can be used in SwiftUI lists.

That custom type has been defined already and it’s the Link class in the Link.swift file. Open it and add the following two properties:

var id: Int?
var metadata: LPLinkMetadata?

Since we’re making use of the LinkPresentation API, we need to import that framework in this file too:

import LinkPresentation

Additionally, adopt the Identifiable protocol to make it possible to list Link items later:

class Link: Identifiable { ... }

Now, as I said already before, we won’t archive and save the metadata only. Instead, we’ll archive and save the entire instance of a Link object so it includes both the id and the metadata. But in order to do that it’s necessary to make Link class:

• A subclass of the NSObject class.
• Conform to the NSSecureCoding protocol (which in turn adopts NSCoding). This will enable us to serialize and deserialize instances of Link (and eventually save to disk) and take advantage of the fact that LPLinkMetadata is serializable.

Note: We’ll make Link class conform to NSSecureCoding instead of NSCoding because metadata, a LPLinkMetadata object, is serializable with the NSSecureCoding protocol. Read more about NSCoding and NSSecureCoding, or make a search on the web for additional information about them.

So, we’ll continue by updating the class header line as shown:

class Link: NSObject, NSSecureCoding, Identifiable { ... }

NSSecureCoding protocol brings along new requirements: We must include a required associated value to support secure coding, and to implement two methods; one for encoding stored properties and one to initialize instances of self from archived objects:

static var supportsSecureCoding = true

func encode(with coder: NSCoder) {

}

required init?(coder: NSCoder) {

}

Both of the above methods must be implemented by any custom type that needs to become capable of being serialized and deserialized.

Starting with the encode(with:) method, here’s how we’re encoding the two properties of the Link class so they become serializable:

func encode(with coder: NSCoder) {
    guard let id = id, let metadata = metadata else { return }
    coder.encode(NSNumber(integerLiteral: id), forKey: "id")
    coder.encode(metadata as NSObject, forKey: "metadata")
}

You might find peculiar that we’re encoding properties as Objective-C objects (NSNumber, NSObject) instead of encoding as Swift Int value and Any object respectively. Actually, that’s because of a requirement that results from the way decoding must be done when adopting NSSecureCoding protocol.

To be more precise, you will see next that in order to decode in the init(coder:) we’re making use of the decodeObject(of:forKey:) method as Apple recommends. The first argument must be a subclass of the NSObject class, and that creates the need to encode accordingly. Also, given what I just said and after you’ll see the implementation of the init(coder:) method right next you can understand why we made Link class a NSObject subclass. I leave it up to you to read more information about NSSecureCoding.

Going to the init(coder:) initializer now, here we have to decode previously encoded values and objects and assign them to the proper properties.

required init?(coder: NSCoder) {
    id = coder.decodeObject(of: NSNumber.self, forKey: "id")?.intValue
    metadata = coder.decodeObject(of: LPLinkMetadata.self, forKey: "metadata")
}

Before we move on to the next step, add the next default initializer to Link class that will allow us to create new objects:

override init() {
    super.init()
}

The implementation of the Link class is complete. You’ll see why the above are necessary in the following parts.

Saving Link Data

Switch to the LinksModel.swift file now where the actual saving of Link objects is going to take place. However, before we get to that, let’s start by creating a method in the LinksModel class that will create new Link objects using metadata that is provided as an argument:

func createLink(with metadata: LPLinkMetadata) {
    let link = Link()
    link.id = Int(Date.timeIntervalSinceReferenceDate)
    link.metadata = metadata
    links.append(link)
    saveLinks()
}

Here’s what is going on in the above method:

• First a new Link object is being created.
• As a quick solution for the value of id property we’re getting the current timestamp and we keep only its integer part. We assign it to the id property as the identifier of the link object.
• We assign the metadata parameter value to the respective property of the link object.
• We append the newly created object to the links collection (already defined in the LinksModel class).
• Finally we call the saveLinks() method that we’re going to implement as the next step in the whole process we’re doing here.

So, time to work on the actual saving of the link data to the disk. Using the NSKeyedArchiver class we’ll serialize the entire links array, but note that this is possible because its items are serializable (we took care of that in the previous part). Doing so isn’t difficult at all, and it’s recommended to be done in a do-catch statement since an exception might be thrown. Here it is:

fileprivate func saveLinks() {        
    do {
        let data = try NSKeyedArchiver.archivedData(withRootObject: links, requiringSecureCoding: true)
        guard let docDirURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first else { return }
        try data.write(to: docDirURL.appendingPathComponent("links"))
    } catch {
        print(error.localizedDescription)
    }
}

Note: To be able to find and see the created file in Finder just add the next line as the last one in the do statement above:

print(docDirURL.appendingPathComponent("links"))

When the NSKeyedArchiver.archivedData(withRootObject: links, requiringSecureCoding: true) code is executed, the encode(with:) encoder method we met before gets called for every Link object in the links array so each one of them to be serialized. The resulting archived object is assigned to the data object as shown above. It’s important to stress here that the above wouldn’t work if we hadn’t made Link class serializable by conforming to NSSecureCoding protocol!

Note #2: Read more about NSKeyedArchiver and NSKeyedUnarchiver here and here respectively.

In case you wanted to serialize the metadata only and avoid all the hassle we went through in the Link class, then you should provide a LPLinkMetadata object as the first argument above instead of links, and probably to choose a specific name for the file that will contain the serialized metadata. Here’s an example method that does that:

func save(metadata: LPLinkMetadata) {        
    do {
        let data = try NSKeyedArchiver.archivedData(withRootObject: metadata, requiringSecureCoding: true)
        guard let docDirURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first else { return }
        try data.write(to: docDirURL.appendingPathComponent("SOME_BETTER_FILE_NAME"))
    } catch {
        print(error.localizedDescription)
    }
}

And by having said that, there’s one last action remaining to be done in order to save fetched link metadata to disk. We must call the createLink(with:) method from the InputLinkView structure.

Open the InputLinkView.swift file and find the comment saying:

// Save link metadata.

This closure gets called when the top-right button with the checkmark is tapped in the app. The goal is to save the link metadata and dismiss the modal view.

With that in mind, replace the above comment with the next simple call:

self.keepLink()

And now let’s implement that keepLink() method which is fairly simple right after the body implementation and inside the InputLinkView structure:

private func keepLink() {
    guard let metadata = self.metadata else { return }

    // Create the Link object and save all links metadata to file.
    self.linksList.createLink(with: metadata)

    // Dismiss InputLinkView instance.
    self.presentationMode.wrappedValue.dismiss()
}

Run the app again now and after having fetched a link’s metadata tap on the top-right button. Link should be saved and the view to be dismissed.

Loading Link Data

Back to LinksModel.swift file and in the LinksModel class, where we’ll add a new method responsible for loading and deserializing previously serialized and stored Link objects. In this method we’ll follow the opposite path from the one we followed earlier; first we’ll load the file contents to a Data object (if the file exists), and then we’ll use the NSKeyedUnarchiver class to unarchive links data.

Here it is:

fileprivate func loadLinks() {
    guard let docDirURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask).first else { return }
    let linksURL = docDirURL.appendingPathComponent("links")

    if FileManager.default.fileExists(atPath: linksURL.path) {
        do {
            let data = try Data(contentsOf: linksURL)
            guard let unarchived = try NSKeyedUnarchiver.unarchiveTopLevelObjectWithData(data) as? [Link] else { return }
            links = unarchived            
        } catch {
            print(error.localizedDescription)
        }
    }
}

See that links property gets a value if only code execution mades it successfully up to that point. This means that the file must exist, a Data object to be properly initialized using the the file contents, and that data to be unarchived without problems.

We want links to be loaded when the app starts running so we can display them. For that reason, the above method must be called when LinksModel class is initialized. Add the following init() method that will be called upon LinksModel initialization:

init() {
    loadLinks()
}

We’re now in the position where we can update the UI and display a list of links.

Listing Cached Links

Open the LinksListView.swift file now, where LinksListView is the place where we’ll list loaded links. At the beginning of the body implementation you’ll find a Text and an empty view inside a vertical stack. We’ll deal with the empty view later, but now we’ll replace the Text control with a List.

To do so, make sure you select and delete this:

Text("Links will appear here")

Next, add the following List in place of Text:

List(linksList.links, rowContent: { link in

})

Listing Link objects the way shown above is the reason why we adopted the Identifiable protocol in the Link class and we added the id property. Inside the List closure now we’ll create a button, so each item to be tappable:

Button(action: {

}) {

}
    .padding(.vertical, 20)

The first closure in the Button is being called every time the button is tapped. Let’s leave it empty for now, and let’s focus on the second closure that regards what the button is going to display as a content.

That content is nothing else but a LinkView that will be using the metadata of each link object in the list:

LinkView(metadata: link.metadata)

Putting everything together, here’s the list along with the button:

List(linksList.links, rowContent: { link in
    Button(action: {

    }) {
        LinkView(metadata: link.metadata)
    }
        .padding(.vertical, 20)
})

Run the app now to see rich links being listed. If you have saved already some links they’ll appear as soon as you launch the app. In any case, add a few more links in the InputLinkView and come back to see them being added to the links list.

Sharing Links With The Activity Controller

If you’ve played a bit with the rich link previews either in the InputLinkView or in the LinksListView then you’ve found out already that by single tapping on a link causes it to open in the browser. If you long press a link, then a preview of the target website along with various options for opening, copying, or sharing the link also appear.

Let’s focus on the sharing feature that shows up after performing a long tap to the link preview. An activity view controller appears with various sharing options available to the device, and we get a nice functionality for free, but there’s a disadvantage. The displayed link details are being fetched again and again from the target URL whenever the activity controller is being presented.

Obviously that’s not a desirable thing to happen, especially when we’ve already fetched and cached link metadata. So the question that arises here is: Can we use the metadata that we have already fetched in order to provide them to the activity view controller and to avoid fetching repeatedly the same data?

The answer is yes, and actually Apple recommends to do so! However we can’t do that in the activity view controller presented from the default link view. We must initialize and present our own activity view controller, and in that case it’s possible to feed it with any data we want to.

That sounds great, but the fact that we’re working in SwiftUI makes things a bit more complicated for us! Activity view controller is a UIKit view controller, so we must “port” it in SwiftUI. Don’t worry though! It’s a quite straightforward process pretty similar to the way we imported LPLinkView previously using the UIViewRepresentable protocol.

What we’re going to do here is a two-step process: First, we’ll create a UIKit view controller that will initialize a new activity view controller and it will adopt the UIActivityItemSource protocol. That will allow to implement specific methods through which we’ll provide link metadata to the activity controller.

Then, we’ll create a new custom structure which we’ll call ShareLinkView. It will conform to UIViewControllerRepresentable protocol; this will make it possible to “port” the UIKit view controller to SwiftUI and use it like any other native View object.

The ActivityController View Controller

With all the above said, let’s keep building on our app. Open the ShareLinkView.swift file, and start by importing the following two frameworks:

import UIKit
import LinkPresentation

Next, define the following class:

class ActivityController: UIViewController, UIActivityItemSource {

}

We create the ActivityController type, which is a view controller and adopts the UIActivityItemSource protocol. That protocol has a couple of required methods that need to be implemented, plus one more that we’ll add specifically for returning link metadata. Before we get there however, it’s necessary to make some other preparations.

So, at first declare the following properties in the ActivityController class:

var metadata: LPLinkMetadata?
var activityViewController: UIActivityViewController?
var completion: UIActivityViewController.CompletionWithItemsHandler?

• metadata is the link’s metadata that will be displayed on the activity controller as a LPLinkMetadata object.
• activityViewController is the actual activity view controller that will be presented.
• completion is a completion handler that will be called by the activity view controller when it gets dismissed.

Next, override the viewDidAppear(_:) that will be called when our view controller has appeared. In it, we’ll initialize the activity controller:

override func viewDidAppear(_ animated: Bool) {
    super.viewDidAppear(animated)

    activityViewController = UIActivityViewController(activityItems: [self], applicationActivities: nil)
    activityViewController?.completionWithItemsHandler = completion
    present(activityViewController!, animated: true, completion: nil)
}

Two important things to notice here:

1. An array of self items is given as the activity items at the controller’s initialization. This is possible because of the UIActivityItemSource protocol and the methods we’ll implement next.
2. The completion property is assigned to the completionWithItemsHandler property of the activity view controller. When it’ll be dismissed, that completion handler will be called and let the ShareLinkView that we’ll implement next know about that event.

Finally, time for the UIActivityItemSource methods. The two of them that are required are these:

func activityViewControllerPlaceholderItem(_ activityViewController: UIActivityViewController) -> Any {
    return ""
}

func activityViewController(_ activityViewController: UIActivityViewController, itemForActivityType activityType: UIActivity.ActivityType?) -> Any? {
    return metadata?.originalURL
}

In the first method we can return anything we want as placeholder item before an actual item becomes available. It doesn’t really matter here because the link metadata that we’ll provide will appear instantly, so simply returning an empty string value is enough.

The second method returns the actual data that the activity controller should act upon. Since we’re talking about links, this is the original URL of the link that will be used by any activity chosen in the activity controller.

Lastly, there’s one more method specific for the LinkPresentation framework; it feeds the activity controller with the link metadata object:

func activityViewControllerLinkMetadata(_ activityViewController: UIActivityViewController) -> LPLinkMetadata? {
    return metadata
}

The ActivityController class is ready, so let’s go straight ahead to use it.

The ShareLinkView

Previously in this post we created the LinkView structure, a custom type that was adopting the UIViewRepresentable protocol in order to make it possible to use a UIView object in SwiftUI.

In a quite similar fashion we’re going to create a new custom type to use in SwiftUI called ShareLinkView, but this time instead of adopting the UIViewRepresentable protocol, it will adopt the UIViewControllerRepresentable since it’s a view controller what we need to present.

Right after the closing of the ActivityController class add this:

struct ShareLinkView: UIViewControllerRepresentable {

}

Once again, it’s necessary to specify the view controller type that we’ll return to an associated value of the UIViewControllerRepresentable protocol, and implement two methods; one that will return the actual view controller and one that gets called in order to update the view controller when there are changes in SwiftUI environment.

typealias UIViewControllerType = ActivityController

func makeUIViewController(context: Context) -> ActivityController { }

func updateUIViewController(_ uiViewController: ActivityController, context: Context) { }

See that the above match to what we already met in the LinkView:

typealias UIViewType = LPLinkView

func makeUIView(context: Context) -> LPLinkView { }

func updateUIView(_ uiView: LPLinkView, context: Context) { }

Besides the above, we need to declare the following two properties in the ShareLinkView struct:

var metadata: LPLinkMetadata?
var completion: (() -> Void)

metadata is the metadata object that will be passed to the activity view controller. completion is another completion handler that will notify SwiftUI that the activity view controller has been dismissed.

In the makeUIViewController(context:) method we’ll create a new ActivityController instance and we’ll pass the metadata object:

func makeUIViewController(context: Context) -> ActivityController {
    let activityController = ActivityController()
    activityController.metadata = metadata
}

Then, we’ll deal with the completion handler of the activity controller; we’ll just call the completion property of the ShareLinkView structure to communicate to SwiftUI that the controller has been dismissed:

func makeUIViewController(context: Context) -> ActivityController {
    ...

    activityController.completion = { (activityType, completed, returnedItems, error) in
        self.completion()
    }
}

The completion’s arguments shown above are there just for demonstration; we’re not using them here. See Quick Help in Xcode for details what each parameter value is for.

Since the activityController instance we’re creating here is not loaded from any XIB or storyboard file, it’s necessary to call the loadView() method of the UIViewController class; this will trigger the user interface to be presented and the viewDidAppear(_:) we implemented earlier to be called in order to create the activity view controller:

func makeUIViewController(context: Context) -> ActivityController {
    ...

    activityController.loadView()
}

Finally, we must return the activityController instance:

func makeUIViewController(context: Context) -> ActivityController {
    ...

    return activityController
}

Regarding the updateUIViewController(_:context:) method, we’ll leave it empty; there’s nothing we really need to do with it.

Here’s the ShareLinkView structure complete:

struct ShareLinkView: UIViewControllerRepresentable {
    typealias UIViewControllerType = ActivityController

    var metadata: LPLinkMetadata?
    var completion: (() -> Void)

    func makeUIViewController(context: Context) -> ActivityController {
        let activityController = ActivityController()
        activityController.metadata = metadata
        activityController.completion = { (activityType, completed, returnedItems, error) in
            self.completion()
        }
        activityController.loadView()
        return activityController
    }

    func updateUIViewController(_ uiViewController: ActivityController, context: Context) {

    }
}

Using The ShareLinkView

Now that all the preparation has been done, we can go ahead and present the activity view controller passing the existing metadata of a link. Open the LinksListView.swift file, and declare the following property in the LinksListView structure that will keep the Link object matching to any selected link:

@State private var linkToShare: Link?

Now, in the action handler closure of the button inside the list control, add the following:

Button(action: {
    // Add this...
    self.linkToShare = link
}) {
    ...
}

It’s finally time to show the ShareLinkView which will actually present the ActivityController which in turn will present the activity view controller.

We’ll be presenting a new instance of the ShareLinkView every time there’s a link to share, or in other words the linkToShare property is not nil. Inside the body implementation of the LinksListView, find the EmptyView() declaration right after the list and delete it. Add this instead:

if linkToShare != nil {
    ShareLinkView(metadata: linkToShare!.metadata, completion: {
        self.linkToShare = nil
    })
        .frame(width: 0, height: 0)
}

By tapping on any item in the list the state of the linkToShare is changed and SwiftUI updates the UI because of that. While initializing a new instance of the ShareLinkView we pass the metadata of the selected Link object, and in the completion handler’s closure we make linkToShare nil again; this will force SwiftUI to hide the ShareLinkView view.

Also notice that we specify a zero frame using the frame modifier. This is done in order to avoid displaying the empty view of our custom ActivityController view controller; we just need the activity view controller to appear. Feel free to comment it out and see the effect of that modifier.

Run the app now and tap on any item in the list, but not on a link view itself. You’ll see that the activity view controller is presented properly and there is no waiting time until link metadata to be displayed on it.

If you tap or long press on the link view, you’ll see that the previous default behaviour is still there. There will be times where that behaviour will be meaningful and useful, and times where we won’t actually need it. This is one of them; we want to present the activity view controller when tapping on a list item and avoid the confusion when tapping on a link view.

So, let’s disable it, and in order to do that add the following modifier right after the initialization of the LinkView object:

LinkView(metadata: link.metadata)
    .disabled(true)

Note that it’s relatively easy to implement the rest of the missing functionalities in case you need to disable link view’s interaction in your apps, such as copying or opening the URL of the link.

Conclusion

In my opinion, LinkPresentation framework is something that was missing for years from the iOS and macOS SDK. It makes it super easy and fast to present rich links inside apps and actually in a way that’s quite familiar to users.

If you’re working with UIKit (or AppKit on macOS), it might be a bit more straightforward to deal with what LinkPresentation framework provides. However, as you’ve seen in this tutorial, it’s not difficult at all to use it with SwiftUI as well, and on top of that we implemented solutions that bring UIKit parts to SwiftUI environment. Just see how we implemented the LinkView and ShareLinkView structures.

In overall, working with LinkPresentation framework does not hide any hard task, and on the contrary, it’s a pleasure implementing around it. I hope you found this post useful. Thanks for reading!

For the full demo project, please download it on GitHub.

One of the most common tasks that iOS -and not only- developers are called to perform in their programming endeavours is fetching and managing remote images that should be displayed to an app. For instance, suppose that you’re building the next great messaging application where obviously users have contacts. Those contacts have avatar images residing to a server. When users connect to the app then avatars must be downloaded in order to be used by the app. Or, another example, think of an app that manages the event of a worldwide convention, and among its features is the list of all attendees. Their images must be fetched and handled by the app, and displayed properly when necessary.

What makes things complicated is the fact that image fetching is an asynchronous process. When fetching multiple images then it’s really easy to display the wrong image to the wrong place in certain cases. Think of the contact list again, which is a list of images with text at its simplest form. Displaying fetched avatar images to the proper contacts can end up being a hell depending on how image handling has been implemented. It would be really bad to see the app displaying the wrong avatar next to each name, and it would be even worse to have avatar images being replaced by other avatar images when contacts appear and disappear while scrolling. Have you seen this maybe? A perfect chaos and no consistency at all!

Another troubling point can be the way images are handled locally once they are fetched. Are there any rules about naming or storing details? Are there best and worst practices one should follow? Should images be stored in documents or in caches directory? These and more questions will most probably arise when building an app and it’s about time to deal with remote images. The answers though should not be given and implemented when an actual application is being built; that kind of issues should be already fixed and be part of the solution, not the problem.

So, here we come. In this tutorial I will present you a simple, yet quite effective recipe that makes fetching remote images and then handling them locally a really straightforward task. I’ll show you how to create a handy tool that you’ll be able to use right when it’s needed, and you won’t need to worry anymore about all that I described above. Even if you won’t use what we’ll build here eventually, you might get some ideas to apply in your own solutions. So, keep on reading to find out some really interesting stuff.

What We’re Going To Build

The main “component” we’re going to implement in this post is a protocol along with a default implementation that will make possible to fetch an image from a remote source, or load it from a local file. Creating a protocol makes it super-easy to integrate image handling capabilities on any custom type simply by adopting it, so in plain words our soon-to-be-implemented solution can be used out of the box everywhere. However, what’s the point to get into that trouble if we’re just to create a method that will fetch a single image?

The truth is that we won’t stop there; we’ll add more capabilities and we’ll support multiple image fetching, as well as single and batch image deletion. As a bonus point, we’ll add the capability to save a new image locally straight from an app, even though it doesn’t sound like a much related thing to do. It would be a shame though to have a component that fetches, loads and deletes images, but it does not support saving new ones and just for that operation to need a different tool or implementation.

Except for the protocol that will be the primary and most important custom type, there will be two additional custom types, two structures:

1. The first one will be used to specify optionally some settings. It’s necessary so the solution we’ll build to be as much flexible as possible, and you’ll understand pretty soon what I mean by that.
2. The second structure will be working behind the scenes and it will contain a small collection of static methods and properties valuable for the default implementation of the protocol’s methods.

We’ll build everything step by step and by discussing each aspect thoroughly. In order to see the results of our efforts though we’ll use…

A Demo Project for Fetching Remote Images

Before we start doing anything, it’s necessary to download a starter project. It contains a SwiftUI based iOS app called FetchableImageDemo. Let me describe fast its various parts starting from the views:

• UserView: A custom view with an Image and a Text that will display an avatar image and the contact name. It’s embedded to two other views, RandomContactView and ContactDetailsView.
• RandomContactView: A view that besides the UserView, also contains a button that triggers the selection of a contact randomly. We’ll use it for testing single image fetching.
• ContactListView: The primary component here is a List that will be displaying a list of contacts. We’ll use it for testing multiple image fetching and deletion. This view is embedded in a navigation view with two bar items; one for initiating the fetching process, and one for deleting all fetched avatars. When tapping on any contact item, the ContactDetailsView will be shown. Additionally, it contains another custom view, the ProgressView, which will display visually the download progress of all avatars.
• ContactDetailsView: Another view that embeds the UserView, and it also contains a button that will enable us to try out deleting a single image (the selected contact’s avatar), as well as to fetch it back.
• ProgressView: A custom implementation of a progress bar.
• Tabs: FetchableImageDemo is a tab bar application, and Tabs view is the one creating the tabs.

Besides the above, you’ll also find:

• The Contact structure: It represents a single contact. It contains an id, name, avatarURL, and avatar. We’ll be using the avatarURL in order to fetch the image that we’ll be assigning to avatar. It conforms to Decodable protocol so fake contact data can be loaded and decoded from the two JSON files included in the project, and also conforms to Identifiable protocol in order to easily iterate through such objects in the ContactListView using a ForEach loop.
• The RandomContactModel class: We’ll use it as the model for the RandomContactView. It’s partially implemented, and when we’ll finish with it it’ll be able to pick a random contact and fetch its avatar image using the protocol that we’ll build next.
• The ContactListModel class: The model for the ContactListView class. We’ll use it to try out most of the features we’ll add to the protocol, such as multiple image fetching and deletion, as well as single image deletion and download progress.

The fake contact data we’ll use is already embedded to the project. There are two JSON files named fakedata_small.json and fakedata.json, where the first one has data for 10 contacts, and the second data for 50 contacts. We’ll use the first one with the RandomContactModel and the RandomContactView classes, and the other with the contact list and the ContactListModel. Each contact data contains an id, name and a URL to an avatar image.

Note: Fake data was generated with Mockaroo.

Take your time to navigate in the starter project. When you’re ready, then keep reading and get prepared to start adding the missing pieces in the project.

Getting Started

Let’s start by creating a new Swift file where we’ll add the implementation that we’ll do here. Press Cmd+N in your keyboard, select the Swift file template under the Source category, and then continue to create the new file. Name it FetchableImage.swift. Once it’s ready select to open it in the Project Navigator if it doesn’t open automatically.

In order to make things easy for us as future users of the protocol that we’ll make, as well as for other developers, we’ll be based on a main idea: To use the URL of the remote image both for fetching the image from the remote source, and as a file name when saving an image locally after it has been fetched. To perform the latter though we’ll need to do some processing in the URL first; we can’t have a URL as a file name. Regardless, it won’t be necessary to add an additional burden of dealing with custom file names when images exist locally that way; the original URL that is already known will be used for both cases.

We’ll see everything in details, but for starters, let’s define the custom types that we’ll need. Let’s begin with the protocol and its extension. We’ll name it FetchableImage:

protocol FetchableImage {

}


extension FetchableImage {

}

Now, let’s declare the two structures that will help us build it:

struct FetchableImageOptions {

}


fileprivate struct FetchableImageHelper {

}

Note that the FetchableImageHelper structure is marked as file-private. That’s because we want it to be visible within this file only, and accessible only by the methods that we’ll implement next.

Specifying Available Options

Let’s talk a bit now about what the FetchableImageOptions structure will contain. Initially, let me tell you that we’re going to add a few stored properties only and no methods; it’s up to you to enrich it and add any methods that you might find suitable to exist.

Now, let’s focus on the directory where fetched images will be stored into. There are two options here; either have them stored in the documents or in the caches directory. The best practice says that they should be saved in the caches directory, as they can be fetched again if they are deleted by the system (in case of low free space on the device for example). However, there will be cases where it’s more appropriate to have a downloaded image stored in the document directory. Since there’s not a concrete rule about that and it all depends on the situation that FetchableImage comes to serve, then this is the first setting that we’ll allow through the FetchableImageOptions:

struct FetchableImageOptions {
    var storeInCachesDirectory: Bool = true
}

See that we set true as the default value of the storeInCachesDirectory property, so images are automatically saved to caches directory. This can be changed by creating a new FetchableImageOptions object and setting that flag to false.

Next, let’s refer to another scenario. In the majority of the the cases we’ll be keeping fetched images stored locally. In certain circumstances though we might don’t want that to happen; we might want to only fetch but not store locally. To deal with that, let’s add one more option where we’ll specify if fetched images are allowed to be stored locally or not:

struct FetchableImageOptions {
    ...

    var allowLocalStorage: Bool = true
}

We set true as the default value, because most commonly we’ll need fetched images to be saved locally by default.

Finally, one more property that even though we can live without it, we are definitely going to need it later in order to be able to save a new image straight from an app to a local directory. That is the file name of the image:

struct FetchableImageOptions {
    ...

    var customFileName: String?
}

Unless you’re saving new images with the FetchableImage protocol, chances are that you won’t need to use the customFileName property. It doesn’t hurt though to be a bit predictive and provide solutions that we may need in the future.

FetchableImageHelper First Bits

Right above we discussed about the documents and caches directory, but only in a theoretical level. Since we’re going to need to access them pretty soon, now it’s a good time to specify the URL to both.

In the FetchableImageHelper structure add the next two static properties:

fileprivate struct FetchableImageHelper {
    static var documentsDirectoryURL = FileManager.default.urls(for: .documentDirectory, in: .userDomainMask)[0]
    static var cachesDirectoryURL = FileManager.default.urls(for: .cachesDirectory, in: .userDomainMask)[0]
}

Declaring them as static makes it possible to use them without creating a FetchableImageHelper object, just like that: FetchableImageHelper.cachesDirectoryURL.

Adding The First Protocol Method

After having done the first steps, let’s start implementing the FetchableImage protocol. Normally, you’d expect to begin with a method that’s responsible for fetching a remote image, but that’s not the place where we’ll start from. First we’ll implement a method that returns the URL to the local image. We’ll need right next it in order to determine whether an image should be fetched from a remote source or be loaded from a local directory.

Initially, let’s declare it in the FetchableImage protocol:

func localFileURL(for imageURL: String?, options: FetchableImageOptions?) -> URL?

This method expects for two parameter values: The first one is the remote URL of the image. Please don’t be confused here; imageURL is the URL where the image should be fetched from. The method will return the URL to the local file, if that file exists.

Note that it’s declared as an optional, and it can be nil in one case only: If the image’s origin is not a remote site, but the image was created and stored locally directly from an app. In that case the customFileName must be provided through a FetchableImageOptions object, and that brings us to the second parameter value.

If the default values that we specified in the FetchableImageOptions properties are in accordance to our needs (store fetched images, save to caches directory, don’t use a custom file name), then passing nil is just fine. If one of the options should be altered, then a new FetchableImageOptions object must be given with the proper values being set to the properties.

With the above said, let’s pass to the default implementation of the method:

extension FetchableImage {

    func localFileURL(for imageURL: String?, options: FetchableImageOptions? = nil) -> URL? {

    }

}

Even though is not allowed to declare a default parameter value when declaring a method to a protocol, notice above (inside the FetchableImage extension) that we set nil as the default value for the options parameter.

The first thing we have to do is to determine what the containing directory of the file is; document or caches directory. We can determine that through the provided options argument in case it’s not nil. If it’s nil then the caches directory is the storage directory. But we can handle this a bit more generally and optimally; if options parameter value is nil, then we can initialize a new FetchableImageOptions object and “read” the default settings specified:

let opt = options != nil ? options! : FetchableImageOptions()

I can tell you from now that the above is something that we’ll need to do multiple times next. So instead of writing the above condition again and again, let’s just create a new method in the FetchableImageHelper structure that will return the proper object:

static func getOptions(_ options: FetchableImageOptions?) -> FetchableImageOptions {
    return options != nil ? options! : FetchableImageOptions()
}

Make sure that you added the above method in the FetchableImageHelper structure. Simply enough, the given options parameter value is returned by the method if it’s not nil. Otherwise, a new FetchableImageOptions object is being returned instead.

Back to the localFileURL(for:options:) method implementation, replace the first line we added before with this:

let opt = FetchableImageHelper.getOptions(options)

Now it’s easy to determine the containing directory:

let targetDir = opt.storeInCachesDirectory ?
    FetchableImageHelper.cachesDirectoryURL :
    FetchableImageHelper.documentsDirectoryURL

Knowing the target directory, let’s specify the file name of the image. First, let’s make sure that the imageURL parameter values is not nil:

guard let urlString = imageURL else {

}        

If it’s nil and the code execution falls to the else clause above, then we’ll check if there’s a custom file name provided in the options argument or not. If not, we return nil from the method; there’s nothing else to do without a file name. Otherwise, we append the custom file name to the target directory and we return it altogether as a URL object:

guard let customFileName = opt.customFileName else { return nil }
return targetDir.appendingPathComponent(customFileName)

The above handles the case where the imageURL parameter value is nil. If it’s not nil, then we’ll generate a file name based on the URL, and eventually we’ll return it as a full URL along with the target directory:

guard let imageName = FetchableImageHelper.getImageName(from: urlString) else { return nil }
return targetDir.appendingPathComponent(imageName)

getImageName(from:) is a method that we’ll implement right next. It will be possible to return a nil value, so a guard statement is necessary here (or use an if-let alternatively).

Before we get to it, here’s the localFileURL(for:options:) method in one piece:

func localFileURL(for imageURL: String?, options: FetchableImageOptions? = nil) -> URL? {
    let opt = FetchableImageHelper.getOptions(options)
    let targetDir = opt.storeInCachesDirectory ?
        FetchableImageHelper.cachesDirectoryURL :
        FetchableImageHelper.documentsDirectoryURL

    guard let urlString = imageURL else {
        guard let customFileName = opt.customFileName else { return nil }
        return targetDir.appendingPathComponent(customFileName)
    }

    guard let imageName = FetchableImageHelper.getImageName(from: urlString) else { return nil }
    return targetDir.appendingPathComponent(imageName)
}

Generating Image Name From Remote URL

A remote image URL contains special characters, such as the column and slashes, so it cannot consist of a file name when storing fetched images locally. There are various approaches one could follow here. For example, we could create a hash based on the URL path, but the downside is that we’d need to import cryptography frameworks. Or we could just remove all special characters and keep the alphanumeric part of the URL only. We could even come up with a custom algorithm to produce a file name based on the original URL.

In this implementation we’ll do something different. We’ll convert the URL’s path to a Base64 encoded string, and then we’ll remove any non-alphanumeric characters. This is going to be the file name we’ll be using for each image.

Jump to the FetchableImageHelper struct, and define the following method:

static func getImageName(from urlString: String) -> String? {

}

In order to get a Base64 encoded string from the parameter value, we need to convert it first to a Data object as shown next:

guard var base64String = urlString.data(using: .utf8)?.base64EncodedString() else { return nil }
base64String = base64String.components(separatedBy: CharacterSet.alphanumerics.inverted).joined()
return base64String

If the task to create a Data object first and the Base64 encoded representation afterwards does not fail, then we remove all non-alphanumeric characters in the second line. It probably does not look straightforward at first look, but all it does is to break the Base64 encoded string to an array based on the the non-alphanumeric characters (we get them by inverting the alphanumeric character set), and then to rejoin the separated pieces back to one string again.

The above small implementation is fine, however there’s a catch; depending on the length of the URL, we might end up with really long Base64 string values.

To work around that, we’ll add a check in the method. If base64String is longer than 50 characters long, then we’ll drop the first characters and we’ll keep the last 50 only. We won’t keep the first characters because all image URLs start pretty much the same way (“https://some.url/some_path/…”). The fact that we keep 50 characters instead of 10 or 20 ensures that there will be different characters in the resulting Base64 value.

So, before the return statement above add the following:

guard base64String.count < 50 else {
    return String(base64String.dropFirst(base64String.count - 50))
}

Here’s the entire method now:

static func getImageName(from urlString: String) -> String? {
    guard var base64String = urlString.data(using: .utf8)?.base64EncodedString() else { return nil }
    base64String = base64String.components(separatedBy: CharacterSet.alphanumerics.inverted).joined()

    guard base64String.count < 50 else {
        return String(base64String.dropFirst(base64String.count - 50))
    }

    return base64String
}

Fetching An Image

Time to implement one of the core methods of the FetchableImage protocol, maybe the most important one. In the FetchableImage protocol add the following declaration:

func fetchImage(from urlString: String?, options: FetchableImageOptions?, completion: @escaping (_ imageData: Data?) -> Void)

And in the protocol’s extension let’s start implementing it:

extension FetchableImage {
    ...

    func fetchImage(from urlString: String?, options: FetchableImageOptions? = nil, completion: @escaping (_ imageData: Data?) -> Void) {

    }
}

Although we call it “fetchImage”, this method won’t fetch remote images only; it will load images from local files as well. So, depending on the context, when saying fetching from now on we’ll be referring either to remote image fetching only, or both remote fetching and loading from a local file too.

Let’s take a look at its parameters for a moment. The first one is the URL of the remote image as a String value. It’s declared as an optional in order to cover the case of images not fetched from a remote site, but created and saved by the app locally; there will be no remote URL in this case.

The second parameter value is once again a FetchableImageOptions optional object. By default is nil, considering that the default values will consist of the desirable configuration most of the times.

Lastly, there’s the completion handler. Remember that fetching a remote image must be an asynchronous process, as there are many factors that can affect the download process (such as image size, network speed, and more) and therefore an image cannot be returned in real time; required fetching time is undetermined. The parameter value of the completion handler (imageData) is an optional Data object. If fetching an image from a remote source is successful, then we’ll pass its data to the completion handler. If it fails, we’ll just pass nil.

Note #1: We could have used the Result type introduced in Swift 5 as the parameter value of the completion handler and in case of failure to pass the actual error occurred: Result<Data, Error>. However, we’ll stick to the above approach for simplicity.

Note #2: We’ll pass the fetched (or loaded) image to the completion handler as a Data object. Instead of that, we could have a UIImage object, or a CGImage object, or even a NSImage object if we’re talking about macOS. However, we’re trying to implement a generic solution here, flexible enough to work with UIKit, SwiftUI, even AppKit on macOS. Passing a Data object to the completion handler consists of the most general solution, since data can be converted to the most appropriate object afterwards.

With the above being said, the first step in the method’s implementation is to make sure that we’re working on a background thread. That’s important, as we don’t want to block the main thread for as long as the image fetching is in progress:

DispatchQueue.global(qos: .background).async {

}

The next two steps involve getting the options (either from the options argument, or from a new FetchableImageOptions object if it’s nil), and the URL to the local file of the image:

let opt = FetchableImageHelper.getOptions(options)
let localURL = self.localFileURL(for: urlString, options: options)

You can see here that we’re using the getOptions(_:) method we implemented earlier once again, and we’re making use of the localFileURL(for:options:) even from the beginning of this method.

What’s coming next is important. First, we’ll check if the image exists locally. If that’s true, then we’ll try to load it from the local file and pass its data to the completion handler. If the image does not exist locally, or the allowLocalStorage property of the options argument has been set to false (meaning that the image with the given URL is not allowed to be stored locally), then we’ll fetch it using the URL. So, here we go:

if opt.allowLocalStorage,
    let localURL = localURL,
    FileManager.default.fileExists(atPath: localURL.path) {

} else {

}

Inside the if body which is the case of an image existing locally, we’ll call another method that will perform the actual loading from the file using the localURL value:

let loadedImageData = FetchableImageHelper.loadLocalImage(from: localURL)

We’ll implement the loadLocalImage(from:) pretty soon. loadedImageData will contain either the actual image as a Data object, or nil of something has gone wrong and no image data could be read from the file. In any case, we can pass the loadedImageData to the completion handler right after the above line:

completion(loadedImageData)

In the else body now, let’s take care of the case where the image does not exist locally and must be fetched from the given remote URL. Keep in mind that the URL is given as a String value though, so we must create a URL object using it:

guard let urlString = urlString, let url = URL(string: urlString) else {
    completion(nil)
    return
}

If the code execution ends up to the else case and the urlString parameter value is nil, or if it’s not a valid URL and the url object cannot be initialized, then we just pass nil to the completion handler; there’s nothing more to do.

On the other hand, if url object is initialized successfully, then we can download the image from the remote URL. Actual download code will be implemented to another method just like the loadLocalImage(from:), but for now let’s use it and we’ll deal with the missing methods in a while.

FetchableImageHelper.downloadImage(from: url) { (imageData) in

}

The downloadImage(from:completion:) method that we’re about to implement soon, accepts the URL that the image should be downloaded from, and a completion handler that gets called when download has finished. That completion handler contains the actual image data on success, or nil in case of error.

Inside the body of the completion handler we must perform two distinct tasks:

1. To save the fetched image data locally, if allowed.
2. To pass the imageData data (or nil) to the completion handler.

Here are both:

FetchableImageHelper.downloadImage(from: url) { (imageData) in
    if opt.allowLocalStorage, let localURL = localURL {
        try? imageData?.write(to: localURL)
    }

    completion(imageData)
}

And the fetchImage(from:options:completion:) method is now complete. There are two missing methods we’re using here and we’re going to add next, but before doing so, here’s the entire method implemented:

func fetchImage(from urlString: String?, options: FetchableImageOptions? = nil, completion: @escaping (_ imageData: Data?) -> Void) {
    DispatchQueue.global(qos: .background).async {

        let opt = FetchableImageHelper.getOptions(options)
        let localURL = self.localFileURL(for: urlString, options: options)

        // Determine if image exists locally first.
        if opt.allowLocalStorage,
            let localURL = localURL,
            FileManager.default.fileExists(atPath: localURL.path) {

            // Image exists locally!
            // Load it using the composed localURL.
            let loadedImageData = FetchableImageHelper.loadLocalImage(from: localURL)
            completion(loadedImageData)

        } else {
            // Image does not exist locally!
            // Download it.

            guard let urlString = urlString, let url = URL(string: urlString) else {
                completion(nil)
                return
            }

            FetchableImageHelper.downloadImage(from: url) { (imageData) in
                if opt.allowLocalStorage, let localURL = localURL {
                    try? imageData?.write(to: localURL)
                }

                completion(imageData)
            }

        }
    }
}

Performing The Actual Image Download

We’ll implement the missing methods starting from the most recent one, the one that performs the actual image download. In it, we’ll simply initialize a URLSession object and through that, we’ll use a data task in order to fetch the image data from the specified URL. If you’ve ever used URLSession before, then the following implementation will look familiar. Just make sure to add the next implementation in the FetchableImageHelper structure:

static func downloadImage(from url: URL, completion: @escaping (_ imageData: Data?) -> Void) {
    let sessionConfiguration = URLSessionConfiguration.ephemeral
    let session = URLSession(configuration: sessionConfiguration)
    let task = session.dataTask(with: url) { (data, response, error) in
        completion(data)
    }
    task.resume()
}

Notice that we don’t mind about the response or the potential error coming back from the server here. However, you’re free to update the above code and handle them too if they’re important to you. The only thing we’re doing here is to call the completion handler passing the fetched data to it.

Loading A Local Image File

With the downloadImage(from:completion:) in place, let’s pass to the other currently missing method; the loadLocalImage(from:) that we already made use of in the fetchImage(from:options:completion:) method previously.

Inside its body we’re going to perform a simple action; we’ll try to initialize a Data object using the contents of the file specified by the given URL. On success, we’ll return the initialized data object which is nothing else but the image data. On failure, we’ll just return nil. Once again, we’re adding this implementation to the FetchableImageHelper structure:

static func loadLocalImage(from url: URL) -> Data? {
    do {
        let imageData = try Data(contentsOf: url)
        return imageData
    } catch {
        print(error.localizedDescription)
        return nil
    }
}

Trying Out Image Fetching

Time to try out what we’ve done so far! In the starter project you’ve downloaded, you’ll find a file called RandomContactModel.swift. Spot it in the Project Navigator and click to open it.

The first thing that RandomContactModel class does is to load the fake contacts from the fakedata_small.json file, decode the JSON data and populate decoded data to the collection of Contact objects, called contacts.

It also contains a method called pickRandomContact() that selects randomly a Contact object from the collection of contacts. The selected contact will be displayed in the UserView view, which is embedded in the RandomContactView. For every contact, the avatar image will be fetched either from a remote or a local source and when it’ll become available the UI will be updated to display it.

Currently, the picker method does nothing fancy except for just picking a random contact as you can see in the pickRandomContact() method implementation:

func pickRandomContact() {
    let random = Int.random(in: 0..<10)
    guard random < contacts.count else { return }
    contact = contacts[random]
}

Since we want to use the FetchableImage protocol here, the first step we necessarily need to do is to make RandomContactModel class conform to it:

class RandomContactModel: ObservableObject, FetchableImage {
    ...
}

In the pickRandomContact() method, we can now call the fetchImage(from:options:completion:) method in order to fetch the image matching to the avatarURL property of the randomly picked contact object:

func pickRandomContact() {
    ...

    fetchImage(from: contact.avatarURL, options: nil) { (avatarData) in

    }
}

At the time being we pass no options, so the default ones will be used (images will be stored to the caches directory, storing remote images is allowed and no custom file names will be used).

Based on the logic we applied earlier, the above method will first try to load the image from a local file. If it doesn’t exist, then it’ll try to fetch it, and then store it to the caches directory for future use. In suqsequent calls of the method, the image will be loaded from the local file, so the display time will be remarkably smaller. We’ll see all that in action.

Since it’s convenient to work with CGImage images in Image controls in SwiftUI, we’ll first make sure that the avatarData is not nil and then we’ll try to initialize a UIImage object from that data. Then, from the UIImage object we’ll get the CGImage representation we’re looking for. Something important to note is that we must use the main thread in order to do what I just described. Remember that image fething takes place in the background, and altering the UI is only allowed from the main thread of the app.

func pickRandomContact() {
    ...

    fetchImage(from: contact.avatarURL, options: nil) { (avatarData) in
        if let data = avatarData {
            DispatchQueue.main.async {
                self.contact.avatar = UIImage(data: data)?.cgImage
            }
        }
    }
}

The avatar CGImage representation is assigned to the avatar property of the randomly selected contact. Since it’s marked as @Published, that change will trigger the UI to be updated in the UserView view.

Now, open the RandomContactView and add the following line as the action of the button:

Button("Random contact") {
    self.randomDataModel.pickRandomContact()
}

And… run the app to see the results! By default, each fake contact has a default avatar indicating the lack of the proper one. Click on the Random Contact button to pick a random contact, and see that within a few seconds the image will be fetched and displayed on the Image control. Keep clicking the button, so more images for more contacts to be fetched. Then, stop and run the app again so the avatar property of each contact becomes empty again. You’ll see that fetching images that have been downloaded already is almost instant; they’re loaded from the local file!

If you want to see the stored images, go to the pickRandomContact() method, and right before the fetchImage method call add the following print command:

func pickRandomContact() {
    ...

    print(localFileURL(for: contact.avatarURL))

    fetchImage(from: contact.avatarURL, options: nil) { (avatarData) in
        ...
    }
}

Run again and start tapping on the Random Contact button. This time, you’ll see in the console the URLs to local image files. Copy any displayed URL, but make sure not to include the prefix “file://” and the file name (stop in “Caches/”). Go to Finder and press Shift+Cmd+G, then paste the copied URL and press Return. That’s also a good chance to see the generated image names.

You can also play around with the options. Delete all downloaded files, and before calling the fetchImage method create the following FetchableImageOptions object. Pass it as the second argument in the fetchImage call:

let options = FetchableImageOptions(storeInCachesDirectory: true, allowLocalStorage: false, customFileName: nil)

func pickRandomContact() {
    ...

    let options = FetchableImageOptions(storeInCachesDirectory: true, allowLocalStorage: false, customFileName: nil)

    fetchImage(from: contact.avatarURL, options: options) { (avatarData) in
        ...
    }
}

By running again you’ll see that no images are stored anymore locally, and every time you ask for a random contact its avatar is being fetched again.

Let’s try something else. Update the above code with this:

func pickRandomContact() {
    ...

    // let options = FetchableImageOptions(storeInCachesDirectory: true, allowLocalStorage: false, customFileName: nil)
    let options = FetchableImageOptions(storeInCachesDirectory: false, allowLocalStorage: true, customFileName: nil)

    fetchImage(from: contact.avatarURL, options: options) { (avatarData) in
        ...
    }
}

In this configuration we allow local storage, but we set false to the first parameter value of the FetchableImageOptions initializer. Images will be stored to documents directory instead of caches directory.

Run and see that this time contact avatars are stored in the documents directory indeed.

Fetching Batch Images

Let’s move on and let’s make FetchableImage protocol capable of fetching multiple images as well. The good news is that we’ll be based on what we’ve built so far, so fetching batch images shouldn’t be intimidating. Initially, we’ll declare a new method in the FetchableImage protocol (switch to the FetchableImage.swift file):

protocol FetchableImage {
    func fetchBatchImages(using urlStrings: [String?], options: FetchableImageOptions?,
         partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
         completion: @escaping () -> Void)
}

Before we start implementing it in the protocol’s extension, let’s see the parameter values we’re setting here. First, it’s the collection of URLs as String values; we want to fetch multiple images, therefore we need multiple URLs. Next, we have the options that we’ve talked about many times so far.

What’s new and interesting here is the third parameter value, the partialFetchHandler. You might think of it as a progress handler as well. Imagine that you have to download a big number of images, and that task is going to take some time. Most probably you’ll want to keep your users informed about the download progress by showing a progress bar or a relevant message. This handler makes that possible. Every time an image is fetched, its data will be given as the first argument to this handler, with the second argument being the index of the image in the collection of all images. That way we will have every single image at our disposal right when it becomes available, and with the index we’ll be able to show the progress that has been made at any given moment.

Finally, the last parameter value it’s a completion that will be called when the overall process is finished. Notice that we pass no image data here; that happens in the partialFetchHandler.

Let’s go to the implementation part now. In the FetchableImage extension add the method definition:

extension FetchableImage {
    func fetchBatchImages(using urlStrings: [String?], options: FetchableImageOptions?,
         partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
         completion: @escaping () -> Void) {



    }
}

Now, an important clarification. We’ll fetch multiple images recursively, meaning we’ll be calling the same method to fetch an image again and again until all images have been fetched. That method won’t be the one we just defined; for that purpose, we’ll implement a new one.

Still being in the FetchableImage extension, add the next private method. This won’t be visible out of the extension, however it’ll help us achieve our goal:

private func performBatchImageFetching(using urlStrings: [String?], currentImageIndex: Int,
    options: FetchableImageOptions?,
    partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
    completion: @escaping () -> Void) {

}

As you notice, the parameters are almost identical to those of the previous method. The new thing here is the second parameter, which is the index of the URL that should be used in the urlStrings collection.

The implementation is fairly simple. At first, it’s necessary to make sure that the currentImageIndex is within the urlStrings count range, and proceed if only that’s true. Otherwise, we’ll just call the completion handler and return from the method:

private func performBatchImageFetching(using urlStrings: [String?], currentImageIndex: Int,
    options: FetchableImageOptions?,
    partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
    completion: @escaping () -> Void) {

    guard currentImageIndex < urlStrings.count else {
        completion()
        return
    }
}

The next step is to fetch the image pointed by the currentImageIndex. In order to do that, we’ll use the fetchImage(from:options:completion:) method that we have already made:

private func performBatchImageFetching(using urlStrings: [String?], currentImageIndex: Int,
    options: FetchableImageOptions?,
    partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
    completion: @escaping () -> Void) {

    ...

    fetchImage(from: urlStrings[currentImageIndex], options: options) { (imageData) in

    }
}

Inside the body of the above completion handler we’ll do two things. First, we’ll call the partial handler in order to pass to the caller the image data that was just fetched (or nil if no data exists), and the index of the current image:

private func performBatchImageFetching(using urlStrings: [String?], currentImageIndex: Int,
    options: FetchableImageOptions?,
    partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
    completion: @escaping () -> Void) {

    ...

    fetchImage(from: urlStrings[currentImageIndex], options: options) { (imageData) in
        partialFetchHandler(imageData, currentImageIndex)
    }
}

The second thing is to call the same method and recursively fetch the next image. What’s important to take care of here is the second argument; we pass the currentImageIndex increased by 1 to point to the next image. At the end we’ll call the completion handler:

private func performBatchImageFetching(using urlStrings: [String?], currentImageIndex: Int,
    options: FetchableImageOptions?,
    partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
    completion: @escaping () -> Void) {

    ...

    fetchImage(from: urlStrings[currentImageIndex], options: options) { (imageData) in
        partialFetchHandler(imageData, currentImageIndex)

        self.performBatchImageFetching(using: urlStrings,
            currentImageIndex: currentImageIndex + 1,
            options: options, partialFetchHandler: partialFetchHandler) {
            completion()
        }
    }
}

With that method implemented, we can go back to the fetchBatchImages(using:options:partialFetchHandler:completion:) method that we started making, but we never finished it. Simply enough, in it we’ll call the above method indicating the index of the first URL; 0 (zero). At the body of the completion handler of the invoked method, we’ll call the completion handler of this one so the caller knows that the entire process has finished.

func fetchBatchImages(using urlStrings: [String?],
                      options: FetchableImageOptions? = nil,
                      partialFetchHandler: @escaping (_ imageData: Data?, _ index: Int) -> Void,
                      completion: @escaping () -> Void) {

    performBatchImageFetching(using: urlStrings, currentImageIndex: 0,
        options: options, partialFetchHandler: { (imageData, index) in

        partialFetchHandler(imageData, index)

    }) {
        completion()
    }

}

And with that we finish the necessary work in order to enable multiple image fetching. As you understand, all the essential stuff was already made. Here we focused only on the recursion and on using the code that already works.

Trying Out Fetching Multiple Images

So, now that FetchableImage is capable of fetching batch images, let’s give it a try and see that it actually works. To start, open the ContactListModel.swift file where you’ll find the ContactListModel class. What’s already implemented here is the loading and decoding of the fake contact data, read this time by the fakedata.json file that exists in the demo project.

Notice here that there are three observed properties marked as @Published:

1. contacts: An array of Contact objects that contains the fake contact data.
2. progress: A Double value that will keep the download progress. Any change on this property will lead to visual update of the progress bar (implemented in the ProgressView, but embedded in the ContactListView).
3. isFetching: A flag indicating whether a download process is taking place or not. We’re going to need it in order to show or hide the progress, as well as to prevent starting a new download process while another one is already in progress.

Let’s fetch batch images now. First, adopt the FetchableImage protocol:

class ContactListModel: ObservableObject, FetchableImage {
    ...
}

Next, implement a new method called fetchAvatars(). First we’ll make sure that isFetching is false, and if so we’ll turn it to true and we’ll initialize the progress value:

func fetchAvatars() {
    guard !isFetching else { return }
    isFetching = true
    progress = 0.0
}

Next, let’s create a collection of all avatar URLs:

func fetchAvatars() {
    ...

    let allAvatarURLs = contacts.map { $0.avatarURL }
}

Time to call the fetchBatchImages(using:options:partialFetchHandler:completion:) method in order to initiate the batch image fetching:

func fetchAvatars() {
    ...

    fetchBatchImages(using: allAvatarURLs, options: nil, partialFetchHandler: { (imageData, index) in

        DispatchQueue.main.async {
            guard let data = imageData else { return }
            self.contacts[index].avatar = UIImage(data: data)?.cgImage

            self.progress = Double(((index + 1) * 100) / self.contacts.count)
        }

    }) {
        print("Finished fetching avatars!")

        DispatchQueue.main.asyncAfter(deadline: .now() + 0.4, execute: {
            self.isFetching = false
        })
    }
}

See what happens every time the partial handler is called:

• First, we switch to the main thread necessarily because image fetching is taking place in the background.
• Second, we make sure that there’s image data fetched and the returned value is not nil.
• Third, and probably most importantly, we use the index of the image in order to access the matching object in the contacts array and update its avatar. That action will make the UI get updated and display the avatar image.
• Finally, we calculate the progress as a percentage. This will update the progress bar.

Once all downloads are finished, the completion handler is called. Besides the non-important print statement which exists just for verification purposes, we switch the value of the isFetching property back to false. This is taking place on the main thread because it will affect the UI; it’ll make the progress bar become hidden. The small delay of 0.4 seconds is there just for the pleasure of the eye, so the progress bar does not disappear instantly right after the last image fetching.

Open the ContactListView.swift file now, and spot the button that works as the first navigation item. There we’ll call the fetchAvatars() method:

.navigationBarItems(
    leading:
    Button(action: {

        // Add this line:
        self.contactList.fetchAvatars()

    }) {
        Image(systemName: "arrow.clockwise")
    },

Run the app, and open the second tab. You’ll see a long list of fake contacts having the default avatar image. Tap on the refresh button, and see the avatars gradually being updated. On next run, you’ll find out that avatars will be shown almost instantly; that’s because they’re stored locally.

Note: We could have avatars being fetched automatically, but to make it easier to try out what we’re building we keep it as a manual action that gets triggered with the refresh bar button item.

Deleting Single Local Image Files

Fetching images is out of our way now, so let’s focus on the opposite actions for a while. Let’s make it possible to remove either single or batch image files from a local directory, and let’s start in this part with the first one.

Back to the FetchableImage.swift file, let’s declare the following new method:

protocol FetchableImage {
    ...

    func deleteImage(using imageURL: String?, options: FetchableImageOptions?) -> Bool
}

Whatever we’ve discussed about the image URL and options applies here too, so I’m skipping any kind of further discussion about them. However, see that this time there’s not a completion handler here. Instead, the deletion result is returned instantly from the method as a Boolean value. On successful deletion it’ll return true, otherwise it’ll return false.

Let’s jump to its implementation now. In the FetchableImage extension we’ll start by finding the URL to the local image:

extension FetchableImage {
    ...

    func deleteImage(using imageURL: String?, options: FetchableImageOptions? = nil) -> Bool {
        guard let localURL = localFileURL(for: imageURL, options: options),
            FileManager.default.fileExists(atPath: localURL.path) else { return false }
    }
}

If the URL to the local image is nil, or the file cannot be found, we just return false and we stop here. Otherwise, we try to delete the image file in a do-catch statement as shown next:

func deleteImage(using imageURL: String?, options: FetchableImageOptions? = nil) -> Bool {
    ...

    do {
        try FileManager.default.removeItem(at: localURL)
        return true
    } catch {
        print(error.localizedDescription)
        return false
    }
}

And that’s all. We can now go to try out single image deletion.

Trying Out Deleting Single Images

When using the demo app, then by tapping on a contact’s row in the list (second tab) you’re navigating to a details page. That’s nothing more than the contact’s avatar, name, and a button that allows to delete the avatar image. Actually, you’ll find out soon that the button has a double role. When the contact’s avatar exists, it allows to delete it. In the opposite case, it will be used to fetch it again.

So, here’s the place where we’ll test the image deletion. Before we get to the button though, open the ContactListModel.swift file where it’s necessary to add a couple of new methods. In the first one we’ll call the deleteImage(using:options:) method we implemented right above:

func deleteAvatar(for contact: Contact) {
    guard let index = contacts.firstIndex(where: { $0.id == contact.id }) else { return }

    if deleteImage(using: contacts[index].avatarURL, options: nil) {
        contacts[index].avatar = nil
    }
}

The brand new deleteAvatar(for:) method gets a Contact object as an argument, and based on that it tries to find its index among all Contact objects in the contacts array. Then, it calls the deleteImage(using:options:) method of the FetchableImage protocol, and on success it removes the actual avatar image by setting nil to the respective property.

The next method we’ll add in the ContactListModel class is necessary only to let us make the button in the ContactDetailsView have the double role I mentioned previously; delete and fetch. There’s nothing new that we haven’t discussed about already, so I’m presenting it at once:

func fetchAvatar(for contact: Contact, completion: @escaping (_ avatar: CGImage?) -> Void) {
    guard let index = contacts.firstIndex(where: { $0.id == contact.id }) else { return }

    fetchImage(from: contacts[index].avatarURL) { (imageData) in
        DispatchQueue.main.async {
            guard let data = imageData else { return }
            self.contacts[index].avatar = UIImage(data: data)?.cgImage
            completion(self.contacts[index].avatar)
        }
    }
}

Open the ContactDetailsView.swift now, and update the following:

Button(contact.avatar != nil ? "Delete Avatar" : "Fetch Avatar") {

}

with this:

Button(contact.avatar != nil ? "Delete Avatar" : "Fetch Avatar") {
    if self.contact.avatar != nil {
        self.contact.avatar = nil
        self.contactList.deleteAvatar(for: self.contact)
    } else {
        self.contactList.fetchAvatar(for: self.contact) { (fetchedAvatar) in
            guard let avatar = fetchedAvatar else { return }
            self.contact.avatar = avatar
        }
    }
}

Finally, run the app. Go to the details of a contact, and use the button you’ll find there. If the avatar already exists, it’ll be deleted, otherwise it’ll be fetched and displayed.

Deleting Multiple images

Now that single image deletion is working, we can easily add a new method in the FetchableImage protocol that will be deleting multiple images. And I’m saying easily, because we’ll be based on the single deletion method we implemented right before.

Open the FetchableImage.swift file, and add the following method declaration in the FetchableImage protocol:

protocol FetchableImage {
    ...

    func deleteBatchImages(using imageURLs: [String?], options: FetchableImageOptions?)
}

Then, in the protocol’s extension implement it. It’s as simple as it can get:

extension FetchableImage {
    ...

    func deleteBatchImages(using imageURLs: [String?], options: FetchableImageOptions? = nil) {
        DispatchQueue.global().async {
            imageURLs.forEach { _ = self.deleteImage(using: $0, options: options) }
        }
    }
}

First, we use the global background thread for our task. Then, using the forEach higher order function we go through all image URLs provided in the first parameter value, and we’re deleting one by one all image files.

Note: Read more about higher order functions here.

Notice that the above implementation is based on the assumption that the imageURLs parameter value contains the URLs of the images that should be deleted. It won’t work in case we wanted to delete multiple files with custom file names provided to the customFileName property of equal number of FetchableImageOptions objects. In such scenario, we would need to create a new method:

protocol FetchableImage {
    ...

    func deleteBatchImages(using multipleOptions: [FetchableImageOptions])
}

And the implementation in the protocol’s extension:

func deleteBatchImages(using multipleOptions: [FetchableImageOptions]) {
    DispatchQueue.global().async {
        multipleOptions.forEach { _ = self.deleteImage(using: nil, options: $0) }
    }
}

As you understand, each FetchableImageOptions item in the multipleOptions array will match to a single file with custom file name.

Trying Out Batch Image Deletion

We are going to try out here the batch deletion of images based on URLs, so open the ContactListModel.swift file where it’s necessary to add one new method in the ContactListModel class:

func deleteAllAvatars() {

}

In it, we’ll create an array of all avatar URLs, then we’ll call the method we just implemented in order to delete all local image files, and finally we’ll iterate through all contact objects in the contacts array to set nil to all avatar objects:

func deleteAllAvatars() {
    let allAvatarURLs = contacts.map { $0.avatarURL }
    deleteBatchImages(using: allAvatarURLs, options: nil)
    for (index, _) in contacts.enumerated() {
        contacts[index].avatar = nil
    }
}

Finally, go to the ContactListView.swift file, and find the second bar button. In the action closure call the deleteAllAvatars():

.navigationBarItems(
    leading:
    ...,

    trailing:
    Button(action: {

        // Add this line:
        self.contactList.deleteAllAvatars()

    }) {
        Image(systemName: "trash")
    }
)

Run the app and tap on the trash button in the contacts list if you’ve already fetched remote avatar images. All avatars will be removed, and you’ll need to tap on the refresh button to fetch them again.

Saving A Custom Image

There’s one last thing left to be done, and that is to add one last method in the FetchableImage protocol that will make it easy for us to save an image locally straight from the app, instead of fetching it from a remote source. That last method is not something that we need in order to use the FetchableImage protocol; however it would be nice to exist, so we can save custom images if necessary without having to come up with new solutions while developing an app.

So, open the FetchableImage.swift file for one last time, and add the next method declaration in the protocol:

protocol FetchableImage {
    ...

    func save(image data: Data, options: FetchableImageOptions) -> Bool
}

Notice here that:

1. The method accepts the data of the image that we want to save.
2. The options parameter value is not optional here; we need it to get the file name!

Let’s implement it now in the protocol’s extension:

extension FetchableImage {
    ...

    func save(image data: Data, options: FetchableImageOptions) -> Bool {
        guard let url = localFileURL(for: nil, options: options) else { return false }
        do {
            try data.write(to: url)
            return true
        } catch {
            print(error.localizedDescription)
            return false
        }
    }
}

If the URL to the actual file name cannot be formed or writing to file fails, we return false. If writing image data to file succeeds though, then we return true.

I leave that last feature for you to test.

Summary

FetchableImage protocol is now complete, and that brings us to the end of this post. I hope you enjoyed this step-by-step implementation of a really handy tool, which hopefully will fit in your own projects.

If you haven’t implemented your own solution in order to deal with remote images, then you can consider starting with what you read here and to extend or modify it in order to cover your specific needs.

For reference, you can download the complete project on GitHub.

If you want to go even further, you can make it a reusable library and wrap it up in a Swift package. Thanks for reading!

At WWDC 2017, Apple announced MusicKit, a framework to help developers build apps that allow users to play Apple Music and their local music library. Unlike most frameworks like ARKit or CoreML, MusicKit cannot be added to your code with a simple import function. Rather, it’s a combination of the Apple Music API, the StoreKit framework, the MediaPlayer framework, and some other web-based technologies.

You may be wondering why is this framework so difficult to integrate into your applications compared to other API and frameworks. This is because MusicKit was not only built to work on iOS devices, but Android and Web applications as well. Just like CloudKit or Sign in with Apple, you can access a user’s Apple Music account through Android and Web apps. However, for the purpose of this tutorial, we will simply focus on building a music player for iOS using SwiftUI.

Editor’s note: This is a two part series of our MusicKit tutorials. This is the first part and the second one will come next week. If you are new to SwiftUI, you can check out this tutorial and this sample chapter of our Mastering SwiftUI book.

The MusicKit Demo App

In this tutorial series, we’ll be building a very simple music player that searches using the Apple Music API for a song, grabs the relevant details of the song like artist name and album cover, and plays it on the device with the help of MediaPlayer’s MPMusicPlayerController. We’ll also see how to use the MediaPlayer framework to play, rewind, and skip songs.

Before we can do this, we need to first communicate with the Apple Music service to see if a user has a valid Apple Music subscription. This requires us to create a MusicKit identifier and private key to sign your developer tokens using Certificates, Identifiers & Profiles in our Apple Developer Program account. Since this can be very daunting for a lot of iOS developers, a majority of this tutorial will focus on creating these keys and identifiers. The next part of this tutorial will focus more on the MediaPlayer framework.

Our final app will look like this:

This tutorial was made using Xcode 11.3 and Swift 5.1. An active Apple Music subscription will be needed in order to test the app out. You may also need an active Apple Developer account in order to create some of the required identifiers and keys. You will also need to run your app on a real device because as of this time, the Simulator does not support Apple Music playback. We will also be running some Python code in order to generate private keys for us. If you don’t have pip installed, don’t worry, I will explain how to later in the tutorial.

Creating a MusicKit Identifier

The first step in communicating with the Apple Music API is to have a valid MusicKit identifier in your Apple Developer account. This identifier will be linked to our app and will let the Apple Music service know that our app is valid to access its services. Since our identifier is linked to our app’s Bundle Identifier, let’s quickly create the Xcode project first.

Open Xcode and click on Create a new Xcode Project. Under iOS, make sure that Tabbed App is the option selected before clicking on the Next button.

Next, name the project to MusicPlayer but feel free to change it if you like. Make sure that your User Interface  is set to SwiftUI. Finally, take note of the Bundle Identifier as displayed below. For example, my bundle identifier is “com.appcoda.MusicPlayer”. But you should use your own.

Click on Next and choose where you would like to save your project. That’s it! You’ve created the project and just for reference, you should see your Bundle Identifier again on your Project Dashboard.

Now we can create our MusicKit Identifier. Head to your Apple Developer account. Once you login, you should be greeted with a screen that looks like this. Make sure you click on Certificates, Identifiers, & Profiles.

Once the page loads, click on Identifiers on the left side of the page. You’ll get a list of all the App IDs your Apple Developer account is associated with.

Now, let’s create a new MusicKit Identifier. Click on the blue plus button next to the Identifiers title. A new page will load titled Register a New Identifier. Make sure that you select the Music IDs option before pressing Continue.

Next, you’ll be asked to quickly fill out a description and identifier for your Music ID. You can see what I filled out below. For your identifier, make sure that it is in the following format- music.<YOUR-BUNDLE-ID>.

Click Continue and make sure all your details are correct, click on Register.

That’s it! If all worked out, you should see a list of all your Music IDs and one of them being the one you just created!

Next, we’ll create a private key that is associated with the identifier we just created. This key can be used to sign a secret developer token that allows us to communicate with MusicKit.

Creating a Private Key

Generally speaking, private keys are important because they allow you to access and authenticate communication with some app services — such as Push Notifications, MusicKit, and DeviceCheck. We’ll need to create a private key in order to sign something called a developer token. This token will be used when sending requests to the Apple Music API in order to let them know that this request is coming from a verified developer. You’ll use the private key to create the token in the form of a JSON web token (JWT). You can learn more about JWT here.

Still in the Certificates, Identifiers, and Profiles page, click on the Keys tab to the left side of the page. You’ll see a list of all the keys associated with your developer account. If you’ve ever used Push Notifications in your apps, you’ll find the key you used for that service.

Let’s create a new key for our app. Just like before, click on the plus button next to Keys. Name your key (I’m using “AppCoda MusicPlayer”) and select MusicKit from the list of capabilities shown below.

Before you select Continue, we’ll need to configure this capability. Click on Configure and you’ll be shown a page where you can associate this private key with the MusicKit Identifier we created earlier. Select the Music ID you created and then click Save.

You’ll go to the previous page where you can finally click on Continue. After a quick checkup to make sure everything has been entered correctly, click on Register.

If all goes well, you’ll see a page like the one above asking you to download your key. Make sure you download the key and save it in a safe place as you will not be able to download your key again. Also, make sure you take note of your Key ID as we’ll need it soon.

Now that we have the private key downloaded, we’ll need to use it to create the developer token in the JWT format. However, if you look at the file you downloaded, you’ll see that it’s in a .p8 file format and you probably can’t open the file. Luckily, there’s a way to view the private key from this file.

Open a new tab in your browser and head to https://filext.com/file-extension/P8. This website will read your .p8 file and display the private key to you. Drop your file into the placeholder and wait for a couple seconds.

You can see that the private key is given to you. I’ve hidden it from my screenshot above. You should also strive to keep this key as much of a secret since this is directly linked to your developer account. Copy the private key and save it. We’ll also need this in a bit.

Python Script

Now it comes to the fun part, but slightly complicated. You’ll need to create the developer token using a Python script. So, first download the script file here. In order for the script to work, you’ll need to install some Python packages. MacOS already comes with Python installed, so you don’t have to do any extra work to install Python. All you need to do is install those Python packages.

First, let’s download pip. If you’re not familiar with Python, pip is a very simple and easy-to-use package management system used to install and manage software packages written in Python. You can think of it as CocoaPods or Carthage for Python. Open Terminal. Type the following command:

sudo easy_install pip

After entering your password, pip will be installed. If you have pip installed already, you’ll probably see the following screen which means we can move to the next party.

Now we need two packages in order for our script to work: pyjwt and cryptography. Both are used to encrypt all the sensitive information and transform it into the JWT format. Just like before enter the following two commands in terminal:

sudo pip install pyjwt
sudo pip install cryptography

As long as you don’t get any errors (usually in red text), you should be in the clear. We’re almost done. Now we need to edit the script in order to populate it with 3 pieces of information.

1. The MusicKit private key (copied from the website which displayed our .p8 file)
2. The Key ID from the private key we created in the Apple Developer website
3. Your Apple ID Team ID

You can find your Team ID by going to https://developer.apple.com/account and clicking on the Membership tab to the left.

Now, open the script you downloaded earlier. You’ll be able to see where to enter your own details in the script.

Some notes about modifying the script:

• Copy and paste your MusicKit private key exactly as it’s displayed. This means there should be 4 lines in between the BEGIN PRIVATE KEY and END PRIVATE KEY (which should also be included).
• In case you get confused in all the strings we’ve seen, your Key ID is an alphanumeric 10 character string.

When you’re done entering all your information, save the script and head back to Terminal. Type the following command:

cd PATH/TO/WHERE/MUSICTOKEN.PY/IS/LOCATED

Now, for the magic. Type the command into Terminal:

python musictoken.py

If you’ve done everything right, you’ll see your token and a sample CURL request. Your Terminal screen will look like mine below.

If you face any errors, make sure you go back and see if you edited the script correctly. You may also want to make sure that you have the right Key ID, MusicKit private key, and Team ID. If none of these work, leave a comment below and I’ll be happy to help.

If your Terminal screen looks like mine, congratulations! You’ve successfully completed the hardest party of this tutorial. Copy your developer token and save it as we’ll be using it in the next part of the tutorial. For the rest of the tutorial, let’s go back to Swift and build the base layout of our app.

Creating the UI of the Music Player app

Open the Xcode project we created earlier in the tutorial. You can see that we already have our Tab View created for us in ContentView.swift. Let’s create a new SwiftUI view for our player view. Go to File > New > File and make sure you have SwiftUI View selected as your preset.

Save this file as PlayerView.swift. And, you should see the very basic SwiftUI View.

Let’s change this up. Command-click on the Text view and choose Embed in VStack. This will be the view that displays our song’s title. Let’s add another Text view inside the VStack. This can be the view that displays the artist’s name. Change the values of both Text structs to reflect the change.

VStack {
    Text("Song Title")
    Text("Artist Name")
}

In order to offer a clearer distinction between both Text views, I’ll change the font and add some spacing between both views. This is for a purely design-based purpose. You can modify these values to whatever you see fit. Here is the code:

VStack(spacing: 8) {
    Text("Song Title")
        .font(Font.system(.title).bold())
    Text("Artist Name")
        .font(.system(.headline))
}

That’s much better! The users of our app can clearly distinguish between our song’s title and its artist. Now let’s add an Image view so we can show the album cover to our users! Before we do that, let’s copy all the code we have so far and paste it into a GeometryReader. This is a container view that defines its content as a function of its own size and coordinate space. It helps with resizing views for different device sizes.

Click on the Plus button in the top right corner of Xcode and search for Geometry Reader.

Select this and drag it to paste the code about where we declare our VStack. Now copy the VStack struct and paste it inside the GeometryReader’s placeholder. While our preview will still look the same, our code should now look like this.

GeometryReader { geometry in
    VStack(spacing: 8) {
        Text("Song Title")
            .font(Font.system(.title).bold())
        Text("Artist Name")
            .font(.system(.headline))
    }
}

Now we can add an Image view that will function as a place where we can display the album cover. Modify your code to look like this.

GeometryReader { geometry in
    // 1
    VStack(spacing: 24) {
        // 2
        Image(systemName: "a.square")
            .resizable() // 3
            .frame(width: geometry.size.width - 24, height: geometry.size.width - 24) // 4
            .cornerRadius(20)
            .shadow(radius: 10)

        VStack(spacing: 8) {
            Text("Song Title")
                .font(Font.system(.title).bold())
            Text("Artist Name")
                .font(.system(.headline))
        }
    }
}

1. First, I take the existing VStack and wrap it inside another VStack. This offers some spacing in between the Image view and the Text views.
2. Next, I created the Image view. Since the app has no data right now, I’m using an SF Symbol as a placeholder image.
3. In order to resize the Image, it’s important to add the .resizable() function.
4. Finally, I define the frame of my Image view. geometry.size.width is the width of the device. You can think of it like a SwiftUI version of self.view.frame.size.width. I also subtract 24 in order to create some margins.

Your preview should look a little like this now.

Now we need to create our play, rewind, and skip buttons. Click on the plus button like before and drag a button right below the VStack where we define our song’s information.

Let’s modify this button so it functions as our rewind button. Change the Button view code to look like this.

Button(action: {
    // 1
    print("Rewind")
}) {
    // 2
    ZStack {
        // 3
        Circle()
            .frame(width: 80, height: 80)
            .accentColor(.pink)
            .shadow(radius: 10)
        Image(systemName: "backward.fill")
            .foregroundColor(.white)
            .font(.system(.title))
    }
}

1. Currently since we don’t have access to the MediaPlayer framework, we’ll just print a statement when the button is tapped.
2. Next we define the look of our button. I decided to use a ZStack so we can use an SF Symbol as an image and use the Circle view as its background.
3. I create the Circle view first and give it its frame, color, and shadow for design purposes. I then use an SF Symbol to provide an image for the button.

We need two more of these buttons next to each other. Command and click on the Button view and click on Embed in HStack. Copy the button code and paste it inside the HStack struct two more times.

Obviously we don’t want our player to look like this so let’s modify the HStack code.

HStack(spacing: 40) {
    Button(action: {
        print("Rewind")
    }) {
        ZStack {
            Circle()
                .frame(width: 80, height: 80)
                .accentColor(.pink)
                .shadow(radius: 10)
            Image(systemName: "backward.fill")
                .foregroundColor(.white)
                .font(.system(.title))
        }
    }

    Button(action: {
        print("Pause")
    }) {
        ZStack {
            Circle()
                .frame(width: 80, height: 80)
                .accentColor(.pink)
                .shadow(radius: 10)
            Image(systemName: "pause.fill")
                .foregroundColor(.white)
                .font(.system(.title))
        }
    }

    Button(action: {
        print("Skip")
    }) {
        ZStack {
            Circle()
                .frame(width: 80, height: 80)
                .accentColor(.pink)
                .shadow(radius: 10)
            Image(systemName: "forward.fill")
                .foregroundColor(.white)
                .font(.system(.title))
        }
    }
}

In the code above, I’m simply adding a spacing of 40 between my buttons. All of them are similar to the button’s I’ve created earlier. The only difference is that each button prints a different statement regarding its function and has a different image.

Your player view should look like this now:

Next, let’s build the Search View.

Search View

Just like before, create a new SwiftUI view and name this file SearchView.swift. You’ll be greeted with the familiar template view.

Our search view will be very simple. It just needs a TextField view for searching songs and a List view for displaying the results. Meanwhile, we also need to populate some fake data since the app hasn’t connected with the Apple Music API. At the top of the file, right before you declare body, type the following:

@State private var searchText = ""
let songs = ["Blinding Lights", "That Way", "This Is Me"]

var body: some View {
    Text("Hello, World!")
}

The searchText variable is the binding variable we’ll use to connect our TextField view with. We define the songs array to populate our list with some temporary data.

Let’s build the TextField view first. Remove the Text view and replace it with the following code:

VStack {
    TextField("Search Songs", text: $searchText, onCommit: {
        print(self.searchText)
    })
    .textFieldStyle(RoundedBorderTextFieldStyle())
    .padding(.horizontal, 16)
    .accentColor(.pink)
}

I’ve created the TextField view with a placeholder text of “Search Songs” and attached any text type into this View to the searchText variable. The onCommit method is called when the user pressed Return on their keyboard. In the next part of the tutorial, we’ll implement our search functionality within this method.

Next, I’ll create the List view and populate it with the data from above. Underneath the TextField, paste the following code.

List {
    // 1
    ForEach(songs, id:\.self) { songTitle in
        // 2
        HStack {
            // 3
            Image(systemName: "rectangle.stack.fill")
                .resizable()
                .frame(width: 40, height: 40)
                .cornerRadius(5)
                .shadow(radius: 2)

            // 4
            VStack(alignment: .leading) {
                Text(songTitle)
                    .font(.headline)
                Text("Artist Name")
                    .font(.caption)
                    .foregroundColor(.secondary)
            }
            Spacer()
            // 5
            Button(action: {
                print("Playing \(songTitle)")
            }) {
                Image(systemName: "play.fill")
                    .foregroundColor(.pink)
            }
        }
    }
}
.accentColor(.pink)

If you’ve worked with the List view or the ForEach structure before, this should look familiar. If not, I’ll walk you through the code.

1. Inside my List view, I initialize the ForEach structure. This makes it easy to create multiple copies of views that I won’t have to explicitly code. As my data, I pass the songs array we created earlier. I also pass each element in that array as a variable called songTitle. This is so we can easily populate our cells without having to worry about array indexes.
2. Next, to emulate the look of a UITableViewCell, I create an HStack. I’ll be placing the album cover, song title, song artist, and add button within each cell inside the HStack.
3. Next is the Image view. As before, I’m using an SF Symbol as a placeholder image.
4. Next comes our Text views. I place them inside a VStack to conserve space and display multiple Text views in the same place. You’ll notice that the string value of our first Text view is the songTitle variable we created earlier.
5. Finally, I add a Play button which indicates to the user that our music player will play this song. Currently, it only prints to the console. In the next part of the tutorial, we’ll manage adding songs to the queue inside this method.

That’s all! Take a look at your Canvas and make sure it looks something like this:

Putting it all together

We’re almost done! All that’s left is going back to ContentView.swift and modifying the inside of our TabView to make sure it displays the two views we have created. Delete all the code inside TabView and replace it with the following.

TabView(selection: $selection) {
    PlayerView()
        .tag(0)
        .tabItem {
            VStack {
                Image(systemName: "music.note")
                Text("Player")
            }
        }
    SearchView()
        .tag(1)
        .tabItem {
            VStack {
                Image(systemName: "magnifyingglass")
                Text("Search")
            }
        }
}
.accentColor(.pink)

In the above code, I’m calling both PlayerView() and SearchView() as the two views in our TabView. As such, we’ll only have two tabs. I’ve also created the look of those tabs with a simple Image and Text.

By now you must have noticed that I’ve been using Color.pink inside all my buttons and .accentColors. This provides the app with a unique, music-themed design. You’re free to change the colors, fonts, and shapes to however you wish.

What’s Next

Congratulations! You’ve done a lot in this tutorial by venturing out of Swift and into further technologies like Python, Web Token, Encryption, and API handling. This was a lot but if you’ve made it to the end, congrats! The hardest part about using MusicKit is generating your keys and developer tokens. You can download the final project here.

In the next part of this tutorial series, we’ll complete our app. I’ll show you how to access the Apple Music API and make web requests to the API. Furthermore, I’ll teach you how to use MediaPlayer to play music and control playback from your device. If you have any doubts, feel free to leave a comment. Stay tuned!

First, what’s JSON? JSON (short for JavaScript Object Notation) is a text-based, lightweight, and easy way for storing and exchanging data. It’s commonly used for representing structural data and data interchange in client-server applications, serving as an alternative to XML. A lot of the web services we use every day have JSON-based APIs. Most of the iOS apps, including Twitter, Facebook, and Flickr send data to their backend web services in JSON format.

As an example, here is a JSON representation of a sample Movie object:

{
    "title": "The Amazing Spider-man",
    "release_date": "03/07/2012",
    "director": "Marc Webb",
    "cast": [
        {
            "name": "Andrew Garfield",
            "character": "Peter Parker"
        },
        {
            "name": "Emma Stone",
            "character": "Gwen Stacy"
        },
        {
            "name": "Rhys Ifans",
            "character": "Dr. Curt Connors"
        }
    ]
}

As you can see, JSON formatted data is more human-readable and easier to parse than XML. I’ll not go into the details of JSON. This is not the purpose of this chapter. If you want to learn more about the technology, I recommend you to check out the JSON Guide at http://www.json.org/.

Since the release of iOS 5, the iOS SDK has already made it easy for developers to fetch and parse JSON data. It comes with a handy class called NSJSONSerialization, which can automatically convert JSON formatted data to objects. Later in this chapter, I will show you how to use the API to parse some sample JSON formatted data, returned by a web service. Once you understand how it works, it is fairly easy to build an app by integrating with other free/paid web services.

Since the release of Swift 4, Apple introduced the Codable protocol to simplify the whole JSON archival and serialization process. We will also look into this new feature and see how we can apply it in JSON parsing.

JSON Demo App

As usual, we’ll create a demo app. Let’s call it KivaLoan. The reason why we name the app KivaLoan is that we will utilize a JSON-based API provided by Kiva.org. If you haven’t heard of Kiva, it is a non-profit organization with a mission to connect people through lending to alleviate poverty. It lets individuals lend as little as $25 to help create opportunities around the world. Kiva provides free web-based APIs for developers to access their data. For our demo app, we’ll call up the following Kiva API to retrieve the most recent fundraising loans and display them in a table view:

https://api.kivaws.org/v1/loans/newest.json

The returned data of the above API is in JSON format. Here is a sample result:

loans: (
        {
        activity = Retail;
        "basket_amount" = 0;
        "bonus_credit_eligibility" = 0;
        "borrower_count" = 1;
        description =         {
            languages =             (
                fr,
                en
            );
        };
        "funded_amount" = 0;
        id = 734117;
        image =         {
            id = 1641389;
            "template_id" = 1;
        };
        "lender_count" = 0;
        "loan_amount" = 750;
        location =         {
            country = Senegal;
            "country_code" = SN;
            geo =             {
                level = country;
                pairs = "14 -14";
                type = point;
            };
        };
        name = "Mar\U00e8me";
        "partner_id" = 108;
        "planned_expiration_date" = "2016-08-05T09:20:02Z";
        "posted_date" = "2016-07-06T09:20:02Z";
        sector = Retail;
        status = fundraising;
        use = "to buy fabric to resell";
    },
....
....
)

You will learn how to use the NSJSONSerialization class to convert the JSON formatted data into objects. It’s unbelievably simple. You’ll see what I mean in a while.

To keep you focused on learning the JSON implementation, you can first download the project template from http://www.appcoda.com/resources/swift5/KivaLoanStarter.zip. I have already created the skeleton of the app for you. It is a simple table-based app that displays a list of loans provided by Kiva.org. The project template includes a pre-built storyboard and custom classes for the table view controller and prototype cell. If you run the template, it should result in an empty table app.

Creating JSON Data Model

We will first create a class to model a loan. It’s not required for loading JSON but the best practice is to create a separate class (or structure) for storing the data model. The Loan class represents the loan information in the KivaLoan app and is used to store the loan information returned by Kiva.org. To keep things simple, we won’t use all the returned data of a loan. Instead, the app will just display the following fields of a loan:

• Name of the loan applicant

name = "Mar\U00e8me";

• Country of the loan applicant

location =         {
            country = Senegal;
            "country_code" = SN;
            geo =             {
                level = country;
                pairs = "14 -14";
                type = point;
            };
        };

• How the loan will be used

use = "to buy fabric to resell";

• Amount

"loan_amount" = 750;

These fields are good enough for filling up the labels in the table view. Now create a new class file using the Swift File template. Name it Loan.swift and declare the Loan structure like this:

struct Loan {

    var name: String = ""
    var country: String = ""
    var use: String = ""
    var amount: Int = 0

}

JSON supports a few basic data types including number, String, Boolean, Array, and Objects (an associated array with key and value pairs).

For the loan fields, the loan amount is stored as a numeric value in the JSON-formatted data. This is why we declared the amount property with the type Int. For the rest of the fields, they are declared with the type String.

Fetching Loans with the Kiva API

As I mentioned earlier, the Kiva API is free to use. No registration is required. You may point your browser to the following URL and you’ll get the latest fundraising loans in JSON format.

https://api.kivaws.org/v1/loans/newest.json

Okay, let’s see how we can call up the Kiva API and parse the returned data. First, open KivaLoanTableViewController.swift and declare two variables at the very beginning:

private let kivaLoanURL = "https://api.kivaws.org/v1/loans/newest.json"
private var loans = [Loan]()

We just defined the URL of the Kiva API, and declare the loans variable for storing an array of Loan objects. Next, insert the following methods in the same file:

func getLatestLoans() {
    guard let loanUrl = URL(string: kivaLoanURL) else {
        return
    }

    let request = URLRequest(url: loanUrl)
    let task = URLSession.shared.dataTask(with: request, completionHandler: { (data, response, error) -> Void in

        if let error = error {
            print(error)
            return
        }

        // Parse JSON data
        if let data = data {
            self.loans = self.parseJsonData(data: data)

            // Reload table view
            OperationQueue.main.addOperation({ 
                self.tableView.reloadData()
            })
        }
    })

    task.resume()
}

func parseJsonData(data: Data) -> [Loan] {

    var loans = [Loan]()

    do {
        let jsonResult = try JSONSerialization.jsonObject(with: data, options: JSONSerialization.ReadingOptions.mutableContainers) as? NSDictionary

        // Parse JSON data
        let jsonLoans = jsonResult?["loans"] as! [AnyObject]
        for jsonLoan in jsonLoans {
            let loan = Loan()
            loan.name = jsonLoan["name"] as! String
            loan.amount = jsonLoan["loan_amount"] as! Int
            loan.use = jsonLoan["use"] as! String
            let location = jsonLoan["location"] as! [String:AnyObject]
            loan.country = location["country"] as! String
            loans.append(loan)
        }

    } catch {
        print(error)
    }

    return loans
}

These two methods form the core part of the app. Both methods work collaboratively to call the Kiva API, retrieve the latest loans in JSON format and translate the JSON-formatted data into an array of Loan objects. Let’s go through them in detail.

In the getLatestLoans method, we first instantiate the URL structure with the URL of the Kiva Loan API. The initialization returns us an optional. This is why we use the guard keyword to see if the optional has a value. If not, we simply return and skip all the code in the method.

Next, we create a URLSession with the load URL. The URLSession class provides APIs for dealing with online content over HTTP and HTTPS. The shared session is good enough for making simple HTTP/HTTPS requests. In case you have to support your own networking protocol, URLSession also provides you an option to create a custom session.

One great thing of URLSession is that you can add a series of session tasks to handle the loading of data, as well as uploading and downloading files and data fetching from servers (e.g. JSON data fetching).

With sessions, you can schedule three types of tasks: data tasks (URLSessionDataTask) for retrieving data to memory, download tasks (URLSessionDownloadTask) for downloading a file to disk, and upload tasks (URLSessionUploadTask) for uploading a file from disk. Here we use the data task to retrieve contents from Kiva.org. To add a data task to the session, we call the dataTask method with the specific URL request. After you add the task, the session will not take any action. You have to call the resume method (i.e. task.resume()) to initiate the data task.

Like most networking APIs, the URLSession API is asynchronous. Once the request completes, it returns the data (as well as errors) by calling the completion handler.

In the completion handler, immediately after the data is returned, we check for an error. If no error is found, we invoke the parseJsonData method.

The data returned is in JSON format. We create a helper method called parseJsonData for converting the given JSON-formatted data into an array of Loan objects. The Foundation framework provides the JSONSerialization class, which is capable of converting JSON to Foundation objects and converting Foundation objects to JSON. In the code snippet, we call the jsonObject method with the given JSON data to perform the conversion.

When converting JSON formatted data to objects, the top-level item is usually converted to a Dictionary or an Array. In this case, the top level of the returned data of the Kiva API is converted to a dictionary. You can access the array of loans using the key loans.

How do you know what key to use?

You can either refer to the API documentation or test the JSON data using a JSON browser (e.g. http://jsonviewer.stack.hu). If you’ve loaded the Kiva API into the JSON browser, here is an excerpt from the result:

{
  "paging": {
    "page": 1,
    "total": 5297,
    "page_size": 20,
    "pages": 265
  },
  "loans": [
    {
      "id": 794429,
      "name": "Joel",
      "description": {
        "languages": [
          "es",
          "en"
        ]
      },
      "status": "fundraising",
      "funded_amount": 0,
      "basket_amount": 0,
      "image": {
        "id": 1729143,
        "template_id": 1
      },
      "activity": "Home Appliances",
      "sector": "Personal Use",
      "use": "To buy home appliances.",
      "location": {
        "country_code": "PE",
        "country": "Peru",
        "town": "Ica",
        "geo": {
          "level": "country",
          "pairs": "-10 -76",
          "type": "point"
        }
      },
      "partner_id": 139,
      "posted_date": "2015-11-20T08:50:02Z",
      "planned_expiration_date": "2016-01-04T08:50:02Z",
      "loan_amount": 400,
      "borrower_count": 1,
      "lender_count": 0,
      "bonus_credit_eligibility": true,
      "tags": [

      ]
    },
    {
      "id": 797222,
      "name": "Lucy",
      "description": {
        "languages": [
          "en"
        ]
      },
      "status": "fundraising",
      "funded_amount": 0,
      "basket_amount": 0,
      "image": {
        "id": 1732818,
        "template_id": 1
      },
      "activity": "Farm Supplies",
      "sector": "Agriculture",
      "use": "To purchase a biogas system for clean cooking",
      "location": {
        "country_code": "KE",
        "country": "Kenya",
        "town": "Gatitu",
        "geo": {
          "level": "country",
          "pairs": "1 38",
          "type": "point"
        }
      },
      "partner_id": 436,
      "posted_date": "2016-11-20T08:50:02Z",
      "planned_expiration_date": "2016-01-04T08:50:02Z",
      "loan_amount": 800,
      "borrower_count": 1,
      "lender_count": 0,
      "bonus_credit_eligibility": false,
      "tags": [

      ]
    },

     ...

As you can see from the above code, paging and loans are two of the top-level items. Once the JSONSerialization class converts the JSON data, the result (i.e. jsonResult) is returned as a Dictionary with the top-level items as keys. This is why we can use the key loans to access the array of loans. Here is the line of code for your reference:

let jsonLoans = jsonResult?["loans"] as! [AnyObject]

With the array of loans (i.e. jsonLoans) returned, we loop through the array. Each of the array items (i.e. jsonLoan) is converted into a dictionary. In the loop, we extract the loan data from each of the dictionaries and save them in a Loan object. Again, you can find the keys (highlighted in yellow) by studying the JSON result. The value of a particular result is stored as AnyObject. AnyObject is used because a JSON value could be a String, Double, Boolean, Array, Dictionary or null. This is why you have to downcast the value to a specific type such as String and Int. Lastly, we put the loan object into the loans array, which is the return value of the method.

for jsonLoan in jsonLoans {
    var loan = Loan()

    loan.name = jsonLoan["name"] as! String
    loan.amount = jsonLoan["loan_amount"] as! Int
    loan.use = jsonLoan["use"] as! String
    let location = jsonLoan["location"] as! [String: AnyObject]
    loan.country = location["country"] as! String

    loans.append(loan)
}

After the JSON data is parsed and the array of loans is returned, we call the reloadData method to reload the table. You may wonder why we need to call OperationQueue.main.addOperation and execute the data reload in the main thread.

The block of code in the completion handler of the data task is executed in a background thread. If you call the reloadData method in the background thread, the data reload will not happen immediately. To ensure a responsive GUI update, this operation should be performed in the main thread. This is why we call the OperationQueue.main.addOperation method and request to run the reloadData method in the main queue.

OperationQueue.main.addOperation({ 
    self.tableView.reloadData()
})

Quick note: You can also use dispatch_async function to execute a block of code in the main thread. But according to Apple, it is recommended to use OperationQueue over dispatch_async. As a general rule, Apple recommends using the highest-level APIs rather than dropping down to the low-level ones.

Displaying Loans in A Table View

With the loans array in place, the last thing we need to do is to display the data in the table view. Update the following methods in KivaLoanTableViewController.swift:

override func numberOfSections(in tableView: UITableView) -> Int {
    // Return the number of sections
    return 1
}

override func tableView(_ tableView: UITableView, numberOfRowsInSection section: Int) -> Int {
    // Return the number of rows
    return loans.count
}

override func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
    let cell = tableView.dequeueReusableCell(withIdentifier: "Cell", for: indexPath) as! KivaLoanTableViewCell

    // Configure the cell...
    cell.nameLabel.text = loans[indexPath.row].name
    cell.countryLabel.text = loans[indexPath.row].country
    cell.useLabel.text = loans[indexPath.row].use
    cell.amountLabel.text = "$\(loans[indexPath.row].amount)"

    return cell
}

The above code is pretty straightforward if you are familiar with the implementation of UITableView. In the tableView(_:cellForRowAt:) method, we retrieve the loan information from the loans array and populate them in the custom table cell. One thing to take note of is the code below:

"$\(loans[indexPath.row].amount)"

In some cases, you may want to create a string by adding both string (e.g. $) and integer (e.g. loans[indexPath.row].amount) together. Swift provides a powerful way to create these kinds of strings, known as string interpolation. You can make use of it by using the above syntax.

Lastly, insert the following line of code in the viewDidLoad method to start fetching the loan data:

getLatestLoans()

Compile and Run the App

Now it’s time to test the app. Compile and run it in the simulator. Once launched, the app will pull the latest loans from Kiva.org and display them in the table view.

For reference, you can download the complete Xcode project from http://www.appcoda.com/resources/swift5/KivaLoan.zip.

Introducing Codable

Since the release of Swift 4, Apple introduced a new way to encode and decode JSON data using Codable. We will rewrite the JSON decoding part of the demo app using this new approach.

Before we jump right into the modification, let me give you a basic walkthrough of Codable. If you look into the documentation of Codable, it is just a type alias of a protocol composition:

typealias Codable = Decodable & Encodable

Decodable and Encodable are the two actual protocols you need to work with. However, for convenience’s sake, we usually refer to this type alias for handling JSON encoding and decoding.

First, what’s the advantage of using Codable over the traditional approach for encoding/decoding JSON? If you go back to the previous section and read the code again, you will notice that we had to manually parse the JSON data, convert it into dictionaries and create the Loan objects.

Codable simplifies the whole process by offering developers a different way to decode (or encode) JSON. As long as your type conforms to the Codable protocol, together with the new JSONDecoder, you will be able to decode the JSON data into your specified instances. The below figure illustrates the decoding of a sample loan data into an instance of Loan using JSONDecoder.

Decoding JSON

To give you a better idea about how Codable works, let’s start a Playground project and write some code. Once you have created your Playground project, declare the following json variable:

let json = """
{

"name": "John Davis",
"country": "Peru",
"use": "to buy a new collection of clothes to stock her shop before the holidays.",
"amount": 150

}
"""

We will first start with the basics. Here we define a very simple JSON data with 4 items. The value of the first three items are of the type String and the last one is of the type Int. As a side note, if this is the first time you see the pair of triple quotes ("""), this syntax was introduced in Swift 4 for declaring strings with multi-lines.

Next, declare the Loan structure like this:

struct Loan: Codable {
    var name: String
    var country: String
    var use: String
    var amount: Int
}

This Loan structure is very similar to the one we defined in the previous section, except that it adopts the Codable protocol. You should also note that the property names match those of the JSON data.

Now, let’s see the magic!

Continue to insert the following code in your Playground file:

let decoder = JSONDecoder()

if let jsonData = json.data(using: .utf8) {

    do {
        let loan = try decoder.decode(Loan.self, from: jsonData)
        print(loan)

    } catch {
        print(error)
    }
}

In the code above, we instantiate an instance of JSONDecoder and then convert the JSON string we defined earlier into Data. The magic happens in this line of code:

let loan = try decoder.decode(Loan.self, from: jsonData)

You just need to call the decode method of the decoder with the JSON data and specify the type of the value to decode (i.e. Loan.self). The decoder will automatically parse the JSON data and convert them into a Loan object.

If you’ve done it correctly, you should see this line in the console:

Loan(name: "John Davis", country: "Peru", use: "to buy a new collection of clothes to stock her shop before the holidays.", amount: 150)

Cool, right?

JSONDecoder automatically decodes the JSON data and stores the decoded value in the corresponding property of the specified type (here, it is Loan).

Working with Custom Property Names

Earlier, I showed you the simplest example of JSON decoding. However, the decoding process is not always so straightforward. Now, let’s take a look another example.

Sometimes, the property name of your type and the key of the JSON data are not exactly matched. How can you perform the decoding?

Let’s say, we define the json variable like this:

let json = """
{

"name": "John Davis",
"country": "Peru",
"use": "to buy a new collection of clothes to stock her shop before the holidays.",
"loan_amount": 150

}
"""

In the JSON data, the key of the loan amount is changed from amount to loan_amount. How can we decode the data without changing the property name amount of Loan?

Now, update the Loan structure like this:

struct Loan: Codable {
    var name: String
    var country: String
    var use: String
    var amount: Int

    enum CodingKeys: String, CodingKey {
        case name
        case country
        case use
        case amount = "loan_amount"
    }
}

To define the mapping between the key and the property name, you are required to declare an enum called CodingKeys that has a rawValue of type String and conforms to the CodingKey protocol. In the enum, you define all the property names of your model and their corresponding key in the JSON data. Say, the case amount is defined to map to the key loan_amount. If both the property name and the key of the JSON data are the same, you can omit the assignment.

If you’ve changed the code correctly, you should be able to decode the updated JSON data with the following message found in the console:

Loan(name: "John Davis", country: "Peru", use: "to buy a new collection of clothes to stock her shop before the holidays.", amount: 150)

Working with Nested JSON Objects

The JSON data that we have worked on so far has only one level. In reality, the JSON data is usually more complex with multiple levels. Now let’s see how we can decode nested JSON objects.

First, update the json variable like this:

let json = """
{

"name": "John Davis",
"location": {
"country": "Peru",
},
"use": "to buy a new collection of clothes to stock her shop before the holidays.",
"loan_amount": 150

}
"""

We’ve made a minor change to the data by introducing the location key that has a nested JSON object with the nested key country. How can we decode this type of JSON data and retrieve the value of country from the nested object?

Now, modify the Loan structure like this:

struct Loan: Codable {
    var name: String
    var country: String
    var use: String
    var amount: Int

    enum CodingKeys: String, CodingKey {
        case name
        case country = "location"
        case use
        case amount = "loan_amount"
    }

    enum LocationKeys: String, CodingKey {
        case country
    }

    init(from decoder: Decoder) throws {
        let values = try decoder.container(keyedBy: CodingKeys.self)

        name = try values.decode(String.self, forKey: .name)

        let location = try values.nestedContainer(keyedBy: LocationKeys.self, forKey: .country)
        country = try location.decode(String.self, forKey: .country)

        use = try values.decode(String.self, forKey: .use)
        amount = try values.decode(Int.self, forKey: .amount)

    }
}

Similar to what we have done earlier, we have to define an enum CodingKeys. For the case country, we specify to map to the key location. To handle the nested JSON object, we need to define an additional enumeration. In the code above, we name it LocationKeys and declare the case country that matches the key country of the nested object.

Since it is not a direct mapping, we need to implement the initializer of the Decodable protocol to handle the decoding of all properties. In the init method, we first invoke the container method of the decoder with CodingKeys.self to retrieve the data related to the specified coding keys, which are name, location, use and amount.

To decode a specific value, we call the decode method with the specific key (e.g. .name) and the associated type (e.g. String.self). The decoding of the name, use and amount is pretty straightforward. For the country property, the decoding is a little bit tricky. We have to call the nestedContainer method with LocationKeys.self to retrieve the nested JSON object. From the values returned, we further decode the value of country.

That is how you decode JSON data with nested objects. If you’ve followed me correctly, you should see the following message in the console:

Loan(name: "John Davis", country: "Peru", use: "to buy a new collection of clothes to stock her shop before the holidays.", amount: 150)

Working with Arrays

In the JSON data returned from Kiva API, it usually comes with more than one loan. Multiple loans are structured in the form of an array. Now, let’s see how to decode an array of JSON objects using Codable.

First, modify the json variable like this:

let json = """

[{
"name": "John Davis",
"location": {
"country": "Paraguay",
},
"use": "to buy a new collection of clothes to stock her shop before the holidays.",
"loan_amount": 150
},
{
"name": "Las Margaritas Group",
"location": {
"country": "Colombia",
},
"use": "to purchase coal in large quantities for resale.",
"loan_amount": 200
}]

"""

To decode the above array into an array of Loan object, all you need to use is to modify the following line of code from:

let loan = try decoder.decode(Loan.self, from: jsonData)

to:

let loans = try decoder.decode([Loan].self, from: jsonData)

As you can see, you just need to specify [Loan].self when decoding the JSON data.

Now that the JSON data is fully utilized, but sometimes you may want to ignore some key/value pairs. Let’s say, we update the json variable like this:

let json = """
{
"paging": {
"page": 1,
"total": 6083,
"page_size": 20,
"pages": 305
},
"loans":
[{
"name": "John Davis",
"location": {
"country": "Paraguay",
},
"use": "to buy a new collection of clothes to stock her shop before the holidays.",
"loan_amount": 150
},
{
"name": "Las Margaritas Group",
"location": {
"country": "Colombia",
},
"use": "to purchase coal in large quantities for resale.",
"loan_amount": 200
}]
}
"""

This JSON data comes with two top-level objects : paging and loans. Apparently, we are only interested in the data related to loans. In this case, how can you decode the array of loans?

To do that, declare another struct named LoanDataStore that also adopts Codable:

struct LoanDataStore: Codable {
    var loans: [Loan]
}

This LoanDataStore only has a loans property that matches the key loans of the JSON data.

Now modify the following line of code from:

let loans = try decoder.decode([Loan].self, from: jsonData)

to:

let loanDataStore = try decoder.decode(LoanDataStore.self, from: jsonData)

The decoder will automatically decode the loans JSON objects and store them into the loans array of LoanDataStore. You can add the following lines of code to verify the content of the array:

for loan in loanDataStore.loans {
    print(loan)
}

The console should have an output like this:

Loan(name: "John Davis", country: "Paraguay", use: "to buy a new collection of clothes to stock her shop before the holidays.", amount: 150)
Loan(name: "Las Margaritas Group", country: "Colombia", use: "to purchase coal in large quantities for resale.", amount: 200)

Using Codable in the KivaLoan App

Now that I believe you have some ideas about how to decode JSON using Codable, let’s go back to the KivaLoan project and modify it to use Codable.

Open Loan.swift and replace it with the following code:

import Foundation

struct Loan: Codable {

    var name: String = ""
    var country: String = ""
    var use: String = ""
    var amount: Int = 0

    enum CodingKeys: String, CodingKey {
        case name
        case country = "location"
        case use
        case amount = "loan_amount"
    }

    enum LocationKeys: String, CodingKey {
        case country
    }

    init(from decoder: Decoder) throws {
        let values = try decoder.container(keyedBy: CodingKeys.self)

        name = try values.decode(String.self, forKey: .name)

        let location = try values.nestedContainer(keyedBy: LocationKeys.self, forKey: .country)
        country = try location.decode(String.self, forKey: .country)

        use = try values.decode(String.self, forKey: .use)
        amount = try values.decode(Int.self, forKey: .amount)

    }
}


struct LoanDataStore: Codable {
    var loans: [Loan]
}

The code above is exactly the same as the one we developed earlier. The LoanDataStore is designed to store an array of loans.

Next, replace the parseJsonData method of the KivaLoanTableViewController class with the following code:

func parseJsonData(data: Data) -> [Loan] {

    var loans = [Loan]()

    let decoder = JSONDecoder()

    do {
        let loanDataStore = try decoder.decode(LoanDataStore.self, from: data)
        loans = loanDataStore.loans

    } catch {
        print(error)
    }

    return loans
}

Here, we just use the JSONDecoder to decode the JSON data instead of JSONSerialization. I will not go into the code because it is the same as we have just worked on in the Playground project.

Now you’re ready to hit the Run button and test the app in the simulator. Everything should be the same as before. Under the hood, the app now makes use of Codable in Swift 5 to decode JSON.

If you’ve worked with UIKit before, the Text control in SwiftUI is very similar to UILabel in UIKit. It’s a view for you to display one or multiple lines of text. This Text control is non-editable but is useful for presenting read-only information on screen. For example, you want to present an on-screen message, you can use Text to implement it. 

SwiftUI, Apple’s new declarative UI framework, has been officially released for around two months. If you still haven’t explored the framework, this tutorial is written for you. In this tutorial, I’ll show you how to work with Text to present information, customize it with your preferred color, and apply rotation effects to create perspective text.

Creating a New Project for Playing with SwiftUI

First, fire up Xcode and create a new project using the Single View App template. Type the name of the project. I set it to SwiftUIText but you’re free to use any other name. For the organization name, you can set it to your company or organization. The organization identifier is a unique identifier of your app. Here I use com.appcoda but you should set to your own value. If you have a website, set it to your domain in reverse domain name notation. To use SwiftUI, you have to explicitly check the Use SwiftUI checkbox. Click Next and choose a folder to create the project.

Once you save the project, Xcode should load the ContentView.swift file and display a design/preview canvas. If you can’t see the design canvas, you can go up to the Xcode menu and choose Editor > Canvas to enable it.

Displaying a Simple Text

The sample code generated in ContentView already shows you how to display a single line of text. You initialize a Text and pass it with the text (e.g. Hello World) to display like this:

Text("Hello World")

The preview canvas should display Hello World on screen. This is the basic syntax for creating a text view. You’re free to change the text to whatever value you want and the canvas should show you the change instantaneously.

Changing the Font Type and Color

In SwiftUI, you can change the properties (e.g. color) of a control by calling methods that are known as Modifiers. Let’s say, you want to bold the text. You can use the modifier named fontWeight and specify your preferred font weight (e.g. .bold) like this:

Text("Stay Hungry. Stay Foolish.").fontWeight(.bold)

You access the modifier by using the dot syntax. Whenever you type a dot, Xcode will show you the possible modifiers or values you can use. For example, you will see various font weight options when you type a dot in the fontWeight modifier. You can choose bold to bold the text. If you want to make it even bolder, use heavy or black.

By calling fontWeight with the value .bold, it actually returns you with a new view that has the bolded text. What’s interesting in SwiftUI is that you can further chain this new view with other modifiers. Say, you want to make the bolded text a little bit bigger, you can write the code like this:

Text("Stay Hungry. Stay Foolish.").fontWeight(.bold).font(.title)

Since we may chain multiple modifiers together, we usually write the code above in the following format:

Text("Stay Hungry. Stay Foolish.")
    .fontWeight(.bold)
    .font(.title)

The functionality is the same but I believe you’ll find the code above more easy to read. We will continue to use this coding convention for the rest of this book.

The font modifier lets you change the font properties. In the code above, we specify to use the title font type in order to enlarge the text. SwiftUI comes with several built-in text styles including title, largeTitle, body, etc. If you want to further increase the font size, replace .title with .largeTitle. 

Note: You can always to refer the documentation to find out all the supported values of the font modifier.

You can also use the font modifier to specify the font design. Let’s say, you want the font to be rounded. You can write the font modifier like this:

.font(.system(.largeTitle, design: .rounded))

Here you specify to use the system font with largeTitle text style and rounded design. The preview canvas should immediately respond to the change and show you the rounded text.

Dynamic Type is a feature of iOS that automatically adjusts the font size in reference to the user’s setting (Settings > Display & Brightness > Text Size). In other words, when you use text styles (e.g. .title), the font size will be varied and your app will scale the text automatically, depending on the user’s preference.

In case you want to use a fixed-size font, you can write the line of code like this:

.font(.system(size: 20))

This tells the system to use a fixed font size of 20 points.

As said, you can continue to chain other modifiers to customize the text. Now let’s change the font color. To do that, you can use the foregroundColor modifier like this:

.foregroundColor(.green)

The foregroundColor modifier accepts a value of Color. Here we specify to use .green, which is a built-in color. You may use other built-in values like .red, .purple, etc. 

While I prefer to customize the properties of a control using code, you can also use the design canvas to edit them. Hold the command key and click the text to bring up a pop-over menu. Choose Inspect… and then you can edit the text/font properties.

Using Custom Fonts

By default, all text are displayed using the system font. If you want to use other fonts, you can replace the following line of code:

.font(.system(size: 20))

With:

.font(.custom("Helvetica Neue", size: 25))

Instead of using .system, the code above use .custom and specify the preferred font name. The name of the font can be found in Font Book. You can open Finder > Application and click Font Book to launch the app.

Working with Multiline Text

Text supports multiple lines by default, so it can display a paragraph of text without using any additional modifiers. Replace the code with the following:

Text("Your time is limited, so don’t waste it living someone else’s life. Don’t be trapped by dogma—which is living with the results of other people’s thinking. Don’t let the noise of others’ opinions drown out your own inner voice. And most important, have the courage to follow your heart and intuition.")
    .fontWeight(.bold)
    .font(.title)
    .foregroundColor(.gray)

You’re free to replace the paragraph of text with your own value. Just make sure it’s long enough. Once you made the change, the design canvas should render a multiline text label.

To align the text in centre, insert the multilineTextAlignment modifier and set the value to .center like this:

.multilineTextAlignment(.center)

In some cases, you may want to limit the number of lines to a certain number. You can use the lineLimit modifier to control it. Here is an example:

.lineLimit(3)

By default, the system is set to use tail truncation. To modify the truncation mode of the text, you can use the truncationMode modifier and set its value to .head or .middle like this:

.truncationMode(.head)

After the change, your text should look like the figure below.

Earlier, I mentioned that the Text control displays multiple lines by default. The reason is that the SwiftUI framework has set a default value of nil for the lineLimit modifier. You can change the value of .lineLimit to nil and see the result:

.lineLimit(nil)

Setting the Padding and Line Spacing

Normally the default line spacing is good enough for most situations. In case you want to alter the default setting, you can adjust the line spacing by using the lineSpacing modifier.

.lineSpacing(10)

As you see, the text is too close to the left and right side of the edges. To give it some more space, you can use the padding modifier, which adds some extra space for each side of the text. Insert the following line of code after the lineSpacing modifier:

.padding()

Your design canvas should now show the result like this:

Applying Rotation Effect to Create Perspective Text

The SwiftUI framework provides API to let you easily rotate the text. You can use the rotateEffect modifier and pass the rotation degree like this:

.rotationEffect(.degrees(45))

If you insert the above line of code after padding(), you will see the text is rotated by 45 degrees.

By default, the rotation happens around the center of the text view. If you want to rotate the text around a specific point (say, the top-left corner), you can write the code like this:

.rotationEffect(.degrees(20), anchor: UnitPoint(x: 0, y: 0))

We pass an extra parameter anchor to specify the point of the rotation.

Not only can you rotate thee text in 2D, SwiftUI provides a modifier called rotation3DEffect that allows you to create some amazing 3D effect. The modifier takes in two parameters: rotation angle and the axis of the rotation. Say, you want to create a perspective text effect, you can write the code like this:

.rotation3DEffect(.degrees(60), axis: (x: 1, y: 0, z: 0))

With just a line of code, you already re-create the Star Wars perspective text.

You can further insert the following line of code to create a drop shadow effect for the perspective text:

.shadow(color: .gray, radius: 2, x: 0, y: 15)

The shadow modifier will apply the shadow effect to the text. All you need to do is specify the color and radius of the shadow. Optionally, you can tell the system the position of the shadow by specifying the x and y values.

Summary

Do you enjoy creating user interface with SwiftUI? I hope so. The declarative syntax of SwiftUI made the code more readable and easier to understand. As you can see, it only takes a few lines of code in SwiftUI to create fancy text in 3D style.

If you want to learn more about SwiftUI, please check out our new book – Mastering SwiftUI.

Editor’s note: We’ve added a new tutorial to show you how to animate the perspective text like that in Star Wars.

Hello folks! In a time where the App Store is full of apps, users have more than plenty of options to choose from. There is a lot of competition on all kind of apps, and users want to try them before they decide whether they like them or not. On the other hand, developers target on making some profit out of their published apps, but first they need to build an audience for that. Releasing an app as a paid one is not something that guarantees a financial success; unless the app is doing something extraordinary, chances that users will pay for it are low. Thankfully, there is a solution which is satisfying for both sides so users can try apps and developers see some profit; it’s called In-App Purchases.

By offering in-app purchases to an app, we, as developers, can keep content or features locked, hidden or unavailable from users unless they pay for it. Users on their side are happy because they can have a taste of the app using its free parts, and they’ll be willing to buy the premium content if they’re satisfied by it.

Any purchasable digital item through in-app purchases is called a product. App Store offers four different kind of products:

1. Consumable: These are products which can be bought again and again after users have consumed them.
2. Non-consumable: These are products bought just once. In subsequent app installations users do not pay again to acquire them; instead these products are restored from the App Store.
3. Auto-renewable subscriptions: Users can buy content or features for a certain period of time. Upon expiration, subscription is renewed automatically. Users have always the option to cancel.
4. Non-renewing subscriptions: Similar as above, but the subscription does not renew automatically. The content of the in-app purchase also differs.

In this tutorial we won’t discuss at all about subscriptions; We’ll focus on consumable and non-consumable products only because it would be impossible to cover everything in just one tutorial.

When using IAPs, it’s possible to have additional downloadable content that users get only after paying for it. This content can be stored either on your server or in Apple’s servers. That’s also a case which we won’t cover, but I invite you to extend what we’ll do in this tutorial when finishing it and add any missing features if you want so.

Integrating and providing in-app purchases to an app is not a difficult task; it just includes several steps in the way. If you haven’t done that before then you might find it complicated, but trust me that it’s not. Keep reading to learn about the demo application we’ll be using today, and get prepared to implement your first in-app purchases!

About The Demo App

The demo application we’ll be building on in this tutorial is about a part of a hypothetical, fake game. Among the hypothetically provided features, it offers three in-app purchases (these are for real) that let users do the following:

1. Buy extra lives.
2. Buy extra super powers.
3. Unlock all maps in the game.

You’re invited to unleash your imagination and write the plot of the game!

The first two purchases regard consumable products, meaning products that can be bought again and again once they’re consumed. The last one about unlocking all maps, is a non-consumable product which can be bought only once by users.

In order to keep things simple, contents are displayed in a table view with three cells. You can see in the above screenshot that the first two regard the consumable purchases, while the last one the non-consumable one. Tapping on the first two cells has a double role:

• If no extra lives or super powers have been purchased, then by tapping on them the in-app purchase process will begin.
• If extra lives or super powers have been purchased, then tapping on the matching cells will decrease the number of the available lives or super powers respectively until they become zero. Then they’ll be available to be bought again.

As you understand, our goal here is to make the in-app purchases working and the process described right above to be fully functional. Most importantly, by finishing the IAP related implementation, we’ll end up with a class that can be reused as-is or with slight modifications on any project after that.

There’s a starter project for you to download. It contains all the parts of the demo app which are not directly related to in-app purchases already implemented. When you get it, open it in Xcode. For your information, the project was made in Xcode 11.1.

Even though the demo project is really simple, it’s built based on the MVVM architecture so it’s easier to focus on the various parts of it. Take a look around and familiarize yourself with it. You might find interesting the GameData struct which is defined in the Model class in the Model.swift file. The gameData instance of this structure keeps the data that regard the purchased products. GameData structure adopts the SettingsManageable protocol for saving the values of its properties locally. That protocol was presented on a previous tutorial about how to use protocols to manage app configuration.

Note: The way you’ll store any data regarding in-app purchases is always a matter totally dependent on your app itself and the available mechanisms to save data locally. What I strongly advice to avoid is using User Defaults as your storage solution.

With the above said, get ready to learn how to implement in-app purchases into your apps; it’s going to be an interesting exploration from start to finish!

Preparing In-App Purchases On The App Store

The first thing we have to do before we start writing code in Xcode is to create the in-app purchase products that we’ll be offering through the app. This is an action taking place on App Store, where we are going to create a new app (or app record) on it. Besides that, there are a couple of other steps included in the whole process of getting prepared for in-app purchases integraton.

In summary the actions we’ll take before we switch to Xcode are:

1. Create a new app identifier.
2. Take care of any pending agreements on the App Store.
3. Create test users for the in-app purchases.
4. Create a new app record on the App Store.
5. Create the actual in-app purchases.

Let’s see them now in detail.

Creating A New App Identifier

An app identifier is a unique string value that identifies an application on the App Store. It refers to a specific application and it’s related to it through the Bundle identifier of the app. You can find it in the Project Editor under the General tab in Xcode; it’s the com.appcoda.FakeGame value in the starter project.

An app identifier is necessary when creating new apps for the App Store. Let’s create one now, therefore get connected to your Apple developer account in your favorite browser. Once you’re in, click on the Certificates, IDs & Profiles link in the options on the left.

In the next screen, click on the Identifiers menu item, and then click on small blue plus button to create a new one.

You’ll be presented with a list of various types of identifiers to choose from. What we need is the first one, so make sure to click on the App IDs and then press Continue.

It’s time to create the identifier. Make sure to fill in two fields:

1. The Description of the app identifier. Feel free to provide any description you think it’s best, just respect the limitations shown right below the field. For example, what I wrote in that field is: “FakeGame App ID for IAP Demo App by AppCoda“.
2. Next, in the Bundle ID keep the Explicit radio selected, and copy-and-paste the Bundle identifier from Xcode to that field.

Finally, click Continue and then in the last step click Register. By going back to the list of the app identifiers, you should be able to see the one that we just created!

Fix Any Pending Agreements

Let’s leave the developer’s account now and let’s go to the App Store, also known as the iTunesConnect. If you did not sign out from the developer’s account then you’ll be automatically connected, otherwise just provide your credentials to sing in again. Here’s the place where we’ll create a new record for our application and we’ll setup the in-app purchases, as well as the place to perform a few more necessary actions. One of them is to check if you have any pending agreements that should be taken care of, like for example the next one:

Notifications like the one illustrated right above contain links to the page you should visit so you can accept the agreement(s). You can also go there on your own if you select the Agreements, Tax, and Banking options on the home screen of the App Store:

Once you get there, find the agreement or any other pending task (such as setting up a bank account) and proceed to the suggested or required actions so you eliminate all issues.

Even though it’s not necessary to have the above fixed when you’re creating and testing in-app purchases, you still have to take care about it if you’re planning to send your app either for testing to TestFlight, or to release it.

Create Test Users

Until an app is released on the App Store, all in-app purchases should be tested in a sandbox mode and neither you nor anyone else testing should pay with real money. By default, external testers who test in-app purchases through TestFlight don’t actually pay when they are asked to. However, internal testers like the developers of an app should be using test user accounts and not their real Apple ID and iCloud accounts.

Creating test users in the App Store it’s easy, however there’s a negative point: Even though we’re talking about fake accounts, real email addresses are required. A confirmation email is sent by Apple which should be validated before any test account is used!

So, if you want to have multiple test users it might be a bit of a hassle to have the same number of email addresses. I’d recommend to check if your email service provider allows to use aliases along with your normal email address (as GMail does for example). If you have a paid server service (shared hosted, dedicated, or a personal one), then things are easier for you as you can create as many temporary email addresses as you want if you don’t like using aliases, and then delete them easily.

Onto the actual process now, select the Users and Accesses option in the home screen of the App Store. In the next one, you’ll find out a section called Sandbox to the left menu, and right below a link titled Testers:

By clicking on it you’ll be taken in the test users page. There’s a blue plus button on top that you should press. You then fill in the information for the new tester.

If you’re about to create multiple users, then I’d suggest to choose different App Store territories so you can test in-app purchases with different currencies. Also, make sure to remember the password you set, because there’s no way to edit this form again. You’ll have to start over if you forget a test user’s password.

When you’re done typing the test user’s information, click on the Invite button and wait for a confirmation email to come. Repeat the process by completing a new form for each new tester account you need to add.

Created test accounts are listed as shown next. You can remove them by clicking on the Edit button on the top-right side of the window, but you cannot edit them.

Create A New App On The App Store

Let’s go now to some more juicy stuff, and let’s create a new app record on the App Store which we’ll connect to the actual iOS application. Get started by clicking on the My Apps option on the home screen of the App Store. Then, click on the Plus button on the top-left side of the top bar.

In the form that shows up, there are four fields that have to be mandatorily filled in:

1. The name of the app. Make sure to provide a unique app name! If you provide a name already taken you’ll see an error message when you’ll try to create the app, so just go and change it.
2. The primary language the app is using.
3. The Bundle ID. Use the drop down menu to locate the app ID we created earlier which is connected to the Bundle Identifier of the app.
4. The SKU – a unique string for the app not visible on the App Store.

Also, make sure to select the iOS checkbox in the Platforms section.

Right below you can see the form completed. Use it as a guide and fill it in on your side too.

If there’s no missing data and the name you provided is unique, then a new app will be created right after you click on the Create button. You’ll automatically be navigated to the app information page:

Adding In-App Purchases

The most wanted time is finally here! In this part we’ll create the in-app purchases that our app is going to offer. Before we do that, let’s recap on what exactly we’re going to provide:

• A consumable in-app purchase for buying three lives to use in the game.
• A consumable in-app purchase for buying two super powers.
• A non-consumable in-app purchase for unlocking all game maps.

With all the above in mind, let’s start creating them. Click on the Features link in the top bar, and then make sure that the In-App Purchases is selected on the left menu options.

The main area of the screen is where in-app purchases entries are going to be listed. At the time being there’s none, so click the blue plus button to add one. The following popup will ask you to select the kind of in-app purchase you want to create, so click on the Consumable radio button and then on Create.

A new empty form to fill in is appearing once again. Let’s go through its fields and what they’re all about:

• Reference Name: It’s the name of the in-app purchase on the App Store, but it’s for internal use only. It won’t be shown to the users so don’t worry too much about the value you will provide here. However, give a name that makes clear what this in-app purchase regards. For example, “Extra Lives” (without the quotes) is a good name to make us understand that this one is about the additional lives a user can buy in the game.
• Product ID: This must be a unique string (alphanumeric as Apple says) that will be used for reporting, but here’s a recommendation: Use the app’s bundle identifier as a prefix to the ID value you will specify here. That way you ensure that it’ll always be unique. In our case, “com.appcoda.fakegame.extra-lives” (without quotes) is a unique product ID. Important: Note the product IDs you create here somewhere, we’re going to need them later on.
• Cleared for Sale: Keep it always selected if you want the in-app purchase to be available to the public.
• Pricing: Select the price you want for your in-app purchase. Since this is just a demo, choose any price you would like from the drop down menu. Scroll to bottom to find alternative prices as well.
• App Store Information – Display Name: This is the name of the in-app purchase as it will be shown to users in the app. Note this: For each supported language in your app, you should provide a localized version of this and the next field as well. The value I set here for the first in-app purchase is “Get Extra Lives“.
• App Store Information – Description: A description of the in-app purchase publicly shown, but also optional to be presented by the app. I would recommend to always show it to your users so they can get more details on what they’re about to purchase. For example: “Acquire three (3) additional lives!“.

Here it is completed:

By scrolling to the bottom of the page you’ll notice two more sections lying there:

• App Store Promotion (Optional): By default, right below the app’s name on the App Store there will be a string saying something like: “Free – Offers In-App Purchases”. However, if you want to advertise the offered in-app purchases in the app’s page on the App Store, then provide a promotional image as described by Apple.
• Review Information: This is not required when implementing and testing in-app purchases, but it’s required when an in-app purchase is about to be reviewed either for releasing it to the App Store along with the app, or for TestFlight testing. Review Notes are not mandatory to be provided, however a Screenshot is needed. You can take a screenshot of the app where in-app purchases are offered and upload it, it’ll be suffice. For now, however, leave it empty; we can proceed without it.

When you finish providing the in-app purchase details, click on the Save button you will find on the top-right side of the form. Then go back and start creating the second in-app purchase. Select a consumable IAP again, and fill in the form using the following information:

• Reference Name: Super Powers
• Product ID: com.appcoda.fakegame.superpowers (change it according to your own Bundle ID)
• Price: Any price you want
• Display Name: Additional Super Powers
• Description: Get two (2) additional super powers!

Save this in-app purchase, and then create the last one. This time, select a non-consumable one:

On the contrary of the previous two, this is a kind of purchase that each user will make once. However, this doesn’t change the way it’s being set up. The exact same kind of data must be given in this case too:

• Reference Name: Unlock All Maps
• Product ID: com.appcoda.fakegame.unlock_maps (change it according to your own Bundle ID)
• Price: Any price you want
• Display Name: Unlock Maps
• Description: Unlock all maps in the game forever!

The in-app purchases that our app will be offering are now ready and you can find them all together listed in the IAP home page:

You’ll notice that the Status of all in-app purchases is set to Missing Metadata. That’s because we didn’t set a review image to any of them. Don’t worry though, by adding a review image and saving the status will change to Waiting For Review. Such an action isn’t necessary here; we’re not going to publish this demo app.

Using Product IDs In Xcode

Finally, all necessary preparation has come to its end. It’s now time to leave App Store and go to the starter project in Xcode. The final goal in this post is to create a reusable class that will manage in-app purchases, but this class will need to know about the available product identifiers created on the App Store. So, let’s start with that and let’s add all IAP product identifiers to a special file you’ll find in Xcode under the In-App Purchases group, called IAP_ProductIDs.plist.

The purpose of this file is to let us keep product identifiers gathered in one place, in a simple and code-unrelated fashion. The class we’ll implement next will get all product identifiers by just reading the contents of this file.

So, open IAP_ProductIDs.plist in Xcode and make sure that the type of the Root item is set to Array. Then, add three items one after another, and each time copy and paste the product ID of a different in-app purchase created earlier. In the end you should end up with this:

Start Implementing The In-App Purchases Managing Class

Let’s focus on implementing the reusable class now that will manage in-app purchases in our demo app and not only. In the starter project in Xcode, open the file called IAPManager.swift which you’ll also find under the In-App Purchases group in Project Navigator. It’s currently empty, but we’ll change that here and in the following parts.

The first move is to import the StoreKit framework; it’s the one that will allow us to deal with all in-app purchase related concepts and entities in programming level. Right after the first import statement add the following:

import StoreKit

Let’s declare the new class now which will have the same name to the file: IAPManager. Leave a couple of empty lines and add this:

class IAPManager: NSObject {

}

Note: Later on the IAPManager class will adopt a protocol called SKPaymentTransactionObserver. This protocol requires any conforming types to also conform to NSObjectProtocol, and that’s something we can do with no hassle simply by just making IAPManager a subclass of the NSObject class.

In order to keep things simple and to avoid potential troubles by having multiple instances of this class, we’ll apply the Singleton pattern and we’ll be using one instance only, the shared instance. Adopting the Singleton pattern requires two things:

1. To have an instance of the class initialized as a static property.
2. Keep the initializer private so no more instances of the class can be created anywhere in the app.

Here it is:

class IAPManager: NSObject {
    static let shared = IAPManager()

    private override init() {
        super.init()
    }
}

As you will see next, there will be cases where the various operations won’t return the desired or expected results. It’s important to treat these cases gracefully and make IAPManager class indicate them properly in any custom type using it. For that purpose, we’re going to create the following enum with a few custom errors:

enum IAPManagerError: Error {
    case noProductIDsFound
    case noProductsFound
    case paymentWasCancelled
    case productRequestFailed
}

Make sure to add the enum inside the class body. The meaning of them:

• noProductIDsFound: It indicates that the product identifiers could not be found.
• noProductsFound: No IAP products were returned by the App Store because none was found.
• paymentWasCancelled: The user cancelled an initialized purchase process.
• productRequestFailed: The app cannot request App Store about available IAP products for some reason.

Along with the above enum, it’s necessary to add the following extension right after the closing of the IAPManager class. In it, the localized descriptions of the custom errors are specified:

extension IAPManager.IAPManagerError: LocalizedError {
    var errorDescription: String? {
        switch self {
        case .noProductIDsFound: return "No In-App Purchase product identifiers were found."
        case .noProductsFound: return "No In-App Purchases were found."
        case .productRequestFailed: return "Unable to fetch available In-App Purchase products at the moment."
        case .paymentWasCancelled: return "In-App Purchase process was cancelled."
        }
    }
}

Reading Product Identifiers

Before we perform any action related to in-app purchases, it’s necessary to get the product identifiers that we previously added to the IAP_ProductIDs.plist file. For that purpose, we’ll implement a small helper method that will do just that: Reading the property list file from the app bundle and returning the product identifiers as an array of String elements.

fileprivate func getProductIDs() -> [String]? {

}

The method is marked as fileprivate because we want it to be visible in this file only. There’s no reason to be accessible by other entities out of this class. However, remove the fileprivate keyword if you ever need to have the product identifiers available somewhere else besides here.

The first step in this method is to get a URL pointing to the IAP_ProductIDs.plist file in the app bundle:

guard let url = Bundle.main.url(forResource: "IAP_ProductIDs", withExtension: "plist") else { return nil }

Always make sure to type the file name and its extension correctly when using the above method; a typo error is enough to make you lose time and wonder why the file cannot be found even though it exists in the project. If the file is found, its URL is assigned to the url property.

Next, we’ll load the file contents to a Data NSObject:

do {
    let data = try Data(contentsOf: url)
} catch {
    print(error.localizedDescription)
    return nil
}

Initializing a Data object as shown above can throw an exception, so including it in a do-catch statement is necessary. In case the file cannot be read and the data property to be initialized, then we print the description of the error occurred, and we return nil.

Since the above data object was created by using the contents of a property list file, we’ll decode it and convert it to an array of String elements with the help of the PropertyListSerialization class; it allows to encode to and decode from property list objects. If you’re familiar to the JSONSerialization class, then this one is the equivalent for property lists, not JSON objects.

Still inside the do-catch statement, let’s convert the loaded property list data into an array as shown right below:

let productIDs = try PropertyListSerialization.propertyList(from: data, options: .mutableContainersAndLeaves, format: nil) as? [String] ?? []

In case it’s not possible to convert to a collection of String values, then we just assign an empty array to productIDs.

Finally, we need to return it:

return productIDs

Here’s the whole method:

fileprivate func getProductIDs() -> [String]? {
    guard let url = Bundle.main.url(forResource: "IAP_ProductIDs", withExtension: "plist") else { return nil }
    do {
        let data = try Data(contentsOf: url)
        let productIDs = try PropertyListSerialization.propertyList(from: data, options: .mutableContainersAndLeaves, format: nil) as? [String] ?? []
        return productIDs
    } catch {
        print(error.localizedDescription)
        return nil
    }
}

Requesting App Store For Available IAP Products

Making IAPManager capable of loading the product IDs, the next step is to fetch all available products offered for purchase from the App Store. This action will return a collection of SKProduct objects, where each one describes an in-app purchase and contains its details as it was configured on the App Store.

We’ll start by defining a new method:

func getProducts(withHandler productsReceiveHandler: @escaping (_ result: Result<[SKProduct], IAPManagerError>) -> Void) {

}

Let’s talk a bit about the parameter of this method, and let me start with a question: Are you aware of the Result type that was first-introduced in Swift 5? Citing the official documentation:

A value that represents either a success or a failure, including an associated value in each case.

In simple words, this type makes it easy to return the outcome of an operation and to indicate whether it was successful or not. On success, it’s possible to carry any necessary custom data; on failure it carries the error that caused the operation to fail.

In our case, our Result value will carry the collection of fetched products from the App Store if we get them successfully, and a custom error type (IAPManagerError) on failure.

Fetching IAP products from the App Store is an asynchronous process. That means that the above method cannot return the products fetching result instantly. So, the parameter of the method has to be a closure (or a callback handler in other words) which will be called when StoreKit notifies our class that has got a response from the App Store. The parameter of that closure is the Result value as shown in the method definition above.

The above borns a new requirement now: To declare a property in the IAPManager class which will keep a reference to the handler (closure) even when the execution of the getProducts(withHandler:) method is finished. Go to the beginning of the class in the properties declaration area and add the following:

var onReceiveProductsHandler: ((Result<[SKProduct], IAPManagerError>) -> Void)?

Back to the getProducts(withHandler:) method now. We start implementing it by assigning the productsReceiveHandler parameter value to the onReceiveProductsHandler property so we can call it at any time later in the future:

onReceiveProductsHandler = productsReceiveHandler

Next, let’s get the collection of the identifiers for the products that we want to fetch data for. For this purpose, we’ll make use of the getProductIDs() method we implemented earlier:

guard let productIDs = getProductIDs() else {
    productsReceiveHandler(.failure(.noProductIDsFound))
    return
}

Remember that it’s possible for the above to return nil. If that happens, then the wanted product identifiers could not be read for some reason and we should return from this method immediately. However, it’s necessary to notify the caller of the class about the error that occurred. We call the productsReceiveHandler handler and we pass a Result type with a failure indicating the exact error happened.

Note: If you find yourself being uncomfortable by the syntax of the Result type, then I’d recommend to search for more usage examples on the Internet, as well as to play with it in a playground until you master it.

Continuing to the implementation, let’s initialize a products request for the App Store:

let request = SKProductsRequest(productIdentifiers: Set(productIDs))

The initializer shown above awaits for a Set of product identifiers, not an array. That’s why we initialize a new Set using the productIDs array.

As mentioned already, requesting the App Store is not a synchronous operation. The result of it, meaning the response from the App Store, is available through a couple of methods provided by the SKProductsRequestDelegate and SKRequestDelegate protocols. IAPManager class will adopt them, but first it must be set as the request’s delegate:

request.delegate = self

We are now ready to make the request:

request.start()

The getProducts(withHandler:) method is a quite important one, and even though it’s small, it contains vital steps for making in-app purchases possible. Here it is all together:

func getProducts(withHandler productsReceiveHandler: @escaping (_ result: Result<[SKProduct], IAPManagerError>) -> Void) {
    // Keep the handler (closure) that will be called when requesting for
    // products on the App Store is finished.
    onReceiveProductsHandler = productsReceiveHandler

    // Get the product identifiers.
    guard let productIDs = getProductIDs() else {
        productsReceiveHandler(.failure(.noProductIDsFound))
        return
    }

    // Initialize a product request.
    let request = SKProductsRequest(productIdentifiers: Set(productIDs))

    // Set self as the its delegate.
    request.delegate = self

    // Make the request.
    request.start()
}

Handling App Store Response

Right above we set the IAPManager class as the delegate of the SKProductsRequest object (the request). Now, it’s mandatory to adopt the SKProductsRequestDelegate protocol and implement at least one required method.

Go after the closing curly bracket of the IAPManager class, and add the following extension:

extension IAPManager: SKProductsRequestDelegate {

}

In it we’ll implement the following method which gets called when the App Store sends back a response to the original request:

func productsRequest(_ request: SKProductsRequest, didReceive response: SKProductsResponse) {

}

The first parameter value regards the request that triggered the response. The second (the response) is what we really care about here, as it contains a collection of SKProduct objects; the available content for purchase. Each product contained in the response matches to a single product identifier in the list of identifiers existing in our property list file.

Going into the logic that we’ll apply here, at first we’ll get the collection of products:

let products = response.products

Then we’ll check if there are any products returned or not. If so, we’ll call the onReceiveProductsHandler and we’ll pass a Result value indicating a success and supplying it with the array of products:

if products.count > 0 {
    onReceiveProductsHandler?(.success(products))
} else {

}

Among others, the above illustrates how a Result value with the case of success is being constructed.

In case there are no products returned, we’ll call the onReceiveProductsHandler once again, but this time the Result value will indicate an error, and we’ll pass the noProductsFound custom error as its associated value:

onReceiveProductsHandler?(.failure(.noProductsFound))

Even though finding no products is not necessarily an error (when, for example, there are no IAP entries on App Store or the given product identifiers do not match to any IAPs), treating it the way shown above ensures that the two cases of having and not having products will be handled separately by the caller of this class.

Here’s the entire method:

func productsRequest(_ request: SKProductsRequest, didReceive response: SKProductsResponse) {
    // Get the available products contained in the response.
    let products = response.products

    // Check if there are any products available.
    if products.count > 0 {
        // Call the following handler passing the received products.
        onReceiveProductsHandler?(.success(products))
    } else {
        // No products were found.
        onReceiveProductsHandler?(.failure(.noProductsFound))
    }
}

Note: The response object provides a property called invalidProductIdentifiers. It’s a collection of identifiers regarding products that are not valid to be purchased. Although we do not use it here, keep it in mind in case you ever need it.

What the above method does not do is to let us know if the request failed for some reason. We can deal with that by implementing another delegate method:

func request(_ request: SKRequest, didFailWithError error: Error) {

}

Regardless of the error that caused the request to fail, it’s necessary to let the IAPManager caller know about it. For one more time we’re going to call the onReceiveProductsHandler closure passing the productRequestFailed custom error.

onReceiveProductsHandler?(.failure(.productRequestFailed))

Finally, there’s one more delegate method which can be optionally implemented:

func requestDidFinish(_ request: SKRequest) {

}

We won’t use it here, but be aware of it in case there’s any additional custom logic you want to add.

Getting Product’s Price As A String

In the next part we’re going to have a first taste about everything we’ve done so far, but before that it’s necessary to implement a small assistive, yet necessary method in the IAPManager class. Its purpose is to return a product’s price as a formatted currency string. The implementation shown right next is taken as-is from Apple’s documentation but it’s not hard to understand it:

func getPriceFormatted(for product: SKProduct) -> String? {
    let formatter = NumberFormatter()
    formatter.numberStyle = .currency
    formatter.locale = product.priceLocale
    return formatter.string(from: product.price)
}

Notice that the string is created based on the product’s localized price value which is taken by accessing the priceLocale property.

Testing What Is Built So Far

Let’s take a small break from building the IAPManager class now, and let’s make a few updates to other parts of the app so we can run it and test what we’ve done so far. There are two things that we’ll do here:

1. We’ll fetch the available in-app purchases from the App Store (the products).
2. We’ll display product information while trying to initiate a buy process.

Note: Since the goal of this post is to focus on the creation of a reusable component to perform in-app purchases, I’ll save us some time by being less thorough when presenting missing parts of the app which are not related directly to the IAPManager class. Please take your time and have a good look at the project if you need so.

Fetching IAP Products

Before users are able to make any in-app purchase, the app must fetch all information about available IAPs from the App Store. There’s no rule when exactly apps should do that, it always depends on each specific app itself. Sometimes it’s suitable to fetch products right before an in-app purchase is started. Some other times it’s better to fetch them right after the app has launched.

In this demo application, and given the fact that we have one view controller only, we’ll fetch the available IAP products right after the view of the view controller has appeared and it’s ready to be used. If you check the viewDidAppear(_:) method in the ViewController.swift file, you’ll see that it calls the viewDidSetup() method of the ViewModel instance; that is place where we’ll initiate the product fetching.

Open the ViewModel.swift file and locate the viewDidSetup() method. When there are no products fetched recently by the App Store, no in-app purchase process can start, so users have to wait a bit. Let’s do that, and let’s trigger the appearance of an overlay view with an activity indicator in it while the app “talks” to the App Store and requests for any in-app purchases:

func viewDidSetup() {
    delegate?.willStartLongProcess()
}

Now we can use the getProducts(withHandler:) we implemented earlier in the IAPManager class. Remember that IAPManager is a singleton class therefore we access the method through its shared instance:

IAPManager.shared.getProducts { (result) in

}

We can remove the overlay view shown in the app when the fetching process is finished and the above closure gets called. Just remember to do that on the main thread.

DispatchQueue.main.async {
    self.delegate?.didFinishLongProcess()
}

Let’s handle the result now. If it’s a success, then it’ll contain the fetched IAP products; if it’s a failure it’ll carry the error that caused it. In the first case we keep the fetched products in the products property of our model; in the second we trigger the appearance of an alert that will display the localized description of the returned error:

switch result {
    case .success(let products): self.model.products = products
    case .failure(let error): self.delegate?.showIAPRelatedError(error)
}

Find the implementation of the showIAPRelatedError(_:) delegate method in the ViewController.swift file to find out how the error is being displayed.

Run the app now if you want (either in Simulator or on a device) and look the overlay view with the spinner appearing while the app is fetching the in-app purchase information from the App Store. If it just disappears after a few moments, then products have been successfully fetched and kept by the app. If any error occurs, you’ll see an alert showing up with the error description.

Here’s the entire method we just implemented:

func viewDidSetup() {
    delegate?.willStartLongProcess()

    IAPManager.shared.getProducts { (result) in

        DispatchQueue.main.async {
            self.delegate?.didFinishLongProcess()

            switch result {
            case .success(let products): self.model.products = products;
            case .failure(let error): self.delegate?.showIAPRelatedError(error)
            }
        }
    }
}

Note: The way you’ll temporarily keep fetched IAP products in your app is totally up to you. Here I chose to create the products property in the Model class, which is an array of SKProduct elements.

Displaying Product Information

By having the available in-app purchase products at the app’s disposal, we can use them for presenting certain information to users when a purchase process is about to start. To be specific, we’ll be displaying the following for each product a user wants to purchase:

• The localized title.
• The localized description.
• The price.

Remember that all of them have been set on the App Store. And here’s something to avoid: Do not hardcode any title, description or price into the app; all these can change at any time if you edit your in-app purchase records on the App Store.

Let’s focus on our demo app again now. We will initiate a purchase process when a user taps on a cell and any of the following is true:

• The first cell is tapped and there are no extra lives available.
• The second cell is tapped and there are no super powers available.
• The third cell is tapped (Note: Third cell won’t be visible after unlocking all maps by buying the matching product; it regards a non-consumable product).

This logic is already applied in the tableView(_:didSelectRowAt:) table view delegate method in the ViewController.swift file. Go there and take a look now. When any of the above conditions is true, the getProductForItem(at:) method of the ViewModel is called in order to return the matching product to the tapped cell. That product is given as an argument to the showAlert(for:) method defined in the ViewController class.

The purpose of the showAlert(for:) method is to display an alert controller which will contain all the product details mentioned above right before an in-app purchase is made. It actually displays an alert that asks users if they want to continue with the purchase of the described item.

Note: I would recommend against of using a default alert controller for displaying a product’s information, unless your app is implements a really basic UI. You’d better describe products in custom designed views that match to the look and feel of your app.

Locate the showAlert(for:) method in the ViewController.swift file. We’ll start implementing it by getting the localized price of the product given as an argument:

func showAlert(for product: SKProduct) {
    guard let price = IAPManager.shared.getPriceFormatted(for: product) else { return }
}

Let’s initialize an alert controller now. We’ll display the product’s title as the alert’s title, and the product’s description as the alert’s message:

let alertController = UIAlertController(title: product.localizedTitle,
                                        message: product.localizedDescription,
                                        preferredStyle: .alert)

The product’s price will be used as part of the title of an alert action that will prompt users to buy the item. Here it is:

alertController.addAction(UIAlertAction(title: "Buy now for \(price)", style: .default, handler: { (_) in
    // TODO: Initiate Purchase!
}))

The button’s title will say something: “Buy now for $0.49”. When users tap on it, the actual purchase process will be initiated, but that’s for later. For now we just leave a “TODO” comment.

Finally, let’s add a cancel action to the alert, and let’s present it:

alertController.addAction(UIAlertAction(title: "Cancel", style: .cancel, handler: nil))
self.present(alertController, animated: true, completion: nil)

Here’s the entire method:

func showAlert(for product: SKProduct) {
    guard let price = IAPManager.shared.getPriceFormatted(for: product) else { return }
    let alertController = UIAlertController(title: product.localizedTitle,
                                            message: product.localizedDescription,
                                            preferredStyle: .alert)

    alertController.addAction(UIAlertAction(title: "Buy now for \(price)", style: .default, handler: { (_) in
        // TODO: Initiate Purchase!
    }))

    alertController.addAction(UIAlertAction(title: "Cancel", style: .cancel, handler: nil))
    self.present(alertController, animated: true, completion: nil)
}

Run the app now and tap on any cell. You should see the alert we just created is showing up and describing the selected product! Of course no purchase can take place now, and that’s something that we’re going to change right now!

Implementing Purchases

In programming level, a purchase is a transaction (aka a SKPaymentTransaction object), and responsible for managing transactions is a class called SKPaymentQueue. The payment queue is the one which communicates with the App Store and handles all the payment process. It’s also responsible for presenting the built-in UI that appears when performing purchases.

Every app that wants to offer in-app purchases is required to add a payment transaction observer to the queue (a SKPaymentTransactionObserver object) so it’s capable of catching any updates regarding a purchase. As you understand, making purchases is an asynchronous process just like fetching the products information from the App Store. The outcome of a transaction should always become absolutely clear to users, regardless of whether it was successful or not.

Observing The Payment Queue

So, with that new information in mind, let’s go back to the IAPManager.swift file where we’ll implement two new small methods. With them we’ll be adding and removing a payment transaction observer to the payment queue respectively:

func startObserving() {
    SKPaymentQueue.default().add(self)
}


func stopObserving() {
    SKPaymentQueue.default().remove(self)
}

Just implementing the above two is not enough. It’s also necessary to call them appropriately so the app can start and stop observing for payment transaction updates as needed.

Open the AppDelegate.swift file. In the application(_:didFinishLaunchingWithOptions:) method make call to startObserving() so the app starts observing each time is launched:

func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
    IAPManager.shared.startObserving()
    return true
}

Next, implement the applicationWillTerminate(_:) to stop observing:

func applicationWillTerminate(_ application: UIApplication) {
    IAPManager.shared.stopObserving()
}

That’s it. The app is now capable of watching properly for updates in the payment queue, and we are ready to move forward and make it possible to create new transactions (meaning to make new purchases), as well as to handle any transaction related updates coming from the App Store.

Note: Don’t be bothered by the couple of errors currently shown by Xcode; they’ll be fixed soon.

Checking For IAP Eligible Device

Before an app triggers a purchase process, it’s important to know whether in-app purchases are actually allowed or supported on a device. It’s possible, for example, that parents have turned off IAPs through the device settings. That’s a state that apps must always check for and notify users accordingly.

The information of whether in-app purchases can be made is something that’s taken from the payment queue once again. Go to the IAPManager.swift file and implement the following method:

func canMakePayments() -> Bool {
    return SKPaymentQueue.canMakePayments()
}

As you see, it’s as simple as that. We will call canMakePayments() method later on, when we’ll keep adding the missing parts from the demo app that will make purchases possible.

Making Purchases

With the payment queue being observed and with the ability to know whether a device can actually make payments or not, let’s go to the implementation of a new method that triggers a new purchase:

func buy(product: SKProduct, withHandler handler: @escaping ((_ result: Result<Bool, Error>) -> Void)) {
    let payment = SKPayment(product: product)
    SKPaymentQueue.default().add(payment)

    // Keep the completion handler.
    onBuyProductHandler = handler
}

This method accepts two arguments:

1. The first one is the actual product (a SKProduct object) that is about to be purchased.
2. The second parameter is a callback handler ( a closure) similar to the one we met in the getProducts(withHandler:) method. It has a Result type as a parameter with a boolean value as the associated value for the success case, and an error object as the associated value for the failure case. This callback handler is necessary because making a payment is an asynchronous operation; it’ll be called when the payment process is finished.

Inside the method now, we initialize a payment object (SKPayment) using the given product. Then, we add the payment object to the payment queue and we’re done! Once a new payment object is added to the payment queue, things will start working without any other action on our behalf, so we just have to wait for the entire process to come to its end.

The callback handler that is given as an argument to this method should be kept in a class property, so it’s available when the method’s execution is finished. This is what’s happening in the last line above; the handler parameter value is assigned to the onBuyProductHandler. However, the second one does not exist in the IAPManager class yet.

Go to the properties declaration area of the class and add the following line:

var onBuyProductHandler: ((Result<Bool, Error>) -> Void)?

Handling Transaction Results

Now that we made it possible to make a payment, let’s handle the results of a transaction. For this purpose, we need to adopt the SKPaymentTransactionObserver protocol and implement a required method.

Go after the closing curly bracket of the IAPManager class once again, and add one more extension:

extension IAPManager: SKPaymentTransactionObserver {

}

In it we’ll implement the following method:

extension IAPManager: SKPaymentTransactionObserver {
    func paymentQueue(_ queue: SKPaymentQueue, updatedTransactions transactions: [SKPaymentTransaction]) {

    }
}

This method is called when the state of a transaction changes. The second parameter value, transactions, is an array of SKPaymentTransaction objects as multiple transactions can be running at the same time. We’ll start with that and we’ll go through each transaction:

transactions.forEach { (transaction) in

}

Next, we’ll add a switch statement which we’ll use to examine the various states of each transaction:

switch transaction.transactionState {
case .purchased:

case .restored:

case .failed:

case .deferred, .purchasing: break
@unknown default: break
}

What we are really interested about is the first three cases:

1. purchased: When a payment is finished and the product has been purchased.
2. restored: When a previously purchased item has been restored from the App Store.
3. failed: When a transaction fails.

In the first case where a purchase is successful, we’ll call the onBuyProductHandler closure passing true as the value of the result object. This will notify the IAPManager‘s caller about the successful outcome. On top of that, there’s one more action that’s mandatory to be done; to indicate to the payment queue that it can now consider the transaction as finished.

case .purchased:
    onBuyProductHandler?(.success(true))
    SKPaymentQueue.default().finishTransaction(transaction)

Actually, the last line is something we’ll also do in the next cases too.

We’ll postpone the restored case for a bit later. For now add this:

case .restored: break

In the failed case, we’ll check at first if the transaction’s error object is nil or not. A transaction’s error is a SKError object and it has a specific code that describes the error. There’s a special one called paymentCancelled and it’s happening when a user cancels a purchase right before the actual payment is done. If this is the case, then we’ll pass a failure value to the result with the custom IAPManagerError.paymentWasCancelled error. If not, then we’ll pass failure with the actual error that took place. That way we’ll make it easy for the app to know whether the payment failed because of the user or not.

case .failed:
if let error = transaction.error as? SKError {
    if error.code != .paymentCancelled {
        onBuyProductHandler?(.failure(error))
    } else {
        onBuyProductHandler?(.failure(IAPManagerError.paymentWasCancelled))
    }
    print("IAP Error:", error.localizedDescription)
}
SKPaymentQueue.default().finishTransaction(transaction)

Implementing the capability to make in-app purchases is now complete. Time to try it out!

Enable App To Make Purhases

Open the ViewModel.swift file where you’ll find a defined, yet still empty method called purchase(product:). In this one we’ll add the missing logic which will make use of what we just did in the previous parts and ultimately make payments come true for our demo app.

As you can see the returned value from the method is a boolean one. The only case where this method will return false is going to be when no payments can be made on a specific device. Otherwise it’ll trigger the new payment using the product provided as an argument and it’ll return true. Let’s see that:

func purchase(product: SKProduct) -> Bool {
    if !IAPManager.shared.canMakePayments() {
        return false
    } else {

    }

    return true
}

Here’s the place where we make use of the canMakePayments() method which we implemented earlier.

Focusing on the else case now, we’ll indicate to the View part of our MVVM based app that a long process is about to start. This will make the overlay view with the activity indicator appear:

delegate?.willStartLongProcess()

Let’s trigger the purchase:

IAPManager.shared.buy(product: product) { (result) in

}

We’re calling the buy(product:) method of the IAPManager class passing the product parameter value given to the purchase(product:) as its argument. The above closure is the one assigned to the onBuyProductHandler property in the IAPManager class. It’ll be called back when the purchase will have either finished successfully or not; it’s the result value to tell that.

Since the purchase is an asynchronous operation taking place on the background, whatever we’ll do in the above closure must be done on the main thread. After all, we’re going to initiate UI changes and that’s something that cannot be done using a background thread.

DispatchQueue.main.async {
    self.delegate?.didFinishLongProcess()

    switch result {
    case .success(_): self.updateGameDataWithPurchasedProduct(product)
    case .failure(let error): self.delegate?.showIAPRelatedError(error)
    }
}

By calling the didFinishLongProcess() delegate method we make the overlay view disappear. Next, we examine the result value. On success we call the updateGameDataWithPurchasedProduct(_:) method which is already implemented in the ViewModel class. This is the method which, depending on the purchased product, will add extra lives or extra powers to our fake game (consumable products), or it will mark all maps as unlocked (non-consumable). Go and take a look at updateGameDataWithPurchasedProduct(_:).

If the purchase fails, then we let our view controller know about that and therefore show the error message through the showIAPRelatedError(_:) delegate method. Note that the way a purchase’s results will be handled is totally depending on the app that’s being developed, the logic applied to it, the design, and so on. So, what you see above is something that has a meaning to this demo app only. However, this doesn’t mean you cannot act similarly in your own apps; feel free to use this method as a guide.

Here’s the purchase(product:) all together:

func purchase(product: SKProduct) -> Bool {
    if !IAPManager.shared.canMakePayments() {
        return false
    } else {
        delegate?.willStartLongProcess()
        IAPManager.shared.buy(product: product) { (result) in
            DispatchQueue.main.async {
                self.delegate?.didFinishLongProcess()

                switch result {
                case .success(_): self.updateGameDataWithPurchasedProduct(product)
                case .failure(let error): self.delegate?.showIAPRelatedError(error)
                }
            }
        }
        return true
    }
}

Open the ViewController.swift file now and search for this comment: // TODO: Initiate Purchase!. You should find it in the action of the alert controller that displays the product information that is about to be purchased (showAlert(for:) method). Replace that comment and update the action as shown next:

alertController.addAction(UIAlertAction(title: "Buy now for \(price)", style: .default, handler: { (_) in

    if !self.viewModel.purchase(product: product) {
        self.showSingleAlert(withMessage: "In-App Purchases are not allowed in this device.")
    }

}))

If the purchase(product:) returns false, then the device cannot make payments and we’re showing an alert to let users know about that. Otherwise, things will flow as we built it earlier.

Testing In-App Purchases

In order to test in-app purchases you’ll need a real device. Payments cannot be done using a Simulator. However before you begin make sure to sign out from your iCloud account if you’re already connected in your device. We want all tests to take place on a sandboxed environment.

After you sign out, do not connect to iCloud using a test account. This is not the place to do it! You’ll be asked to sign in when you initiate a purchase process. However, in case you want to set the test account you’ll use in advance, or if you want to change the test user later, then:

• Open Settings.
• Go to iTunes & App Store.
• Scroll to bottom. There’s a section called Sandbox Account. Tap on Sign-In and provide the test user credentials.

Note: Do not provide a real Apple ID here!

In addition to the above, you might want to disallow in-app purchases in your device so you can test that case too. In order to do that, go to Settings and then:

• Screen Time
• Content & Privacy Restrictions
• iTunes & App Store Purchases
• In-app Purchases
• Select Don’t Allow

To enable IAPs again, follow the same steps but at the end select Allow.

Now that you know how to assign or change a test user, and how to turn on and off in-app purchases, compile the app and let it run on your device. Tap on any fake product you want to buy. For example, tap on the extra lives cell and then on the Buy button. Provide the test user’s credentials and confirm the payment. Once it’s successfully done, you’ll see that you have three available lives which you can consume by tapping on the cell. When you’re out of lives, you’ll be asked to buy again.

Note: Pay attention to the system alerts that are showing up. While confirming the purchase you should be seeing the [Environment: Sandbox] indication. If you don’t see that, then you’re not making the purchase using a test account and you should not proceed until you change that.

In case you start the purchase but you cancel it before confirming the buy, here’s what you’ll see:

Note that you can keep buying extra lives and super powers for as long as you want when running out of them. However, unlocking all maps can be purchased just once; it’s a non-consumable product, and when you do so the last cell will no longer be appearing on the table view.

Restoring Purchases

Even though we made it possible to perform in-app purchases and buy digital products through our app, there’s still one thing missing. That is the option to restore previously purchased non-consumable products.

When users pay once for a non-consumable product (like unlocking all maps in our demo app), they should never be asked to pay for it again. However, they should be able to get their purchase back when they install the app on a different device or when they re-install on the same one. Apple requires a restore button to exist in our UI which should be clearly visible and understandable by users. Without the option to restore, get ready to see your app being rejected from the App Store.

Thankfully, App Store keeps record of purchases made on non-consumable products and it will prevent users from paying again for something they’ve already bought. That means that even if someone tries to buy again a non-consumable product will fail; Once StoreKit and the payment queue get the information from the App Store that there is a previous payment, the new process will be considered as a restore and the user won’t be charged for it.

Restoring purchases is something we’re still missing in the IAPManager class, so let’s take care of it now. Open the IAPManager.swift file and add the following new method:

func restorePurchases(withHandler handler: @escaping ((_ result: Result<Bool, Error>) -> Void)) {
    onBuyProductHandler = handler
    totalRestoredPurchases = 0
    SKPaymentQueue.default().restoreCompletedTransactions()
}

Let’s start with the parameter value. Once again it’s the (familiar) callback handler we also had in the buy(product:) method too, so we assign it to the onBuyProductHandler class property for future use.

You might be wondering what the totalRestoredPurchases variable is all about. When having restorable products, it’s good to know whether all of them were restored or not, and by using such a counter you can easily verify it. You’ll see how it’ll be used in a while, but first, declare it in the IAPManager class along with the other class properties:

var totalRestoredPurchases = 0

Finally, by calling the restoreCompletedTransactions() method of the payment queue we’re triggering the process of restoring previous purchases.

Note: Restoring previous purchases regards only non-consumable products. Consumable products are not restored; users might have or might have not consumed them and it’s up to you as a developer to synchronize that information among devices if necessary using cloud solutions (such as iCloud, your server, Firebase, etc).

Now, go to the paymentQueue(_:updatedTransactions:) delegate method in the SKPaymentTransactionObserver extension. If you remember, there’s the restored case there which we had left unmanaged. Since in this method we go through all available transactions, we’ll increase the totalRestoredPurchases value by one each time we have a restored one. At the end we’ll know how many purchases were restored and how many weren’t. Here’s how the restored case should look:

case .restored:
    totalRestoredPurchases += 1
    SKPaymentQueue.default().finishTransaction(transaction)

Notice that calling the finishTransaction(_:) method is necessary so the transaction is marked as finished in the payment queue.

We’ll implement two more delegate methods now in the SKPaymentTransactionObserver extension’s body. The one you’ll see right next is called whenever restoring previous purchases has finished successfully, meaning without any errors, even if there are not any products to restore.

func paymentQueueRestoreCompletedTransactionsFinished(_ queue: SKPaymentQueue) {
    if totalRestoredPurchases != 0 {
        onBuyProductHandler?(.success(true))
    } else {
        print("IAP: No purchases to restore!")
        onBuyProductHandler?(.success(false))
    }
}

Two things to mention here: Firstly, see that in both cases we call the onBuyProductHandler passing success as the result’s value. The associated value of the success cases varies depending on whether we had restored products or not. Secondly, the way totalRestoredPurchases property is used here is something you can change and make it fit to the needs of your own app. In this demo app it’s sufficient to know that totalRestoredPurchases != 0, since we have one non-consumable product only and this condition ensures that it’s restored successfully. If you have more than one restorable products, you might need to change it and calculate the products that were not restored.

The second delegate method is invoked by the payment queue on a failed restore process. Note that a reason for the failure could be users themselves by just cancelling the entire process. You’ll find it familiar as we’re doing the exact same thing as we did in the failed case in the paymentQueue(_:updatedTransactions:) method. Here it is:

func paymentQueue(_ queue: SKPaymentQueue, restoreCompletedTransactionsFailedWithError error: Error) {
    if let error = error as? SKError {
        if error.code != .paymentCancelled {
            print("IAP Restore Error:", error.localizedDescription)
            onBuyProductHandler?(.failure(error))
        } else {
            onBuyProductHandler?(.failure(IAPManagerError.paymentWasCancelled))
        }
    }
}

And that concludes the implementation of the IAPManager class! It’s about time to run our last test.

Enabling App To Restore Purchases

Open the ViewModel.swift file and go to the restorePurchases() method. This one is called by the restorePurchases(_:) IBAction method in the ViewController class. Similarly to what we did before, we’ll indicate a long process to the view controller, and then we’ll initiate the restore process:

func restorePurchases() {
    delegate?.willStartLongProcess()
    IAPManager.shared.restorePurchases { (result) in

    }
}

When the closure gets called, we’ll examine the success value of the result. If it’s true, then restoring the product was successful and we can unlock all maps on our fake game. If not we’ll display an appropriate message, which is also the case when restoring fails.

DispatchQueue.main.async {
    self.delegate?.didFinishLongProcess()

    switch result {
    case .success(let success):
        if success {
            self.restoreUnlockedMaps()
            self.delegate?.didFinishRestoringPurchasedProducts()
        } else {
            self.delegate?.didFinishRestoringPurchasesWithZeroProducts()
        }

    case .failure(let error): self.delegate?.showIAPRelatedError(error)
    }
}

See that we treat separately the case where there are no products to restore (the didFinishRestoringPurchasesWithZeroProducts() delegate method is called). You’re not required, but recommended to do so in your apps as well so you can notify your users properly.

In order to test the restore functionality, first buy the “unlock all maps” in-app purchase and then delete the app from your device. Install it again through Xcode and then tap on the Restore Purchases button using the same test user you used before.

Summary

Offering in-app purchases through an app is not a complicated process, but it involves a lot of steps. Starting with the preparation on the App Store and finishing by adding the last touch in your code consists of a series of actions that must be done really carefully. The detailed parts presented in this post show the way to integrate in-app purchases in your app, and by leaving here you can keep the IAPManager class as a reusable component which you can extend according to your needs. Always make sure to be in align with the Apple’s recommendations and best practices regarding in-app purchases. With that said it’s about time to let you rest because you’ve just gone through a long tutorial, which I hope you found useful. Thanks for reading and… happy earnings!

For the full source code, you can refer to this project on GitHub.

Credits: Images by icons8

2 years ago, at WWDC 2017, Apple released the Vision framework, an amazing, intuitive framework that would make it easy for developers to add computer vision to their apps. Everything from text detection to facial detection to barcode scanners to integration with Core ML was covered in this framework.

This year, at WWDC 2019, Apple released several more new features to this framework that really push the field of computer vision. That’s what we’ll be looking at in this tutorial.

What We’ll be Building in this Tutorial

For many years now, Snapchat has reigned as the popular social media app among teens. With its simple UI and great AR features, high schoolers around the world love to place cat/dog filters themselves. Let’s flip the script!

In this tutorial, we’ll be building Snapcat, the Snapchat for Cats. Using Vision’s new animal detector, we’ll be able to detect cats, place the human filter on them, and take pictures of them. After taking pictures of our cats, we’ll want to scan their business cards. Using a brand new framework, VisionKit, we’ll be able to scan their business cards just like the default Notes app on iOS.

That’s not all! If you remember 2 years ago my tutorial on using Vision for text detection, I ended by saying that even though you could detect text, you would still need to integrate the code with a Core ML model to recognize each character. Well finally, Apple has released a new class under Vision to recognize the text it detects. We’ll use this new class to grab the information from the scanned cards and assign it to our cat. Let’s get started!

The project in this tutorial was built using Swift 5.1 in Xcode 11 Beta 7 running on macOS Catalina. If you face any errors when go through the tutorial, try to update your Xcode to the latest version or leave a comment below.

Getting the Starter Project

To begin, download the starter project here. Build and run on your device. What we have is the iconic Snapchat camera view. To create this, I added a ARSCNView, or an SceneView that is capable of displaying AR content, and overplayed it with a button.

When the shutter button is pressed, the app takes a snapshot of the current frame of this scene view and passes it along the navigation stack to the Cat Profile View where it’s loaded the profile image. In this view, we have empty fields for the data: Name, Number, and Email. This is because we haven’t scanned the business card yet! Clicking on the button has no action yet because this is where we’ll implement the document scanning program via VisionKit.

Image Recognition – Detecting a Cat Using VNRecognizeAnimalsRequest

Our first step is to detect cats in our camera view. Head over to CameraViewController and at the top of the file, type import Vision. This imports the Vision framework into this file, giving us access to all the classes and functions.

Now we want our app to automatically place the human filter (“the human emoji”) on our cat with no input from us whatsoever. This means that we need to scan the frames in our camera view every few milliseconds. To do this, modify your code to look like below:

@IBOutlet var previewView: ARSCNView!
var timer: Timer?

override func viewWillAppear(_ animated: Bool) {
    super.viewWillAppear(animated)
    let configuration = ARWorldTrackingConfiguration()
    previewView.session.run(configuration)
    timer = Timer.scheduledTimer(timeInterval: 0.5, target: self, selector: #selector(self.detectCat), userInfo: nil, repeats: true)

}

We create an instance of timer and define it in our viewWillAppear function. We schedule the timer to perform the function detectCat() in the interval of every half a second.

However, you’ll experience an error displayed because we have not created the function called detectCat. This is a simple fix. Here is the viewWillAppear function. Insert the code in the CameraViewController class:

@objc func detectCat() {
    print("Detected Cat!")
}

Let’s build and run the project so far. The moment our CameraViewController is initialized, we should see the text “Detected Cat!” printed to the console every half second.

Now here comes to the fun part where we’ll use the new VNRecognizeAnimalsRequest, a built-in class that lets you detect cats and dogs.

Modify the detectCat() function to look like this:

@objc func detectCat() {
    guard let currentFrameBuffer = self.previewView.session.currentFrame?.capturedImage else { return }
    let image = CIImage(cvPixelBuffer: currentFrameBuffer)
    let detectAnimalRequest = VNRecognizeAnimalsRequest { (request, error) in
        DispatchQueue.main.async {
            if let results = request.results?.first as? VNRecognizedObjectObservation {
                let cats = result.labels.filter({$0.identifier == "Cat"})
                for cat in cats {
                    print("Found a cat!!")
                }
            }
        }
    }

    DispatchQueue.global().async {
        try? VNImageRequestHandler(ciImage: image).perform([detectAnimalRequest])
    }
}

There’s quite a bit going on here but I’ll go through it line by line. First, we implement a guard function that checks to make sure that the current frame in our Preview View has an image. If this image can be captured, we create a special CIImage using the pixel buffer of that frame’s image.

Here’s some terminology! The CIImage class we’re using is a representation for an image as a CoreImage. This type of images are great when you want to analyze the pixels in them. A pixel buffer stores an image data in the main memory.

Before explaining the next few lines of code, it’ll help to understand how Vision works in case you’re unfamiliar. Basically, there are 3 steps to implement Vision in your app, which are:

1. Requests – Requests are when you request the framework to detect something for you.
2. Handlers – Handlers are when you want the framework to perform something after the request is made or “handle” the request.
3. Observations – Observations are what you want to do with the data provided with you.

So first, we create a VNRecognizeAnimalRequest to detect animals. This is a new VNImageRequest that was introduced in iOS 13 specifically for animal detection.

Next, within the handler of this request, we get the result and filter the labels in our first result. Why are we doing this? This is because currently, the VNRecognizeAnimalRequest can detect both cats and dogs. For our use, we only want cats which is why we define an array called cats where the identifier is equal to “Cats”. Finally, for each cat observation, we print that we found a cat to the console. This is just to see if Vision is detecting cats.

Remember, we’ve only defined the request. Now we need to ask Vision to perform the request. After defining the request, we ask VNImageRequestHandler to perform the request on our CIImage we defined earlier. A VNImageRequestHandler is basically a specific type of VNRequest that is used to process images using one or more image analysis requests.

Build and run the project on a device. I don’t have a cat so I’ll be using some images from online. If Vision correctly detects the cat, then we should be able to see it printed to the console.

It works!! But how should we let users know that the app detects a cat? It’s time to code again.

At the top of the class where we initialized the timer object, type the line: var rectangleView = UIView(). This initializes a UIView which we can modify to be a box. We’ll place this box around the coordinates where our Vision function can detect the cat.

Inside the detectCat() method, insert self.rectangleView.removeFromSuperview() at the beginning. Then modify the for loop to look like this:

for cat in cats {
    self.rectangleView = UIView(frame: CGRect(x: result.boundingBox.minX * self.previewView.frame.width, y: result.boundingBox.minY * self.previewView.frame.height, width: result.boundingBox.width * self.previewView.frame.width, height: result.boundingBox.height * self.previewView.frame.height))

    self.rectangleView.backgroundColor = .clear
    self.rectangleView.layer.borderColor = UIColor.red.cgColor
    self.rectangleView.layer.borderWidth = 3
                        self.previewView.insertSubview(self.rectangleView, at: 0)
}

Here’s what we’re doing. We are defining our rectangleView to have the bounds of the result’s bounding box. However, this is something worth mentioning.

In Vision, the size of the result’s bounding box is smaller than the size of your previewView. This is because when Vision analyzes the image, it scales down the image to speed up computer vision processes. That’s why when we create the bounds for our rectangleView, we have the multiply it by the constant to scale it up.

After defining the frame, all we do is set the view to have a transparent background and a border of width 3 and color red. We add this to our previewView. Now, you may be wondering why we included the line of code self.rectangleView.removeFromSuperview() at the beginning of the function. Remember, our Vision code is running every half a second. If we don’t remove the rectangleView, we’ll have a lot of boxes on our view.

Ok! Build and run the app! Are you seeing the boxes?

Yes! It’s working. Now we have one last thing to do: place a human emoji near the cat. This is really simple. Here’s what we do.

Ath the top of the class, underneath where we define our rectangleView, type the line: var humanLabel = UILabel().

Inside the for loop of our detectCat() function, we need to add this label to our view. Here’s how to do that.

for cat in cats {
    self.rectangleView = UIView(frame: CGRect(x: result.boundingBox.minX * self.previewView.frame.width, y: result.boundingBox.minY * self.previewView.frame.height, width: result.boundingBox.width * self.previewView.frame.width, height: result.boundingBox.height * self.previewView.frame.height))

    self.humanLabel.text = "👦"
    self.humanLabel.font = UIFont.systemFont(ofSize: 70)
    self.humanLabel.frame = CGRect(x: 0, y: 0, width: self.rectangleView.frame.width, height: self.rectangleView.frame.height)

    self.rectangleView.addSubview(self.humanLabel)
    self.rectangleView.backgroundColor = .clear
    self.previewView.insertSubview(self.rectangleView, at: 0)
} 

This is really easy to follow. We still have the same frame for our rectangleView. Now, we set the text of the label to be an emoji followed by the font size. Since we want the labels to cover the entire box, we set its width and height to be the same as the rectangleView. Finally, we removed the code that would create a border and instead add the label to our box and add the box to the previewView.

Build and run the app. Is the code working? Can you see the human emoji on the cat?

It works! We’ve got a human emoji placed on the cat. Now, of course, it’s not perfect because Vision only detects a cat object, not the facial features of the cat. Hopefully Apple will add that soon and we can further refine the filter.

Scanning the Business Card Using VNDocumentCameraViewController

Whew! We made it through the first part. From here on, it gets a little easier. When you snap the picture of your cat, you automatically get pushed to the Cat Profile view where you can see the image you’ve taken, the details of the cat, and a button that says “Scan For Details“. We’ll be pulling up the new document scanner code when this button is tapped on.

Navigate to CatProfileViewController.swift. At the top of the file, underneath import UIKit, type import VisionKit. The difference between Vision and VisionKit is that while Vision lets your perform computer vision analyses on an image, VisionKit is a small framework that lets your app use the system’s document scanner.

Now within the scanDocument() function, all you need to add are three lines of code.

@IBAction func scanDocument(_ sender: Any) {
    let documentCameraViewController = VNDocumentCameraViewController()
    documentCameraViewController.delegate = self
    self.present(documentCameraViewController, animated: true, completion: nil)
}

The VNDocumentCameraViewController() is a special type of view controller created by Apple which is used to scan the documents. We set the delegate of that controller and present the view controller.

Next, you need to implement the VNDocumentCameraViewControllerDelegate in the class. Change the line where we define out CatProfileViewController class to look like this.

class CatProfileViewController: UIViewController, VNDocumentCameraViewControllerDelegate {
    ....
}

Build and run the app. After taking a picture, when you press “Scan for Details”, the device’s document scanner should pop up.

Looks like it’s scanning! However, when we press “Save”, nothing happens. This is because we haven’t implemented any delegate methods yet. Underneath, the scanDocument() function, type the following.

func documentCameraViewController(_ controller: VNDocumentCameraViewController, didFinishWith scan: VNDocumentCameraScan) {
    let image = scan.imageOfPage(at: 0)
    self.catImageView.image = image
    controller.dismiss(animated: true)
}

The documentCameraViewController(didFinishWith:) is a method that runs when the Save button is clicked. We take the first scan, make it an image, and set the catImageView to display this image. We’re doing this to make sure that the scanner is working well.

Build and run the app. When you have to scan for details, scan over any business card and when you click “Save”, it should dismiss the document scanner and set the image to the catImageView.

Now that the app is able to detect the business card, let’s further implement it to extract the textual data from the card.

Text Recognition Using VNRecognizeTextRequest

With the new API, VNRecognizeTextRequest, introduced in iOS 13, it’s pretty easy to finds and recognizes text in an image.

Again, first import VisionKit in the file because the new API is bundled in the framework. As explained earlier, this will bring over the Vision API’s. Next, before the viewDidLoad() function, type the following:

var textRecognitionRequest = VNRecognizeTextRequest()
var recognizedText = "" 

The VNRecognizeTextRequest is just like the VNRecognizeAnimalsRequest class that we’ve used earlier. This request tells Vision to run text recognition on whatever image we pass to it. Later, when Vision detects text in a image, we’ll assign it to the variable recognizedText.

Head over to the viewDidLoad() function and update the code like below:

 override func viewDidLoad() {
super.viewDidLoad()
    self.navigationController?.setNavigationBarHidden(false, animated: true)
    self.title = "New Cat"
    catImageView.image = catImage

    textRecognitionRequest = VNRecognizeTextRequest(completionHandler: { (request, error) in
        if let results = request.results, !results.isEmpty {
            if let requestResults = request.results as? [VNRecognizedTextObservation] {
                self.recognizedText = ""
                for observation in requestResults {
                    guard let candidiate = observation.topCandidates(1).first else { return }
                      self.recognizedText += candidiate.string
                    self.recognizedText += "\n"
                }
                self.catDetailsTextView.text = self.recognizedText
            }
        }
    })
}

In the code, we initiate a VNRecognizeTextRequest. Vision returns the result of this request in a VNRecognizedTextObservation object. This type of observation contains information about both the location and content of text and glyphs that Vision recognized in the input image.

For every item in the requestResults array, we choose the first topCandidates. What is this? Well for every observation Vision makes, it outputs a list of potential candidates for the detected text. In some cases, it may be useful to consider all the possible text. For our case, we only care about the most accurate prediction.

Finally, we take the candidate.string and add it to our recognizedText. After going through every observation in the requestResults array, we set the text of the catDetailsTextView to our recognizedText.

Your code should look a little like the image below:

You can build and run the app now but nothing will happen. Why? This is because we still haven’t defined a handler to perform the request. This can be accomplished in a couple lines of code. Head back to the documentCameraViewController(didFinishWith:) function and modify it like this:

func documentCameraViewController(_ controller: VNDocumentCameraViewController, didFinishWith scan: VNDocumentCameraScan) {
    let image = scan.imageOfPage(at: 0)
    let handler = VNImageRequestHandler(cgImage: image.cgImage!, options: [:])
    do {
        try handler.perform([textRecognitionRequest])
    } catch {
        print(error)
    }
    controller.dismiss(animated: true)
}

The code above should look familiar as we used it in the first section of the tutorial. We define a handler of type VNImageRequestHandler and input the cgImage of our scan.

A VNImageRequestHandler is an object that processes one or more image analysis requests pertaining to a single image. By specifying the image, we can begin executing the Vision request. In the do block, we ask the handler to perform the Vision request we defined earlier. x

Build and run the app. Is it working?

It seems to be working quite well! We’re able to gather all the text from the scanned card and place it onto the text view.

Fine Tuning the Text Recognition

Now in my image above, the text recognition worked great. However, I’ll try to help you fine tune our VNRecognizeTextRequest in case it didn’t work so well for you.

At the end of our viewDidLoad() function, type the following:

textRecognitionRequest.recognitionLevel = .accurate
textRecognitionRequest.usesLanguageCorrection = false
textRecognitionRequest.customWords = ["@gmail.com", "@outlook.com", "@yahoo.com", "@icloud.com"] 

These are some of the values that the VNRecognizeTextRequest has. You can modify them to fine tune your application. Let’s go through these.

1. .recognitionLevel: You have two options between these: .fast and .accurate. When you want the request to recognize text, you can choose between recognizing faster or more accurately. Apple advices to use .accurate because even though it take slightly more time, it works well for recognizing texts in custom font shapes and size. The advantage with .fast is that it takes up less memory on the device.
2. .usesLanguageCorrection: Here’s how Vision Text Recognition Request works. After grabbing the text from each observation, it doesn’t just output that text to you. It goes through another layer of Natural Language to change any misspelled words. Think if it like a spell and grammar check. By changing this Boolean value, you can enable this on or off. When should you set this to true? If your application is being used to recognize text from a book or a document, then you should enable this setting. However, in our scenario, we’re using it to detect names, numbers, and email addresses. If we enable this setting, it’s possible that the request may think the number 1 is the lowercase letter l. Or another example is the last name He which Vision may correct to Hey or Hi.
3. .customWords: This value is an array that can be used for specific words. In our scenario, we’re detecting emails which generally end in @email.com. While this isn’t a word on its own, we wouldn’t want Vision predicting these characters to be something else. By offering custom words, the text recognition request analyzes the image a second time to see if any of those words appear so it can recognize them properly.

Build and run the app. Is there any improvement? Did the recognition system get better or worse? By fine tuning these parameters, you can utilize Vision’s text recognition system to be the best in your app.

Conclusion

In this tutorial, as you can see, it is so easy to take advantage of all the new API’s in Vision to perform object recognition. You learned how we can use the brand new animal detector to recognize cats and dogs in an image. You should also understand how the new framework, VisionKit, makes scanning documents a breeze. Finally, the coveted text recognition APIs became available in Vision. With easy-to-use API’s and lots of parameters for fine tuning the system, you can easily implement text recognition in your apps.

However, this isn’t all that was announced at WWDC 2019. Vision also introduced API’s to classify images into categories, analyze saliency, and detect human rectangles. I’ve attached some great links if you want to continue learning more about Vision.

• Vision Documentation
• WWDC 2019 Session: Understanding Images in Vision Framework
• WWDC 2019 Session: Text Recognition in Vision Framework
• AppCoda Tutorial: Using Vision Framework for Text Detection in iOS 11

I’d love to know what you’re building with Vision! Leave a comment below if you have a question or want to share what you’re working on. You can download the full tutorial here. Thanks for reading! Catch you in the next article!

Hello there! Welcome to the second tutorial of our Flutter series. In the last tutorial, you learned the basics of building a Flutter app. So if you have not gone through it, please take a pit stop here, visit it first before proceeding with this tutorial.

What will you learn?

In today’s tutorial, we will build an app with complex UI with the Flutter framework. We will explore quite a lot of components. By going through the tutorial, you will learn to implement:

• Textfields with Validation
• Centralized App Theme
• Carousel Slider
• List View
• Complex List View
• Basic Form
• Scoped Model

If any of the above raises your eyebrow, then you are here at the right place! Tighten your seatbelt as this tutorial will take you on a ride to learn all these concepts, by building an app called Moments.

Moments App

Moments App is a simple app that has the following screens:

• Login Screen
• Gallery Screen
• Menu Screen
• Form Screen
• List Screen
• Rich List Screen

You will build the screens sequentially, learn the different ways of wiring up views. And, this is how the final app will look like:

Let’s get started!

Getting Started

Please proceed to download the starter project and spend about 10-15 minutes to look through the code. The app has 4 main folders:

• Model
• Pages
• Scoped Models
• Utils

With 2 additional files main.dart and app.dart. Most of the functions are marked with // TODO, so you can implement its logic to build the components.

Login Screen

Let’s first take a look at how the completed Login Screen is:

It has the following UI components:

• Image
• Title Text
• TextField Title Text
• TextField
• Button

Proceed to login.dart. Here you will first implement the login textfield, start by adding this ladder of codes:

Widget _buildLoginTextField() {
  // TODO #1: Implement Login Text Field
  return Theme(
    data: ThemeData(
      primaryColor: kRegistrationBlack,
      primaryColorDark: kRegistrationBlack,
    ),
    child: TextFormField(
      style: Theme.of(context).textTheme.subtitle,
      controller: _mobileNumberController,
      maxLength: 4,
      enableInteractiveSelection: false,
      keyboardType: TextInputType.number,
      textInputAction: TextInputAction.next,
      onFieldSubmitted: (value) {
        loginUser();
      },
      decoration: InputDecoration(
        fillColor: kRegistrationBlack,
        border: OutlineInputBorder(
          borderRadius: BorderRadius.circular(5.0),
        ),
        contentPadding:
            EdgeInsets.symmetric(vertical: 10.0, horizontal: 10.0),
        labelText: LOGIN_LABEL_TEXT,
        suffixIcon: IconButton(
          icon: Icon(
            Icons.clear,
            color: kRegistrationBlack,
          ),
          onPressed: () {
            _mobileNumberController.clear();
            _mobileNumber = '';
          },
        ),
      ),
      validator: (String value) {
        if (value.isEmpty || value.length != 4) {
          return 'Please enter a valid 4 digit code.';
        }
      },
      onSaved: (String value) {
        _mobileNumber = value;
      },
    ),
  );
}

In this code, you returned a Theme Widget with a child of TextFormField. You should take a look at theme.dart to see how the theme data is set before proceeding further. Using a centralised theme, any view that requires a text style can easily reference by calling Theme.of(context).textTheme.[styleType].

You should take a look at these codes to understand how this view is built as they will be repeatedly used in this tutorial. This is how the TextField is created:

• Theme
  • ThemeData
  • TextFormField
    • InputDecoration
      • OutlineInputBorder
      • IconButton

The TextFormField has an OutlineInputBorder and a Clear IconButton, wrapped in Theme. It has a mobileNumberController which handles input text, and a validator for validation checks. onFieldSubmitted is triggered when the textInputAction is activated. To find out more about TextFormField, read the documentation here.

Moving on to the Enter button, a custom button has been created for you in buttons.dart. Here, you will just need to initialise a MomentsButton like this and since the button needs to be centered, wrap it with Center:

  Widget _buildEnterButton(BuildContext context) {
    // TODO #2: Implement Enter Button
    return Center(
        child:
            MomentsButton(text: ENTER, action: loginUser).getButton(context));
  }

The final step to complete this page is to layer everything into the body properties in build function:

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      backgroundColor: Color(0xFFF9F9F9),
      // TODO #3: Implement Login View
      body: SafeArea(
        child: ListView(
          padding: EdgeInsets.symmetric(horizontal: 24.0),
          children: <Widget>[
            SizedBox(height: 80.0),
            Column(
              children: <Widget>[
                Image.asset(
                  'assets/LOGO.png',
                  height: 150.0,
                ),
                SizedBox(height: 12.0),
                Text('Moments', style: Theme.of(context).textTheme.caption),
              ],
            ),
            SizedBox(height: 50.0),
            Text('Login:', style: Theme.of(context).textTheme.subtitle),
            SizedBox(height: 10.0),
            AccentColorOverride(
              color: kRegistrationBlack,
              child: Form(
                key: _formKey,
                child: _buildLoginTextField(),
              ),
            ),
            _buildEnterButton(context)
          ],
        ),
      ),
    );
  }

• Scaffold
  • SafeArea
    • ListView
      • Column
        • Image
        • Text
      • Text
      • Login TextField
      • Enter Button

You are starting to see that the best way to arrange views vertically is to use Column. SafeArea here is used to handle any view changes caused by the keyboard.

ListView automatically helps you handle any overflowing views and it includes a scrolling mechanism. SizedBox is used to specify fixed spaces between views.

With the Login Screen completed, many classes of views and components were introduced, and you will see them in action more as you build the next few pages.

Now let’s run the app, try to key in a wrong code 1111 and then a correct code 1234, and then move on to the Gallery Screen!

You will not dive deep into how Scoped Models work in this app. You can open user.dart to see the login logic, and explore this crash course. In a nutshell, Scoped Model is used to manage variables that will live in the lifetime of the app.

Gallery Screen

Next, we will build the Gallery screen. You will use Carousel Slider to build an image slider, and again, add a button to move to the next screen. Proceed to gallery.dart to get started.

Firstly, if you look at our assets folder, you notice there are already pre-loaded images labelled pic1.jpg etc. These are also declared in an array of strings call imgList.

Using the pre-created map function, you map them each into a Container so the output result is actually a List of Containers:

List _getChildren(MainModel model) {
  // TODO #4: Implement List
  final List<String> images = imgList;
  return map<Widget>(
    images,
    (int index, String url) {
      return Container(
        decoration:
            BoxDecoration(border: Border.all(width: 10.0, color: Colors.white)),
        margin: EdgeInsets.all(5.0),
        child: Image.asset(
          url,
          fit: BoxFit.cover,
          width: 261.0,
          height: 353.0,
        ),
      );
    },
  ).toList();
}

With the List of Containers ready, you can now load them into the Carousel like this in _CarouselWithIndicatorState class:

  @override
  Widget build(BuildContext context) {
    final List<String> images = imgList;
    // TODO #5: Implement Carousel
    return Column(children: [
      CarouselSlider(
        items: _getChildren(),
        autoPlay: true,
        height: 353,
        enlargeCenterPage: true,
        onPageChanged: (index) {
          setState(() {
            _current = index;
          });
        },
      ),
      Row(
        mainAxisAlignment: MainAxisAlignment.center,
        children: map<Widget>(
          images,
          (int index, String url) {
            return Container(
                width: 8.0,
                height: 24.0,
                margin: EdgeInsets.symmetric(vertical: 2.0, horizontal: 2.0),
                decoration: BoxDecoration(
                    shape: BoxShape.circle,
                    color:
                        _current == index ? kRegistrationBlack : Colors.white));
          },
        ),
      ),
    ]);
  }

Again, Column is used to layout 2 views vertically:

• View #1 – Carousel
• View #2 – Row of Carousel Indicators

If you can using Visual Studio Code and using a MAC machine, you can whole CMD and hover on CarouselSlider to view more information about this class.

The MomentsButton can be created just like the previous one you have built:

    Widget _buildMomentsButton(BuildContext context) {
      // TODO #6: Implement Button
      return Center(
        child: MomentsButton(text: 'MOMENTS', action: _goToHomePage)
            .getButton(context),
      );
    }

Having a custom class helps minimise repeated codes.

The last step for this page is to link everything up in build, add these codes after _buildMomentsButton:

    return Scaffold(
      appBar: PreferredSize(
        preferredSize: Size.fromHeight(30.0), // here the desired height
        child: AppBar(backgroundColor: kPlatinum, elevation: 0.0),
      ),
      backgroundColor: kPlatinum,
      // TODO #7: Implement Gallery
      body: Column(
        mainAxisAlignment: MainAxisAlignment.start,
        children: <Widget>[
          Padding(
            padding: EdgeInsets.only(top: 8.0, bottom: 24.0),
            child: Text('Moments', style: Theme.of(context).textTheme.caption),
          ),
          Expanded(
            child: SizedBox(
              height: 200.0,
              child: ListView(children: <Widget>[
                Padding(
                  padding: EdgeInsets.symmetric(vertical: 15.0),
                  child: Column(
                    children: [
                      CarouselWithIndicator(),
                    ],
                  ),
                ),
              ]),
            ),
          ),
          Padding(
              padding: EdgeInsets.only(bottom: 24.0),
              child: _buildMomentsButton(context))
        ],
      ),
    );

PreferredSize is used to give AppBar a fixed size. Using MainAxisAlignment.start positions its children views in a top-down approach.

Run the app and you should see a nice carousel slide show :-).

Menu Screen

You are now down to just 3 more main screens to build, with a Menu Container Screen. In gallery.dart, you ended with a MomentsButton which navigates user to MainPage.

Open up main.dart in pages folder, and in this page, there will be 3 menu buttons at the top, which when each one is being tapped, it will switch the child view to its respective view.

Let’s start by implementing buildMenuItems:

  List<Widget> buildMenuItems(MainModel model) {
    // TODO #8: Implement Building of Menu Items
    final List<String> images = [
      'assets/menu_form.png',
      'assets/menu_list.png',
      'assets/menu_richList.png'
    ];

    final List<String> menuTitles = ['Form', 'List', 'RichList'];

    final List<MenuItem> menuItems = [
      MenuItem.form,
      MenuItem.list,
      MenuItem.richList
    ];

    final List<Widget> columnItems = List<Widget>();

    for (var i = 0; i < menuTitles.length; i++) {
      bool isSelectedMenuItem = selectedMenuItem == menuItems[i];
      columnItems.add(
        Container(
          decoration: BoxDecoration(
              borderRadius: BorderRadius.all(Radius.circular(4.0)),
              border: Border.all(
                  color:
                      isSelectedMenuItem ? kRegistrationBlack : Colors.white)),
          child: InkWell(
            onTap: () {
              setState(() {
                selectedMenuItem = menuItems[i];
              });
            },
            child: Padding(
              padding: EdgeInsets.all(12.0),
              child: Column(
                mainAxisSize: MainAxisSize.min,
                mainAxisAlignment: MainAxisAlignment.center,
                children: <Widget>[
                  Image(
                    image: AssetImage(images[i]),
                    height: 40,
                    width: 40,
                    color: kRegistrationBlack,
                  ),
                  Container(
                    margin: EdgeInsets.only(top: 4),
                    child: Text(menuTitles[i],
                        style: Theme.of(context).textTheme.display1),
                  )
                ],
              ),
            ),
          ),
        ),
      );
    }

    return columnItems;
  }

Wow! That’s really a lot of code. It does look intimidating, but actually, they are really simple. Logically, every MenuButton is made up of:

1. Image
2. Title
3. MenuItem Type (Enum)

At the end, you get the complete view likes this:

• Container
  • InkWell (Allows Tap)
    • Column
      • Image
      • Text

This is one way you actually build a customised button. InkWell brings in the ability to implement onTap event. The function also uses isSelectedMenuItem to render a bordered or non-bordered view. At the end of this function, it returns the columns as a List.

Next, implement getPage which returns the selected page object:

  Widget getPage(MainModel model) {
    // TODO #9: Implement Menu Items Population
    switch (selectedMenuItem) {
      case MenuItem.form:
        return FormPage();
      case MenuItem.list:
        return ItemListPage(model);
      case MenuItem.richList:
        return RichListPage();
        break;
    }
  }

And, finally, implement the Column view which has a Row that contains all the customised buttons:

  @override
  Widget build(BuildContext context) {
    final MainModel _model = ScopedModel.of(context);

    return Scaffold(
      backgroundColor: kMomentsWhite,
      appBar: AppBar(
        centerTitle: true,
        title: Text('Menu', style: Theme.of(context).textTheme.headline),
        backgroundColor: Colors.white,
        elevation: 0.0,
        actions: <Widget>[
          IconButton(
            icon: Icon(Icons.exit_to_app),
            onPressed: () {
              _model.logout();
              Navigator.of(context).pushReplacementNamed('/login');
            },
          ),
        ],
      ),
      // TODO #10: Implement Child View
      body: Column(
        children: <Widget>[
          Padding(
            padding: EdgeInsets.only(top: 12.0, bottom: 12.0),
            child: Container(
              color: Colors.white,
              child: Row(
                  mainAxisAlignment: MainAxisAlignment.spaceEvenly,
                  children: buildMenuItems(_model)),
            ),
          ),
          Padding(
            padding: EdgeInsets.only(bottom: 8),
            child: Text("Hello ${_model.user.name}!",
                style: Theme.of(context).textTheme.headline),
          ),
          Expanded(child: getPage(_model))
        ],
      ),
    );
  }

In the Row here, MainAxisAlignment.spaceEvenly helps to spread the views evenly as the screen size changes. In the next layer, a simple Welcome message is shown. Followed by the child view which is rendered by getPage.

Run the app now and you should be able to tap on each of the button, see them being selected! Nothing much happens at the child view yet as they are not implemented. That is up next!

Form Screen

Next, we’ll implement the form screen. Open form.dart. In most of the apps you are using now, it’s common to implement a page with a form to submit information. Here’s a quick way to implement a form with radio buttons, checkbox and also a counter.

Name Text Field

The textfield is completely similar to the one you have implemented in the login screen, with the exception that here a ListTile is used with leading and title used to specify the location of the views:

  ListTile getNameInputListTile() {
    // TODO #12: Implement Name Section
    return ListTile(
      leading: Padding(
        padding: EdgeInsets.only(left: 8.0, top: 16.0),
        child: Text('Your Name:', style: Theme.of(context).textTheme.subtitle),
      ),
      title: Padding(
        padding: EdgeInsets.only(top: 10.0),
        child: Theme(
          data: ThemeData(
            primaryColor: kRegistrationBlack,
            primaryColorDark: kRegistrationBlack,
          ),
          child: Container(
            height: 40.0,
            child: TextField(
              style: Theme.of(context).textTheme.subtitle,
              controller: _nameTextController,
              textCapitalization: TextCapitalization.words,
              decoration: InputDecoration(
                fillColor: kRegistrationBlack,
                border: OutlineInputBorder(
                  borderRadius: BorderRadius.circular(5.0),
                ),
                contentPadding:
                    EdgeInsets.symmetric(vertical: 10.0, horizontal: 10.0),
                suffixIcon: IconButton(
                  icon: Icon(
                    Icons.clear,
                    color: kRegistrationBlack,
                  ),
                  onPressed: () {
                    _nameTextController.clear();
                  },
                ),
                hintText: NAME_HINT_TEXT,
                labelText: NAME_LABEL_TEXT,
              ),
            ),
          ),
        ),
      ),
    );
  }

I will not go into further explanation here as bulks of the code are the same. You could challenge yourself by placing them in a centralised class like how I did it for MomentsButton, give it a custom initiliser and call it to reduce code.

Guests Counter

The counter section contains a label text, a - button, a counter text and a + button:

  Widget getGuestsCounter() {
    // TODO #13: Add Guest Counter
    return ListTile(
      leading: Padding(
        padding: EdgeInsets.only(left: 8.0, top: 8.0),
        child: Text('No. of Guests (Including yourself):',
            style: Theme.of(context).textTheme.subtitle),
      ),
      title: Row(
        mainAxisAlignment: MainAxisAlignment.end,
        children: <Widget>[
          IconButton(
            color: kRegistrationBlack,
            icon: Icon(Icons.remove),
            onPressed: () {
              if (_guestsCounter > 0) {
                setState(() => _guestsCounter--);
                _formData['guests'] = _guestsCounter;
              }
            },
          ),
          Text(
            _guestsCounter.toString(),
            style: Theme.of(context).textTheme.title,
          ),
          IconButton(
            color: kRegistrationBlack,
            icon: Icon(Icons.add),
            onPressed: () {
              if (_guestsCounter < 9) {
                setState(() => _guestsCounter++);
                _formData['guests'] = _guestsCounter;
              }
            },
          ),
        ],
      ),
    );
  }

As you have understood how a ListTile works, the leading view is the label, where as the title view contains a Rowwhich contains:

• IconButton (-)
• Text
• IconButton (+)

Section Header

In every section, we are also giving it a nice header:

  Widget getSectionHeader(String title) {
    // TODO #14: Add Section Header
    return Column(
      crossAxisAlignment: CrossAxisAlignment.start,
      children: <Widget>[
        Padding(
          padding: EdgeInsets.only(top: 18, left: 22.0),
          child: Text(
            title,
            style: Theme.of(context).textTheme.headline,
          ),
        ),
        Padding(
          padding: EdgeInsets.only(left: 18, right: 18),
          child: Divider(
            color: kYankeesBlue,
          ),
        )
      ],
    );
  }

Gender Radio Option

This is radio button type option, so go ahead and add this:

  Widget getGenderOptions() {
    // TODO #15: Add Gender Options
    return ListTile(
      leading: Padding(
        padding: EdgeInsets.only(left: 8.0, top: 4.0),
        child: Text('Gender:', style: Theme.of(context).textTheme.subtitle),
      ),
      title: Row(
        mainAxisAlignment: MainAxisAlignment.start,
        children: <Widget>[
          Radio(
            activeColor: kRegistrationBlack,
            value: 0,
            groupValue: _formData['gender'] as int,
            onChanged: _handleGenderChange,
          ),
          Text(
            GENDER_CHOICES[0],
            style: Theme.of(context).textTheme.subtitle,
          ),
          Radio(
            value: 1,
            activeColor: kRegistrationBlack,
            groupValue: _formData['gender'] as int,
            onChanged: _handleGenderChange,
          ),
          Text(
            GENDER_CHOICES[1],
            style: Theme.of(context).textTheme.subtitle,
          ),
        ],
      ),
    );
  }

In Visual Studio Code, you can actually use OPTION+SHIFT+F to perform an auto-indentation.

Up to this point, I believe you are already a ListTile expert. Here we introduced Radio.

Transport Checkbox Option

The last option type is Checkbox:

  Widget getBusServiceOption() {
    // TODO #16: Add Transport Options
        return Padding(
      padding: EdgeInsets.only(bottom: 12),
      child: ListTile(
        leading: Checkbox(
          activeColor: kRegistrationBlack,
          onChanged: _handleTransportOption,
          value: _formData['bus'] as bool,
        ),
        title: InkWell(
          onTap: () {
            setState(() {
              _formData['bus'] = !_formData['bus'];
            });
          },
          child: Text(
            'I need transport',
            style: Theme.of(context).textTheme.subtitle,
          ),
        ),
      ),
    );
  }

Nothing really special here, we used InkWell to improve the tap area of the Checkbox.

Alert Dialog

For the alert dialog, you may have already seen how an AlertDialog is created. Here’s a simple dialog you should add to validate user’s input:

  void _showDialog(String title, String content) {
    // TODO #11: Implement Dialog
    showDialog<AlertDialog>(
      context: context,
      builder: (BuildContext context) {
        return AlertDialog(
          title: Text(title),
          content: Text(content),
          actions: <Widget>[
            FlatButton(
              child: Text(ALERT_OK),
              onPressed: () {
                Navigator.of(context).pop();
              },
            ),
          ],
        );
      },
    );
  }

Lastly, the Submit Button which you are implemented it for the third time:

  Widget getSubmitButton() {
    // TODO #17: Add Submit Button
    return Padding(
      child: Center(
        child:
            MomentsButton(text: SUBMIT, action: _submitForm).getButton(context),
      ),
      padding: EdgeInsets.only(bottom: 16),
    );
  }

Let the views assemble! Take a look at renderForm and you can see how we can render the form and order of each section easily! Here, the build function is completed for you so go ahead and run the app!

Try submitting the form without a name, you should be prompted with the AlertDialog you added.

If you are unable to see your page rendered, try to delete the app and perform a fresh installation. Sometimes, flutter may unknowingly cache it.

List Screen

Displaying a List of items is also a common feature of every app that we use. Headover to item_list.dart and here, you will use ListView to build a list of data:

  Widget _buildItemList() {
    // TODO #18: Implement Item List
    return ListView.builder(
      itemBuilder: (BuildContext context, int index) {
        return Column(
          children: <Widget>[
            Padding(
              child: ListTile(
                title: Text(
                  _itemList[index].title,
                  style: Theme.of(context).textTheme.title,
                ),
                subtitle: Text(_itemList[index].subtitle,
                    style: Theme.of(context).textTheme.subtitle),
              ),
              padding: EdgeInsets.all(10.0),
            ),
            Divider(
              height: 5.0,
            )
          ],
        );
      },
      itemCount: _itemList.length,
    );
  }

The data has been pre-loaded from scoped-models/item_list.dart. The builder method in ListView automatically helps you iterate through the data List, so you can implement the rendering logic in itemBuilder. The build function has been implemented for you. Go ahead and run the app, you should see a nice scrolling view in List tab 🙂

Rich List Screen

The last and final screen is the RichList screen. I personally named it as RichList because it has an image and a customised look. Headover to rich_list.dart and you should already see everything implemented for you. But, you are seeing an empty screen because each card view is not implemented yet.

Now go to rich_list_item_card.dart and here is where all the wires are missing. You will be giving each image a circular frame, so go ahead and add these in itemImage:

  Widget get itemImage {
    // TODO #19: Implement List Item Image View
    return Container(
      width: 125.0,
      height: 125.0,
      decoration: BoxDecoration(
        border: Border.all(color: kPlatinum),
        shape: BoxShape.circle,
        image: DecorationImage(
          fit: BoxFit.cover,
          image: AssetImage(item.imagePath),
        ),
      ),
    );
  }

Every image is also accompanied by a Card with description on the right:

  Widget get listItemCard {
    // TODO #20: Implement List Item Card View
    return Padding(
      padding: EdgeInsets.only(top: 18, bottom: 16),
      child: Container(
        constraints: BoxConstraints(minWidth: 300),
        height: 130.0,
        child: Card(
          elevation: 0.0,
          shape: RoundedRectangleBorder(
            side: BorderSide(color: kPlatinum, width: 1.0),
            borderRadius: BorderRadius.circular(4.0),
          ),
          color: Colors.white,
          child: Padding(
            padding: EdgeInsets.only(
              top: 10.0,
              bottom: 8.0,
              left: 95.0,
            ),
            child: Padding(
              padding: EdgeInsets.only(top: 8),
              child: Column(
                crossAxisAlignment: CrossAxisAlignment.start,
                mainAxisAlignment: MainAxisAlignment.spaceEvenly,
                children: <Widget>[
                  Text(widget.item.name,
                      style: Theme.of(context).textTheme.title),
                  Text(widget.item.description,
                      style: Theme.of(context).textTheme.subtitle),
                ],
              ),
            ),
          ),
        ),
      ),
    );
  }

And finally, let’s combine both into build function:

  @override
  Widget build(BuildContext context) {
    return Padding(
      padding: EdgeInsets.symmetric(horizontal: 16.0, vertical: 8.0),
      // TODO #21: Implement List Item Image Card View
      child: Container(
        height: 150.0,
        child: Stack(
          children: <Widget>[
            Positioned(
              left: 45.0,
              child: listItemCard,
            ),
            Positioned(top: 20.5, child: itemImage),
          ],
        ),
      ),
    );
  }

While Column and Row help you to layer the views vertically down and horizontally across, Stack helps you to layer your views on top of each other. Here, the image overlaps the Card View by putting it in the second postion. 

Run the app and you should see the completed RichList!

Moving Forward

You can download the finished project here. You have learnt a lot about creating more complex UIs in this tutorial using Flutter. Flutter is natively developed by Google so it will generally have better performance on Android devices. However, given its natively-compiled capability, it actually runs smoothly on most iOS devices as well.

Flutter is still at its infant stage, and it surely has a very bright future. It can help you create beautiful and complex UIs with lesser code. I encourage you to use its documentation diligently to build your own flavor of apps.

Thank you for following through this tutorial!

One of the most innovative inventions Apple has come up with in the past year is its True Depth camera. An ode to hardware and software engineers, the True Depth camera is what powers its secure facial recognition system, FaceID. As developers, the True Depth camera opens up a world of possibilities for us, especially in the field of face-base interactions.

Before we begin this ARKit tutorial, let me quickly brief you on the different parts of the camera. Like most iPhone/iPad front cameras, the True Depth camera comes with a microphone, a 7 megapixel camera, an ambient light sensor, a proximity sensor, and a speaker. What makes the True Depth camera itself is the addition of a dot projector, flood illuminator, and infrared camera.

The dot projector projects more than 30,000 invisible dots onto your face to build a local map (you’ll see this later in the tutorial). The infrared camera reads the dot pattern, captures an infrared image, then sends the data to the Secure Enclave in the A12 Bionic chip to confirm a match. Finally, the flood illuminator allowed invisible infrared light to identify your face even when it’s dark.

These parts come together to create some magical experiences like Animojis and Memojis. Special effects that require a 3D model of the user’s face and head can rely on the True Depth Camera.

Introduction to the Demo Project

I believe that it’s important for developers to learn how to utilize the True Depth camera so they can perform face tracking and create amazing face-based experiences for users. In this tutorial, I will show you how we can use the 30,000 dots to recognize different facial movements using ARFaceTrackingConfiguration, that comes with the ARKit framework.

The final result will look like this:

Let’s get started!

You will need to run this project on either an iPhone X, XS, XR, or iPad Pro (3rd gen). This is because these are the only devices which have the True Depth camera. We will also be using Swift 5 and Xcode 10.2.

Editor’s Note: If you’re new to ARKit, you can refer to our ARKit tutorials.

Creating a ARKit Demo for Face Tracking

First, open Xcode and create a new Xcode project. Under templates, make sure to choose Augmented Reality App under iOS.

Next, set the name of your project. I simply named mine True Depth. Make sure the language is set to Swift and Content Technology to SceneKit.

Head over to Main.storyboard. There should be a single view with an ARSCNView already connected to an outlet in your code.

What we have to do is really simple. All we need to do is add a UIView and a UILabel inside that view. This label will inform the user of the face expressions they are making.

Drag and drop a UIView into the ARSCNView. Now, let’s set the constraints. Set the width to 240pt and height to 120pt. Set the left and bottom constraints to 20pt.

For design purpose, let’s set the alpha of the view to 0.8. Now, drag a UILabel into the view you just added. Set the constraints to 8 points all around as shown below.

Finally, set the alignment of the label to centralized. Your final storyboard should look like this.

Now, let’s set the IBOutlets to our ViewController.swift file. Switch to the Assistant editor. Control and click on the UIView and UILabel and drag it over to ViewController.swift to create the IBOutlets.

You should create two outlets: faceLabel and labelView.

Creating a Face Mesh

Let’s clean up the code a little bit. Because we chose the Augmented Reality App as our template, there’s some code which we don’t need. Change your viewDidLoad function to this:

override func viewDidLoad() {
    super.viewDidLoad()

    // 1 
    labelView.layer.cornerRadius = 10

    sceneView.delegate = self
    sceneView.showsStatistics = true

    // 2
    guard ARFaceTrackingConfiguration.isSupported else {
        fatalError("Face tracking is not supported on this device")
    }
}

With the template, our code loads a 3D scene. However, we don’t need this scene, so we delete it. At this point, you can delete the art.scnassets folder in the project navigator. Finally, we add two pieces of code to our viewDidLoad method.

1. First, we round the corners of the labelView. This is more of a design choice.
2. Next, we check to see if the device supports the ARFaceTrackingConfiguration. This is the AR tracking setting we’ll be using to create a face mesh. If we don’t check to see if a device supports this, our app will crash. If the device does not support the configuration, then we will present an error.

Next, we’ll change one line in our viewWillAppear function. Change the constant configuration to ARFaceTrackingConfiguration(). Your code should look like this now.

Next, we need to add the ARSCNViewDelegate methods. Add the following code below // MARK: - ARSCNViewDelegate.

func renderer(_ renderer: SCNSceneRenderer, nodeFor anchor: ARAnchor) -> SCNNode? {
    let faceMesh = ARSCNFaceGeometry(device: sceneView.device!)
    let node = SCNNode(geometry: faceMesh)
    node.geometry?.firstMaterial?.fillMode = .lines
    return node
}

This code runs when the ARSCNView is rendered. First, we create a face geometry of the sceneView and set it to the constant faceMesh. Then, we assign this geometry to an SCNNode. Finally, we set the material of the node. For most 3D objects, the material is usually the color or texture of a 3D object.

For the face mesh, you can use two materials- either a fill material or a lines material. I prefer the lines which is why I set fillMode = .lines, but you can use what you prefer. Your code should look like this now.

If we run the app, you should see something like this.

Updating the Face Mesh

You may notice that the mesh does not update when you change your facial features (blinking, smiling, yawning, etc.). This is because we need to add the renderer(_didUpdate:) under the renderer(_nodeFor) method.

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
    if let faceAnchor = anchor as? ARFaceAnchor, let faceGeometry = node.geometry as? ARSCNFaceGeometry {
        faceGeometry.update(from: faceAnchor.geometry)
    }
}

This code runs every time the sceneView updates. First, we define a faceAnchor as the anchor for the face it detects in the sceneView. The achor is the information about the pose, topology, and expression of a face detected in the face-tracking AR session. We also define the constant faceGeometry which is a topology of the face detected. Using these two constants we update the faceGeometry every time.

Run the code again. Now, you’ll see the mesh updating every time you change your facial features, all running at 60 fps.

Analyzing the features

First, let’s create a variable at the top of the file.

var analysis = ""

Next, type the following function at the end of the file.

func expression(anchor: ARFaceAnchor) {
    // 1
    let smileLeft = anchor.blendShapes[.mouthSmileLeft]
    let smileRight = anchor.blendShapes[.mouthSmileRight]
    let cheekPuff = anchor.blendShapes[.cheekPuff]
    let tongue = anchor.blendShapes[.tongueOut]
    self.analysis = ""

    // 2    
    if ((smileLeft?.decimalValue ?? 0.0) + (smileRight?.decimalValue ?? 0.0)) > 0.9 {
        self.analysis += "You are smiling. "
    }

    if cheekPuff?.decimalValue ?? 0.0 > 0.1 {
        self.analysis += "Your cheeks are puffed. "
    }

    if tongue?.decimalValue ?? 0.0 > 0.1 {
        self.analysis += "Don't stick your tongue out! "
    }
}

The above function takes an ARFaceAnchor as a parameter.

1. The blendShapes are a dictionary of named coefficients representing the detected facial expression in terms of the movement of specific facial features. Apple provides over 50 coefficients which detect various different facial features. For our purpose we’re using only 4: mouthSmileLeft, mouthSmileRight, cheekPuff, and tongueOut.
2. We take the coefficients and check the probability of the face performing these facial features. For detecting a smile, we add the probabilities of both the right and left side of the mouth. I found that the 0.9 for the smile and 0.1 for the cheek and tongue work best.

We take the possible values and add the text to the analysis string.

Now that we have our function created, let’s update our renderer(_didUpdate:) method.

func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
    if let faceAnchor = anchor as? ARFaceAnchor, let faceGeometry = node.geometry as? ARSCNFaceGeometry {
        faceGeometry.update(from: faceAnchor.geometry)
        expression(anchor: faceAnchor)
        
        DispatchQueue.main.async {
            self.faceLabel.text = self.analysis
        }
        
    }
}

We run the expression method every time the sceneView is updated. Since the function is setting the analysis string, we can finally set the text of the faceLabel to the analysis string.

Now, we’re all done coding! Run the code and you should get the same result as we saw in the beginning.

Conclusion

There is a lot of potential behind developing face-based experiences using ARKit. Games and apps can utilize the True Depth camera for a variety of purposes. One of my favorite apps is Hawkeye Access, a browser you can control using your eyes.

For more information on the True Depth camera, you can check out Apple’s video Face Tracking with ARKit. You can download the final project here.