Notable Apple WWDC 2019 news and features.

I’m compiling this simple point form document mainly for myself, to keep track of some of the more general exciting features from WWDC 2019. I will not be covering niche APIs such as Health Kit or Home Kit. Please excuse the rough format.

Feature list from State of the Union:

  • iOS Apps now portable to Mac with “1 checkbox” (Mac OS)
  • Watch apps now able to run independently from your iPhone (optionally). This combines with direct downloads of watch apps, without downloading to your phone first.
  • Semantic Colors and Adaptive Materials are color systems where you have a standard color pallette for your UI, popover views, dark mode or high contrast mode, and transparency options. (“Adaptive unified look”)
  • Default presentation style changed from full screen to cards.
  • Contextual action menus in iOS, MacOS and iPadOS.
  • New standard symbol library (of around 1500 symbols) that have weights to adjust to the fonts you use.
  • Compositional layer API for collection views.
  • New share sheet API.
  • iPadOS’s UIWindowScene API and multitasking allows your app to be open in multiple spaces at the same time, and showing different states in each space.
  • In iPadOS you can now have multiple Scene Delegate objects, each with it’s own UI Lifecycle, but still have only 1 App Delegate.
  • iPadOS: NSUserActivity handles state restoration per user activity.
  • New set of gestures on the iPad, such as three finger forward/back swipe for redo/undo .
  • Voice control. (Commands to open apps, dictation, calendar events, etc)
  • Pre iOS 13 AI/ML functionality: Still image analysis, Image sequence annalysis, Named entity recognition, Linguistic tags, Face landmarks, Rectangle detection, Face detection, Horizon detection, Image alignment, Object tracking, Text detection, Part of speech, Barcode detection, Tokenization, Lemmatization, Language Identification, Speech recognizer, Object recognition.
  • New iOS13 AI/ML functionality: Speech saliency, Word embeddings, Sound analysis, Text catalog, Image similarity, Image classification, On-device speech, Face capture quality, Image saliency, Text Recognition, Sentiment classification.
  • New machine learning models: BERT, Mask R-CNN, Open AI GPT, WaveRNN, ELMo, Wavenet, Deep Speech. (can be used for Natural Language Processing)
  • On-device personalization now available for Core ML, allowing updating our ML models with individual user generated data from the phone. (Resulting in an updated and personalized model for the user)
  • CreateML is a MacOS program that lets you build ML models without writing any code, on your map, and comes with templates. Realmtime feedback on training performance.
  • Conversational shortcuts in Siri for apps.
  • Automation support: set specific triggers on when to run any shortcut. Your apps shortcuts can be combined with other apps in multi step shortcuts. (Steps have In and Out parameters)
  • Reality composer is a AR tool to compose 3D environments on iOS, iPadOS and MacOS.
  • RealityKit rendering 3D framework focused on realistic representation of virtual objects (lighting, shadows, blending techniques). Has angular force you can apply to objects.
  • ARKit 3.0, use front and back camera at the same time. So can use face tracking to drive AR experiences. People occlusion allows you to be part of an AR scene, and be occluded by virtual objects. Captures person motion (converting you to a 2D articulated stick figure, then converted into a 3D motion).
  • Metal consolidates GPU families into 3 groupings: Metal Common GPUs, Apple GPU Features and Mac GPU. Direct access to device GPU for image processing.

Sign in with Apple

  • An auto-created and verified surrogate email address created for you (per app).
  • Doesn’t retain messages.
  • Auto forwards.
  • Any email communication.
  • A verified email (verifies for you automatically)
  • No new passwords to remember or forget.
  • 2FA protected by default. No password.
  • No cost to you.
  • Cross platform. Bespoke UX per apple experience, and javascript API for Web, Windows and Android.
  • 4 integration steps: Button (ASAuthorizationAppleIDButton), Authorization (ASAuthorizationAppleIDProvider pass a requestedScopes array, ex: [.fullName, .email]), Verification Handling Changes.
  • ASAuthorizationController and delegatemethod: didCompleteWithError.
  • Unique User ID (team scoped, stable, unique). Stable across all platforms.
  • Identity token, authorization code. (Identity Token + Refresh token).

Swift Packages

  • Github is adding support to Swift Packages in the Github Package registry.
  • Swift package manager has been introduced in Swift 3.
  • Swift packages let you manage dependency versions.
  • Share code among your apps.
  • XCode11 now allows you to access swift packages.
  • TO import a package into your XCode project: File → Swift Packages → Add Package Dependency.
  • You have to add your Github account to swift preferences, so you can see package repositories in that account, OR enter a package URL at the top.
  • After being imported the Package dependency looks like this in XCode:
A package imported into XCode 11
  • Then do import YourPackageName
  • xcshareddata / swiftpm / Package.resolved needs to be checked in to source control. (Stores versions about the packages that were resolved. You want everyone on your team to have the same versionse, so check that in. Essentially a lock file)
  • A package contains a Package.swift file, Sources, and Tests.
  • Here’s a sample Package.swift file for a swift package:
  • Source code from packages is compiled in a way that makes it compatible with your project. Includes architectures and platform settings. Links everything together into an App.
  • Static libraries by default.
  • A package can also depend on other packages. Here’s an example of a package dependency graph that describes a package’s dependency on another package:
Package dependency graph
  • Version convention rules. 1.2.4 as an example. Major, Minor, Patch. A major is only incremented when breaking changes are introduced that require clients to adapt. Minor is incremented when functionality is added without breaking existing clients (added methods for instance). And the patch version is incremented when no semantic meaning changes are introduced (bugfixes).
  • By default, XCode includes a library up until but not including the next major version. So if you set the package to 2.0.0 — Next Major, it means it will never go to 3.0.0, but everything before it. (Ex: 2.9.9). In other words, major package upgrades should always be explicit actions from the developer (because your code needs to be updated to adapt to the new major version of the library)
  • If you are directly using a sub-package dependency, then you should also directly link to it, because you don’t want your code to break IF your 1st-level package dependency updates their code to suddenly not use that sub-dependency any longer. Example: (We should import DesignFont explicitly because we’re using it directly — even though at the moment we are also getting it indirectly through DesignTheme):
Adding explicit dependencies on nested sub-packages in XCode 11, IF you use them directly…
  • To update a package: File → Swift Packages → Update to Latest Package Versions. Then commit the changes to Package.resolved file to source control so your team gets the updated dependencies. (It’s a generated file that gets updated when you update to the latest version)
  • Package.resolved ensures consistency in package versions among your team.
  • If you use a different version between a sub-package and a direct package reference, you will run into a dependency version conflict:

You can only have one version of a package in any given workspace. Typically you would use the later of the versions.

  • Major version updates often have API breaking changes or contract changes to the Library in question.

Advances in App Background Execution

Two ways to enter background state:

  • App request. (Finishing some background work, updates, downloads, specific use cases, etc)
  • Event trigger. (App responded to something that happened in the world, such as a health state update, location update, etc)

There are 3 main considerations Apple has when it comes to background execution: Power, Performance, Privacy.

Power / Performance: If you are executing in the background, this execution may be happening while another app is running in the foreground. You therefore have a resource limit imposed on you to ensure that resources are not excessively consumed. If going over these limits, your app may get force-terminated by Apple, resulting in slower further startup times.

Privacy: Apple has specific APIs for different use cases. Each API has user education messages that let the user know up-front about what’s being shared. (Examples: VoIP, Audio, SiriKit Intent, Accessory, Bluetooth, etc)

Sample Messaging App analysis for background tasks:

  • Message sending. (May take additional time to send a message. We need to protect completion of this task. Background task completion API gives additional time for the app to finish doing its workload. Use UIApplication.baginBackgroundTask(expirationHandler:) OR ProcessInfo.performExpiringActivity(withReason: using:) // for calling from extensions
  • Can be used for completing any user initiated request.
  • After your beginBackgroundTask completion block fires, you need to call endBackgroundTask(identifier) to close the task and free resources.
  • Above we can see that we should post a User Notification if congested conditions or other issues prevented the task from getting completed before expiration elapsed.

Phone Calls

VoIP push notifications. A push notification type that, when activated, presents to the user the fact that they’re being called and the user can answer the phone call.

Here’s how you register your app to do VoIP pushes:

Important: You must report incoming calls with CallKit in didReceiveIncomingPush callback. If you fail to report incoming calls, the system will stop launching your app for VoIP pushes.

  • It’s recommended to set apns-expiration parameter to 0 to ensure that the push notification arrives at the same time a call is being made.
  • It’s also recommended to include as much caller info in push payload as possible, for the best UX on the receiver end.

Background Pushes (can be used for a feature like Muted Threads)

If a user chooses to mute a chatty thread, we can still receive data for that thread to be saved in the app for later viewing, however the user will not be notified. To do this, send a push with: “content-available: 1” but without “alert”, “sound” or “badge”. The system then decides when the best time is to launch your app to download the content, based on power and performance conditions.

  • Additionally, you need to set: apns-priority = 5 (or app will not launch), and also: apns-push-type = background

Discretionary Background URL Session allows to defer non-critical downloads to a later point in time, such as when the device is charging. This is accomplished by setting .discretionary = true on your URLSessionConfiguration object:

Some other optional parameters are: timeoutInterval, earliestBeginDate, and expected payload size:

To summarize this section:

To finalize background message sending: We use background task completion.

To launch phone calls through notification: We use VoIP pushes.

To deliver data in background for muted threads: We use Background pushes.

To defer downloads of past attachments: We use Discretionary URL Sessions.

New BackgroundTasks Framework

A new framework for deferrable maintenance tasks, such as: Syncing, DB cleanup or Backups (that should be done in the background). Apps are currently doing too much of this maintenance work right as they enter the background. This work ideally should be deferred to a later time (when the device is on idle).

“Gives your app several minutes of runtime at system-friendly times for deferrable maintenance work such as Core ML training/inference”.

While the device is plugged in, you are now able to turn off CPU Monitor in order to perform intensive work (taking full advantage of the hardware).

App needs to have been recently used in the foreground, or is in foreground at the time of operation request.

Background App Refresh Task:

  • New API, same policies
  • 30 seconds of runtme to keep app up-to-day throughout the day.
  • Eligibility based on usage patterns. The system actually learns when are the usage times that your app is used, and launches the background scheduling tasks before that time, so the app gets a chance to update it’s contents.
  • If your app is not used frequently, it’s not going to be background upgrade frequently either.

BGTaskScheduler is the main class we will be using to schedule background work. BGAppRefreshTaskRequest object is used to ask for your app to get refreshed later. (used to pull new data). BGProcessingTaskRequest is used for other maintenance operations. Can also be submitted from extensions.

You can queue multiple pending BGProcessingTaskRequest. They are queued by BGTaskScheduler and then delivered back to your app to be executed in background. The app should call “set task completed” after it’s done, allowing your app to suspend again. The app has finite time to perform these tasks, and tasks are delivered concurrently. The main app always handles processing tasks, and never the extension code.

  • Background Modes capability needs to be added to the target. (with “Background fetch” and “Background processing” mode)
  • Add a plist key: “Permitted background task scheduler identifier” — array of reverse dns notation strings. (Such as: com.colorfeed.refresh, com.colorfeed.dbcleaning)

Implementation in AppDelegate:

Scheduling the background refresh task:

You can also call scheduleAppRefresh() in handleAppRefresh() as the first call. This will queue another refresh operation without the user having to foreground the app. (This chains refreshes onto other refreshes in the background)

And here is a similar background task, but for a database refresh operation, as opposed to background fetch. The last cleaned time is kept on file for the database operation.

This method ensures that cleaning only occurs once a week, and not more frequently than that.

BGProcessingTaskRequet.requiresNetworkConnectivity defaults to false, so make sure to set it to true if you need internet for the task to work.

BGProcessingTaskRequet.requiresExternalPower — recommended for more intense power. Preserves user’s battery life, and turns off CPU monitor automatically — allowing greater leverage of the user’s hardware.

To make it easier to test using the debugger, Click the Pause button to enter the debugger. Then paste a command: e -l objc — (void)[[BGTaskScheduler sharedScheduler] _simulateLaunchForTaskWithIdentifier:@”com.colorfeed.refresh”]

This will explicitly simulate the launch of your task so you can test it.

Considerations:

  • earliestBeginDate should not be set to too far into the future. Keep this window to 1 week or less.
  • Make sure all files needed by background tasks have accessibility of at most: FileProtectionType.completeUntilFirstUserAuthentication. (because your task may be launched while the phone is locked and sleeping).
  • Call BGTaskScheduler.submit(_:) on a background queue, because it’s a blocking, synchronous call. (So you don’t impede your app’s launch performance)

Binary Frameworks

  • Can now distribute closed-source binary libraries using XCFramework format.
  • XCFrameworks work in all versions of XCode going forward and contain variants for both the simulator and the device.
  • A single XCFramework also supports all platforms supported by XCode (iOS, Watch OS, Mac OS, TVOS)
  • Varient for AppKit vs UIKit
  • Can bundle up static libraries and their headers.
  • Supports Swift and Objective-C.
  • You need to make sure you trust all of your frameworks, and their sub-frameworks as well, because they get access to all your permissions and entitlements.
  • You need to add permissions and entitlements manually in your app, if frameworks require them.

Swift UI

New UI framework targeted at writing less code and eliminate entire categories of errors, and offering readability. It’s also targeted at reusing your layout code across all Apple platforms.

Here’s an example of the code needed to create a simple list view using Swift UI:

The views are refreshed automatically when underlying data changes.

4 Core principles:

Swift UI, Principle #1. Declarative syntax. Example:

Changing the label to a button is done by adding an action: attribute: and changing the type to Button .

Swift UI, Principle #2. Provide automatic functionality whenever possible. Provides the following features for free, by convention:

  • Spacing and Insets
  • Localizability
  • Right-to-left
  • Dynamic type
  • Dark mode

Conventional animations are also enabled with one line of code: .animation(.default)

Transition views in-and-out with: .transition(.offset(size))

Animations are all interruptible, so your app is more responsive, letting the user do other actions while an animation is still taking place.

Swift UI, Principle #3. Compositional APIs are easier to learn and let you iterate a lot faster.

Compose UI by applying modifiers (fluent syntax), on top of a view. This is a compositional approach that lets you learn less repeatable modifiers and views, and combine them together to achieve modern interfaces.

Swift UI, Principle #4: Your data and UI should be in sync at all times.

Swift UI updates your interface automatically every time the data changes.

There are 2 places where your data comes from: Model objects, which now can be conformed to the new BindableObject protocol. Example:

class Alarms : BindableObject

It has a requirement that you specify when the data in your model changes.

And the second place is UI state, and there is a new “state wrapper” that you can apply to any property on your view:

@State var editing = true

Formatting a date in-line:

Text(“\(alarm.date, formatter: formatter)”) // fully localized, formatted date

To summarize, the 4 core principles of Swift UI are:

Declarative, Automatic, Compositional, Consistent.

Demo: Swift UI

Takes care of basic fundamental UI elements at a high level, providing all best practice capabilities, so you can spend more time focusing on custom elements or features that make your app unique.

Code matches the view hierarchy very closely. (Code is declarative, composed, concise and sequential). Declarative over Imperative.

There’s a new form of interface builder that renders your Swift UI in real time. You can drag SwiftUI objects from the Library onto the visual representation, and it generates the code-behind for you. (There are no constraints)

You can control-click objects in this new UI designer, and apply attributes such as: font, weight, color, alignment, line limit, etc. It updates the generated code automatically.

A “preview” is a snippet of code in my application that configures it for design time.

You can have many different previews for each Swift UI element, with different styling attributes or data, and different dynamic type sizes.

withAnimation { } block adds a default animation around your state change action. Play button in the preview allows testing code changes in real time.

Every animation is cancellable and reversible, keeping app responsive the whole time.

Specify duration like this:

withAnimation(.basic(duration: 2.0)) { }

Here’s a list item with a NavigationButton around each cell (shows the chevron next to each item), and a delete option for each item:

You also can now work on layouts without ever building and running the application by using Previews.

Swift UI makes our UI code Debuggable, Diffable, Searchable, Understandable and Flexible. Provides the same controls as any other native iOS app, and is available on all Apple environments (UIKit, AppKit, WatchKit). Can be used selectively.

A new style of interactive documentation has been developed to assist with learning the new tooling.

ABI Stability (previously introduced)

Reduces size of your apps by using a single, shared Swift runtime.

Module Stability (new)

Ensures compatibility for your binaries with the current and future versions of the swift compiler.

XCode configurability

We can now arrange XCode editors in more flexible layouts. (Maximize, put back to sub-section).

Blank panes without content automatically disappear.

Method documentation auto-updates with missing parameters. Enhanced documentation styling. Enhanced code highlighting.

Changing method param names automatically refactors the scoped mentions of that param:

File commit history is now shown in the Side inspector.

Showing local changes now allows showing previous code in-line to compare your changes against previous code. (no need to use a source control tool for this). Updates the before-after code comparison live as-you-type.

Minimap feature

Structural overview of your file. Hold down ctrl key and it shows you all method names and “symbolic landmarks” for the file.

Test Plans

New feature that allows combining multiple configurations to run your tests against, to test for your global audience (for example: Locations, Localizations, Locales). It can be used to generate App Store screenshots as well, or collect details for your localizers.

“See your app from every angle” by adding combinations of Sanitizers, Diagnostic, Parameters and device combinations.

One command now does all the testing for our apps.

Device conditions can let you set network conditions and thermal conditions. When the conditions are active, a banner is displayed at the top.

TestFlight Feedback

Users can now share screenshots of your apps along with a comment, for support purposes. You can review these on app store connect, and download details for bug tracking systems.

Alexei: App maker, interested in many topics.

Written by

Follow me for summaries of top-selling business books. My programming career book on Amazon: “Technology Stories - 125 Secrets for a Programming Career”