Concurrency in iOS. Async and Declarative Layouts. Async Rendering.

The previous articles

GCD. Readers–Writers Problem

Async Data Source.

In today’s discussion on Concurrency and Asynchronicity in iOS, we’ll delve into three key topics:

1. Async Layout: Exploring asynchronous layout techniques.
2. Declarative Layout: Investigating both concurrent and UI-thread-based layouts, including SwiftUI and Auto Layout, to compare their underlying concepts and implementation outcomes.
3. Async Rendering: Analyzing the benefits and considerations of asynchronous rendering in iOS development.

By examining these areas, we aim to gain a comprehensive understanding of how concurrency and asynchronicity are utilized in iOS app development, particularly in the context of layout and rendering processes.

Async Layout

Definition

To delve deeper into our discussion, it’s essential to establish clear definitions. Let’s start by clarifying the concept of “Layout”:

Layout

1) The comprehensive dataset necessary for visual representation.

2) Encompasses UI properties such as sizes, positions, and other relevant attributes.

3) Includes supplementary rendering information essential for proper display.

For the asynchronous concurrent version, we can refine the definition as follows:

Async Concurrent Layout

1) Pre-computation of all UI properties.

2) Calculations executed outside the UI-thread, ensuring concurrency.

3) All properties computed prior to the moment of view binding, optimizing performance and responsiveness.

We might find a use case for Async Concurrent Layout in the following scenarios.

Highly Complex UI

When dealing with intricate user interfaces, optimizing layout calculations becomes crucial to maintain smooth performance and responsiveness.

Frequent UI Updates

In scenarios where certain UI components experience frequent and rapid changes, the demand for responsiveness can strain the UI thread. Examples include:

• Live stream comment lists.
• Active group chat screens.

To prevent UI thread overload and maintain responsiveness, it’s imperative to offload UI update calculations to a background thread. This ensures that the user experience remains smooth despite the high frequency of updates.

Scheme of Queues

In this section, we won’t provide specific code implementations but will delve deeper into the concept of an Async Data Source and its interplay with different Queues.

The finalized solution scheme was as follows:

Let’s enhance the system by introducing an additional queue dedicated to layout calculations:

You might wonder why I propose having a distinct queue for layout calculations, separate from the data source queue. Here are the reasons:

• Independence from UI: The data source primarily deals with processing business events, such as adding or modifying objects, without direct concern for UI updates. Conversely, the layout queue focuses solely on optimizing layout calculations for UI presentation.
• Optional Optimization: The layout queue serves as an optimization tool, which may not always be necessary. Its separate existence allows for easier integration or removal without affecting the functionality of the data source.
• Decoupled UI Handling: The UI and layout can operate independently to handle UI events like device rotation or animations, without reliance on the data source. This decoupling enhances flexibility and simplifies maintenance.

In essence, the primary objective is to maintain the autonomy of the data source from UI-specific events. Similarly, it’s crucial to ensure that the UI, including layout functionalities, remains ignorant of data source intricacies, particularly events unrelated to direct UI updates. This separation guarantees a modular and flexible system architecture, enabling efficient management of both business logic and user interface concerns without unnecessary coupling.

Handling Only-UI-Related Events

As previously discussed, certain UI-specific events, such as screen rotation, don’t directly impact the data source but necessitate synchronous recalculation of the layout.

Screen rotation is a prime example where the UI undergoes significant changes, requiring immediate adjustments to maintain visual coherence and user experience. Therefore, the ability to synchronously recalculate the layout ensures smooth transitions and optimal presentation, enhancing the overall usability of the application during dynamic UI interactions like screen rotation:

As you observe, the data source remains uninvolved in this process. When a screen rotation occurs and the layout queue is already processing an update based on recent data source changes, we take the following steps:

1. Synchronously Recalculating and Performing Current UI State: In the current animation block, we synchronously recalculate and perform the current UI state based on the latest UI constraints (i.e., the new screen size). This ensures that the UI adapts seamlessly to the new orientation without delay.
2. Cancelling and Ignoring Current Async Layout Operation: Any ongoing asynchronous layout operation in the layout queue that pertains to the previous data source state and UI constraints is canceled and disregarded. This prevents conflicts and ensures consistency between the UI presentation and data source.
3. Scheduling a New Layout Calculation Operation: A new layout calculation operation is scheduled in the layout queue, considering the most recent data source state and the updated UI constraints resulting from the screen rotation.
4. Applying the Result of the New Operation: Once the new layout calculation operation is complete, its result is applied to update the UI accordingly, ensuring that the latest data source changes are reflected accurately in the UI layout.

By following these steps, we maintain synchronization between the data source and UI layout, guaranteeing a seamless and responsive user experience even during dynamic UI events like screen rotation.

Compromises

Undoubtedly, merging the data source and layout states is possible if desired or necessary. In such a scenario, the compromises would resemble those outlined in the Compromises section from the previous article, with one notable addition: the need to validate and disregard data source updates if they are computed based on outdated UI constraints. This ensures that only relevant and up-to-date data source changes are considered for displaying on screen, maintaining consistency and accuracy in the presentation of information.

For example, if layout information is incorporated into the data source state, and we permit thread-safe, non-blocking reading of it from the UI thread, the process for handling a screen size change could resemble the following scheme:

Certainly, such compromises inevitably introduce complexity into state management. Additionally, they may potentially lead to performance issues since we cannot prioritize either data updates or layout updates. This lack of prioritization can result in situations where the system struggles to efficiently handle both types of updates simultaneously, potentially impacting overall performance and responsiveness.

Calculating Layout out of UI Thread

When we decide to perform layout calculation before updating the corresponding UI objects (UIView), the immediate concern is how to calculate the layout in advance. For straightforward UI hierarchies, the simplest approach might involve directly calculating view geometry, such as UIView.frame. However, for more complex UI structures or to ensure a more versatile and scalable solution, leveraging Declarative Layout becomes essential.

Declarative Layout

Indeed, declarative layouts are not solely designed for asynchronous calculation; their primary aim is to enable developers to specify “what to do” rather than “how to do it.”

By abstracting away the exact rendering objects (such as UIView) and operating with higher-level View/Widget objects, declarative layouts provide a more intuitive and expressive way to describe UI structures and behaviors.

However, this abstraction also lends itself well to asynchronous layout calculation, as it allows for more flexible and efficient processing of layout updates without being tightly coupled to specific rendering objects. Therefore, leveraging declarative layouts can be an effective approach for implementing asynchronous layout calculation in iOS development.

Definition

Let’s summarize the requirements:

• At the moment of UI description (Declaration), no UI objects (such as UIView, etc.) are involved.
• The interface is described using additional entities.
• The entire layout is calculated using these additional entities.
• The UI objects are populated with precomputed values.
• UI event handlers are described (declared) in advance.

Recall of an abstract scheme for “classical,” non-declarative layout. In traditional layout approaches, we map data directly to the rendered hierarchy.

Certainly, in a system with declarative layout, the input (Data, Model) and output (Rendered View Tree) remain the same. However, we introduce two additional intermediate steps to the layout update pipeline:

And typically, we interact solely with the first, Abstract Tree. The Layout and Rendered trees are encapsulated details of a declarative framework.

Here’s an abstract comparison of different layout systems:

Indeed, while Auto Layout may not be entirely declarative, it does exhibit some declarative characteristics. Additionally, the algorithm it’s based on, Cassowary, can be applied to custom interface units that are not reliant on UIKit.

Comparing Declarative Solutions

I’d like to highlight some key aspects of the frameworks that contribute to their API design and even their performance.

Declaration and Calculation Principles

Two opposing approaches exist.

Single Layout Rules, Single Layout Engine

The first approach involves all interface items adhering to the same constant rules (layout properties), with all calculations executed by a single layout engine. This approach is exemplified by:

• Yoga layout.
• Texture (AsyncDisplayKit), as it’s based on Yoga.
• Cassowary (the engine solves the system of Linear inequalities of the layout properties).
• Auto Layout, as it’s based on Cassowary.

In such systems, introducing new layout rules requires modifying both the API and the layout engine.

A distinguishing characteristic of these systems is that all layout properties are declared within the base Item class.

Decomposed (Decorative) Units

In the second approach, while there may still be a common layout engine and shared layout properties, each unit describes the layout rules for its children and implements the corresponding calculations. In essence, each Item serves as a decorator for its child or children items.

The examples:

• SwiftUI.
• Flutter.
• UIView to some extent, as each UIView can have its unique layout logic, especially since children layouts can be placed within the layoutSubviews implementation.

For me, a characteristic of such systems, apart from UIView, is the presence of Inset and other decoration Items.

Note. I made the differentiation solely based on the base layout and item design principles. Undoubtedly, a system with Single Layout Rules may provide some API for customizing the layout of child items, and conversely, every system with Decorative Unit Design has its root layout engine.

Relations Between Items

The next key aspect of layout systems is the types of relations allowed between Items.

Once more, two opposing approaches emerge.

Only Parent-Child Relations

“In systems of this type, relations are limited to no further than parent-child connections. Relations between siblings (items with the same parent) are not available, and connections cannot be established between items from different branches of the Item tree.

The examples of systems with only parent-child relations are:

• SwiftUI.
• Texture (AsyncDisplayKit).
• Flutter.

This restriction simplifies layout calculation by confining it within a single pass of the Item tree.

Relations Between Any Item

The opposing approach permits the establishment of relations between any items in the hierarchy. An example of such systems:

• Autolayout (Cassowary algorithm).

Indeed, this approach is believed to be a major contributor to the majority of performance issues with Auto Layout. Cassowary, if I’m not mistaken, utilizes the simplex method, which in the worst case has exponential complexity.

. According to Apple engineers from WWDC 2018 session 220 “High Performance Auto Layout” (though the video seems to have been removed), performance scales linearly, not exponentially, when there is no interaction between “far components.”

Async Rendering

The last aspect I’d like to consider is async rendering of view content. This approach is implemented in the Texture (AsyncDisplayKit) framework.

The concept behind this approach involves the addition of a rendering queue, separate from the layout queue. Rendering is considered a heavy operation, and it occurs after the view hierarchy has been laid out:

“And here’s how the concept might be implemented in code:

protocol Scheduler {
    func schedule(_ operation: @escaping () -> Void)
}

class LayoutNode <DeclarativeItem, View: CALayer> {
    let model: DeclarativeItem
    /// Balances Main Thread pressure
    let mainThreadScheduler: Scheduler
    /// Balances Render Queue pressure
    let renderQueueScheduler: Scheduler
    
    private var layer: View?
    private var cachedContent: UIImage?
    private var isRendering = false
    
    func bindView(layer: View) {
        self.layer = layer
        guard let cachedContent else {
            drawContent()
            return
        }
        layer.contents = cachedContent.cgImage
    }
    
    func drawContent() {
        if isRendering {
            return
        }
        isRendering = true
        renderQueueScheduler.schedule { [weak self] in
            guard let self else { return }
            let image: UIImage
            // draw content image
            self.mainThreadScheduler.schedule { [weak self] in
                guard let self else { return }
                self.isRendering = false
                self.cachedContent = image
                self.layer?.contents = image.cgImage
            }
        }
    }
    
    func unbindView() {
        layer = nil
    }
}

In the case of Texture, the mainThreadScheduler is responsible for balancing the pressure on the main thread. It ensures that only a limited number of operations are performed on each main run loop iteration.

Async rendering offers several advantages and disadvantages.

Advantages:

• Minimized UI (Main) thread utilization ensures smooth UI performance, including scrolling.
• The application remains fully responsive even during heavy rendering operations.

Disadvantages:

• Delayed rendering of layer content after the layout phase can result in empty UI elements being displayed momentarily.
• Some UI elements may appear blank until their content is rendered, potentially leading to a less polished user experience.

These characteristics may be familiar to those who have used apps built with the Texture framework.

Conclusion

In conclusion, our journey through concurrency and asynchronicity in iOS development has come to an end.

Throughout our discussions, the focus has been on highlighting key aspects and potential challenges that developers may face in this field.

Despite the intricacies, we’ve covered several important topics, including:

• Understanding Readers-Writers Problem and its solutions.
• Implementing Async Data Source to manage data retrieval and updates asynchronously.
• Exploring Compromises necessary for immediate access to Shared Resource States.
• Grasping the essentials of implementing Async Layout for efficient UI updates.
• Understanding the key aspects and differences of Declarative Layouts for flexible UI design.
• Exploring a basic implementation of Async Rendering for optimized performance.

I trust that you found this journey enlightening and that you’ve gained valuable insights along the way. It’s been a pleasure guiding you through these topics, and I hope you’ve enjoyed the experience as much as I have.