Open in app

Sign in

Write

Sign in

Making Flutter and REST API Work Together — Seamless Data Caching Integration (Part 11)

Alex Josef Bigler
Full Struggle Developer
11 min readJul 7, 2024

--

Welcome to Part 11 of the “Making Flutter and REST API Work Together” series! First, let me apologize for the long hiatus — it’s been over a year since our last article. Life happens, and I accidentally messed up the entire project in my IDE 😢.

Ironically, this turned out to be a blessing in disguise. It forced me to rebuild the project from scratch, following my own articles step by step. And guess what? It worked! 🫡

In this part, we’ll dive into data caching. We’ll explore how to seamlessly integrate data caching into our Flutter application using Hive to optimize performance and reduce the number of API calls.

Given that we’ve already written hundreds of lines of code, crafted complex widget logic, and more, the last thing we want is to break everything by switching to local data storage. But here’s the kicker — in this article, I’ll show you how to painlessly make this transition, ensuring your project remains intact and fully functional.

The following posts are suggested reading for this post to get you started:

For our work, we will be using a custom fake API based on json-server. It contains a sufficient amount of data for debugging purposes, as well as resources that are interrelated, which we can add at any time if we need something new.

If you want to deploy such a server yourself, you can read my article here 👈.

The Problem

In almost any application, numerous screens display data that rarely changes and is fetched from an API. This could include the main page content or core sections, or static data items transferred via API. Users often navigate between these sections without any specific purpose, causing our app to repeatedly request the same data from the API. This approach is inefficient and results in unnecessary API calls.

Why implement local caching?

  • Reduce redundant API requests: By caching data locally, you store API responses, allowing your app to retrieve this data quickly without making repeated calls to the server. This enhances performance and optimizes network usage, providing a smoother experience for users.
  • Enable new features: Local caching opens up new possibilities for implementing additional features. Since the data is already stored locally, there’s no need to design new API methods to fetch specific data. This flexibility allows for quicker development cycles and implement more functionalities without being constrained by API limitations.

However, combining cached data with live API requests requires a thoughtful approach. One of the key principles to follow is to avoid creating unnecessary entities (Occam’s Razor). This means not caching data that is rarely used or frequently changes. By adhering to this principle, we ensure that our app remains efficient and maintainable, leveraging caching where it provides the most value.

The Catch

The main point of our approach is to write converters that transform API objects into Hive objects and vice versa. This strategy allows us to leverage all the existing logic and functionality we’ve already developed for our widgets. Since the application will continue to operate with the same API objects and classes, there will be no need to rewrite or adapt the existing codebase extensively.

Okay, let’s do it

First, add the necessary dependencies in our pubspec.yaml file:

dependencies:

  hive: ^2.2.3
  hive_flutter: ^1.1.0
  shared_preferences: ^2.0.5


dev_dependencies:

  build_runner: ^2.4.9
  hive_generator: ^2.0.1

Flutter Benchmark Tuesday: SQLite vs. Hive — how popular articles deceive us

Hello everyone. This is another story of our regular segment “testing obvious things on Tuesdays”. And this time, the…

medium.com

Then, start creating Hive models for artists and albums. Essentially, these models are similar to the classes defined in our API provider, but tailored for Hive with specific annotations and structure.

import 'package:hive/hive.dart';
import 'artist_hive_model.dart';

part 'album_hive_model.g.dart'; // <-- This file will be generated

@HiveType(typeId: 1) // <-- This should be a unique type id for this model
class AlbumHiveModel extends HiveObject {
  @HiveField(0) // <-- Field number must be unique within this model and should not change once assigned
  int id;

  @HiveField(1)
  String title;

  @HiveField(2)
  int year;

  @HiveField(3)
  String cover;

  @HiveField(4)
  String color;

  @HiveField(5)
  String label;

  @HiveField(6)
  int artistId;

  @HiveField(7)
  int listens;

  @HiveField(8) // <-- This is an embedded object of type ArtistHiveModel, allowing null
  ArtistHiveModel? artist;

  AlbumHiveModel({
    required this.id,
    required this.title,
    required this.year,
    required this.cover,
    required this.color,
    required this.label,
    required this.artistId,
    required this.listens,
    this.artist, // <-- Allowing null for the artist field
  });
}

The album model outlines the structure of album data and includes an embedded artist object. This allows related data to be stored together, simplifying access and boosting performance. We again use Hive annotations to define the model.

import 'package:hive/hive.dart';

part 'artist_hive_model.g.dart'; // <-- This file will be generated

@HiveType(typeId: 2) // <-- This should be unique id
class ArtistHiveModel extends HiveObject {
  @HiveField(0) // <-- Field number must be unique within this model and should not change once assigned
  int id;

  @HiveField(1)
  String name;

  @HiveField(2)
  String bio;

  @HiveField(3)
  List<String> genres;

  @HiveField(4)
  String country;

  @HiveField(5)
  int yearsActive;

  @HiveField(6)
  int listens;

  @HiveField(7)
  String image;

  ArtistHiveModel({
    required this.id,
    required this.name,
    required this.bio,
    required this.genres,
    required this.country,
    required this.yearsActive,
    required this.listens,
    required this.image,
  });
}

And some additional comments for better understanding:

  1. Unique type ID: The @HiveType annotation requires a unique type ID for each model. This ID is used internally by Hive to identify the type of the object.
  2. Unique field numbers: Each field in the model is annotated with @HiveField and a unique number. These field numbers must be unique within the model and should not be changed once assigned, as they are used for encoding and decoding the data. Changing these numbers can lead to data corruption.
  3. Embedded objects: Fields like @HiveField(8) ArtistHiveModel artist demonstrate that Hive supports nested objects. This allows complex data structures to be stored and retrieved efficiently.
  4. Generated files: The part directive (part 'album_hive_model.g.dart') is necessary for code generation. This file will be generated by running the build runner command and will contain the necessary code for serialization and deserialization of the model.

To generate the necessary adapter files, run the following command:

flutter packages pub run build_runner build



(base) ➜  flutter_api git:(main) ✗ flutter packages pub run build_runner build
Deprecated. Use `dart run` instead.
Building package executable... (3.9s)
Built build_runner:build_runner.
[INFO] Generating build script completed, took 160ms
[INFO] Precompiling build script... completed, took 3.9s
[INFO] Building new asset graph completed, took 781ms
[INFO] Checking for unexpected pre-existing outputs. completed, took 0ms
[INFO] Generating SDK summary completed, took 2.7s
[INFO] Running build completed, took 3.3s
[INFO] Caching finalized dependency graph completed, took 58ms
[INFO] Succeeded after 3.3s with 9 outputs (24 actions)

Next, we have to initialize Hive adapters and open boxes.

Adapters in Hive are responsible for converting your custom objects into a format that Hive can store and retrieve. The names of these adapters typically follow the format YourModelNameAdapter, such as AlbumHiveModelAdapter and ArtistHiveModelAdapter. These names are generated automatically by the build_runner tool.

Boxes in Hive are containers where your data is stored. Hive is designed to handle concurrent access efficiently, so opening boxes in the main.dart file is safe and allows different parts of your app to access the same data simultaneously. By opening boxes at the start, you ensure that the necessary data containers are always available, improving the reliability and performance of your app.

import 'package:flutter/material.dart';
import 'package:hive/hive.dart';
import 'package:hive_flutter/hive_flutter.dart';
import 'album_list_widget.dart';
import 'album_hive_model.dart';
import 'artist_hive_model.dart';

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  // Initialize Hive
  await Hive.initFlutter();

  // Register Adapters
  Hive.registerAdapter(AlbumHiveModelAdapter());
  Hive.registerAdapter(ArtistHiveModelAdapter());

  // Open Boxes
  await Hive.openBox<AlbumHiveModel>('albums');
  await Hive.openBox<ArtistHiveModel>('artists');

  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({super.key});

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter REST API Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: AlbumListWidget(),
    );
  }
}

Writing сonverters

Next, we have to focus on one of the key aspects of our article: creating converters for Hive models. Converters are essential for transforming data between your API models and Hive models, enabling seamless data storage and retrieval. We’ll create these converters in a dedicated file named converters.dart.

import 'album_hive_model.dart';
import 'api_data_provider.dart';
import 'artist_hive_model.dart';

class ArtistConverter {
  static ArtistHiveModel fromApi(Artist artist) {
    return ArtistHiveModel(
      id: artist.id,
      name: artist.name,
      bio: artist.bio,
      genres: artist.genres,
      country: artist.country,
      yearsActive: artist.yearsActive,
      listens: artist.listens,
      image: artist.image,
    );
  }

  static Artist toApi(ArtistHiveModel hiveModel) {
    return Artist(
      id: hiveModel.id,
      name: hiveModel.name,
      bio: hiveModel.bio,
      genres: hiveModel.genres,
      country: hiveModel.country,
      yearsActive: hiveModel.yearsActive,
      listens: hiveModel.listens,
      image: hiveModel.image,
    );
  }
}

class AlbumConverter {
  static AlbumHiveModel fromApi(Album album, ArtistHiveModel artistHiveModel) {
    return AlbumHiveModel(
      id: album.id,
      title: album.title,
      year: album.year,
      cover: album.cover,
      color: album.color,
      label: album.label,
      artistId: album.artistId,
      listens: album.listens,
      artist: artistHiveModel,
    );
  }

  static Album toApi(AlbumHiveModel hiveModel, Artist artist) {
    return Album(
      id: hiveModel.id,
      title: hiveModel.title,
      year: hiveModel.year,
      cover: hiveModel.cover,
      color: hiveModel.color,
      label: hiveModel.label,
      artistId: hiveModel.artistId,
      listens: hiveModel.listens,
      artist: artist,
    );
  }
}

Now let’s move on to the most interesting part — implementing the caching logic.

The Logic

We have roughly outlined what we want to achieve:

  • Caching “new” albums on app launch: We want newest albums (on the home screen) to be instantly available without waiting for them to load.
  • Periodic cache updates: Keep the data up-to-date by refreshing the cache every 24 hours.
  • Pagination: Load new data as the list is scrolled and cache it.
  • Filtering and searching directly from the API: These functions may involve data that hasn’t been cached yet, so they should interact directly with the API.

First, we need to ensure that we load new albums first. Typically, this means adding default sorting by ID in the API provider. Then, we will set up periodic data caching at app launch, ensuring it refreshes every 24 hours.

Step 1: Updating the API Provider

Add a sorting parameter by ID in descending order (_sort=id&_order=desc) to the fetchAlbums method. This ensures that new albums always appear on the first pages.

  Future<List<Album>> fetchAlbums({
    required int page,
    required int pageSize,
    int? minYear,
    int? maxYear,
  }) async {
    String url = "$_baseUrl/albums?_page=$page&_limit=$pageSize&_sort=id&_order=desc";
    if (minYear != null) {
      url += "&year_gte=$minYear";
    }
    if (maxYear != null) {
      url += "&year_lte=$maxYear";
    }

    final response = await http.get(Uri.parse(url));

    final responseBody = processResponse(response);

    List jsonResponse = json.decode(responseBody);
    return jsonResponse.map((album) => Album.fromJson(album)).toList();
  }

Step 2: Setting up periodic caching in main.dart

Now, we will cache data at the app launch and refresh it every 24 hours. Lets create cacheInitialData() function:

Future<void> cacheInitialData() async {
  final apiProvider = ApiProvider(); // Initialize the API provider
  final albumBox = Hive.box<AlbumHiveModel>('albums'); // Get the album box
  final artistBox = Hive.box<ArtistHiveModel>('artists'); // Get the artist box
  var prefs = await SharedPreferences.getInstance(); // Get shared preferences
  var lastUpdate = prefs.getInt('lastAlbumsUpdate') ?? 0; // Get the last update time
  var currentTime = DateTime.now().millisecondsSinceEpoch; // Get the current time in milliseconds

  print('Last update time: $lastUpdate');
  print('Current time: $currentTime');

  // Check if the data needs to be updated or if the box is empty
  if (currentTime - lastUpdate > 86400000 || albumBox.isEmpty) {
    print('Updating cache...');
    // Fetch and cache the first two pages of albums
    for (int page = 1; page <= 2; page++) {
      print('Fetching albums for page $page');
      final albums = await apiProvider.fetchAlbums(page: page, pageSize: 20); // Fetch albums
      final artistIds = albums.map((album) => album.artistId).toSet().toList(); // Get unique artist IDs
      print('Fetching artists for IDs: $artistIds');
      final artists = await apiProvider.fetchArtists(ids: artistIds); // Fetch artists

      // Cache artist data
      for (var artist in artists) {
        artistBox.put(artist.id, ArtistConverter.fromApi(artist));
        print('Cached artist: ${artist.name}');
      }

      // Cache album data
      for (var album in albums) {
        final artist = artistBox.get(album.artistId);
        if (artist != null) {
          albumBox.put(album.id, AlbumConverter.fromApi(album, artist));
          print('Cached album: ${album.title} by artist ${artist.name}');
        } else {
          print('Artist not found for album: ${album.title}');
        }
      }
    }
    // Update the last update time
    await prefs.setInt('lastAlbumsUpdate', currentTime);
    print('Cache updated.');
  } else {
    print('Cache is up to date.');
  }
}

And add it to themain():

void main() async {
  WidgetsFlutterBinding.ensureInitialized();

  // Initialize Hive
  await Hive.initFlutter();

  // Register Adapters
  Hive.registerAdapter(AlbumHiveModelAdapter());
  Hive.registerAdapter(ArtistHiveModelAdapter());

  // Open Boxes
  await Hive.openBox<AlbumHiveModel>('albums');
  await Hive.openBox<ArtistHiveModel>('artists');

  // Cache the first two pages of data on app launch
  await cacheInitialData();

  runApp(const MyApp());
}

Step 3: Updating widget

Now, we can update the widget to use cached data and load new data when necessary.

Define boxes inside _AlbumListWidgetState:

late Box<AlbumHiveModel> _albumBox;
late Box<ArtistHiveModel> _artistBox;

then create _loadInitialData():

void _loadInitialData() {
  print('Loading initial data from cache...');
  setState(() {
    _albums.addAll(
      _albumBox.values.map((hiveAlbum) {
        final artistHiveModel = _artistBox.get(hiveAlbum.artistId);
        final artist = artistHiveModel != null ? ArtistConverter.toApi(artistHiveModel) : null;
        print('Loaded album from cache: ${hiveAlbum.title} by artist ${artist?.name ?? 'Unknown'}');
        return AlbumConverter.toApi(hiveAlbum, artist!);
      }).toList(),
    );
  });
  print('Initial data loaded from cache.');
}

add it to initState() with boxes initialization:

 @override
  void initState() {
    super.initState();
    _albumBox = Hive.box<AlbumHiveModel>('albums');
    _artistBox = Hive.box<ArtistHiveModel>('artists');
    _loadInitialData();

    _scrollController.addListener(() {
      if (_scrollController.position.pixels > _scrollController.position.maxScrollExtent * 0.7 && !_isLoading) {
        _fetchAlbums();
      }
    });
  }

and finally modify existing _fetchAlbums() function:

_fetchAlbums([String? query, RangeValues? yearRange]) async {
  setState(() {
    _isLoading = true;
  });

  List<Album> newAlbums;
  if (query?.isNotEmpty == true) {
    _clearAlbums();
    print('Searching albums with query: $query');
    newAlbums = await _apiProvider.searchAlbums(query!);
  } else {
    print('Fetching albums for page $_currentPage');
    newAlbums = await _apiProvider.fetchAlbums(
      page: _currentPage,
      pageSize: _pageSize,
      minYear: yearRange?.start.toInt(),
      maxYear: yearRange?.end.toInt(),
    );
  }

  await _fetchArtists(newAlbums);

  for (var album in newAlbums) {
    final artistHiveModel = _artistBox.get(album.artistId);
    if (artistHiveModel != null) {
      _albumBox.put(album.id, AlbumConverter.fromApi(album, artistHiveModel));
      print('Cached album: ${album.title} by artist ${artistHiveModel.name}');
    } else {
      print('Artist not found for album: ${album.title}');
    }
  }

  setState(() {
    _isLoading = false;
    _albums.addAll(newAlbums);
    _currentPage++;
  });
  print('Fetched and cached albums for page $_currentPage');
}

Aaaaand that’s it! No, really, you’ve successfully added caching to your application.

Now, without wasting any more time, you can start implementing new features immediately (for example, a widget with recently viewed albums, a list of the newest albums, and so on) because we already have data models for Artist and Album. Your imagination is the limit. And importantly, you won’t need to ask backend developers to update the API for this anymore.

Summary

The key takeaway is that we have implemented a straightforward caching strategy that does not require major code refactoring or complex data structures. This approach provides significant advantages:

  • Immediate data access: The main screen no longer makes unnecessary API requests. Data is loaded instantly from local storage, enhancing the user experience.
  • Simplicity: The logic remains uncomplicated. Data that might be missing from the cache is still fetched from the API, supporting functions like filtering and search.
  • Flexibility for new features: With locally stored data, we can now implement a wide range of new features and work with the saved data as if it were local.

Overall, this caching strategy provides a robust improvement to our application’s performance and flexibility without adding unnecessary complexity.

In future articles, we can cover topics such as:

  • Implementing authentication and authorization in a Flutter application using a REST API.
  • Using Firebase for data storage and real-time implementation.
  • Implementing push notifications in a Flutter application using Firebase Cloud Messaging.

Thank you for reading, and I hope this article has helped you. Subscribe and don’t miss new materials!

Flutter
Rest Api
Hive
Dart
Programming

--

--

Alex Josef Bigler
Full Struggle Developer

Written by Alex Josef Bigler

150 Followers
Editor for

Enthusiast of new technologies, entrepreneur, and researcher. Writing about IT, economics, and other stuff. Exploring the world through the lens of data.

Help

Status

About

Careers

Press

Blog

Privacy

Terms

Text to speech

Teams