Using OpenJPEG from Dart with ffi to decode jp2 images

I’m working on extracting images from PDF, and as part of its response, I investigated how to decode JPEG 2000 compressed image data on Dart. Today, I’d like to introduce my notes.

I referred to OpenJPEG examples and applications for implementation.

openjpeg/src/bin/jp2/opj_decompress.c at 41c25e3827c68a39b9e20c1625a0b96e49955445 ·…

Also, my implementation is here.

GitHub - CORDEA/openjpeg_dart_ffi_demo

Steps 1 and 2 are common steps used in dart:ffi.

1. Generate bindings

I used to ffigen to generate bindings, but you don’t have to use it.

name: OpenJpegBindings
description: Bindings for `openjpeg.h`.
output: 'lib/openjpeg_generated_bindings.dart'
headers:
  entry-points:
    - 'openjpeg.h'
compiler-opts:
  - '-Wno-nullability-completeness'
preamble: |
  // ignore_for_file: type=lint, unused_element, unused_field

$ dart run ffigen --config ffigen.yaml

2. Load the library

const _lib = 'libopenjp2.2.5.0';
final _dylib = () {
  if (Platform.isMacOS || Platform.isIOS) {
    return ffi.DynamicLibrary.open('$_lib.dylib');
  }
  if (Platform.isAndroid || Platform.isLinux) {
    return ffi.DynamicLibrary.open('$_lib.so');
  }
  if (Platform.isWindows) {
    return ffi.DynamicLibrary.open('$_lib.dll');
  }
  throw UnsupportedError('Unknown platform: ${Platform.operatingSystem}');
}();

final _bindings = OpenJpegBindings(_dylib);

Future<void> decode(String fileName) async {
  // TODO
}

3. Set up the decoder

final parameters = calloc<opj_dparameters_t>();
bindings.opj_set_default_decoder_parameters(parameters);
final codec = bindings.opj_create_decompress(CODEC_FORMAT.OPJ_CODEC_JP2);
if (bindings.opj_setup_decoder(codec, parameters) <= 0) {
  throw ArgumentError('Failed to set up the decoder.');
}

The format is hardcoded here to focus on decoding (OPJ_CODEC_JP2). The defined magic number can be checked on RFC3745. For example,

$ hexdump -C test.jp2 | head -n 1
00000000  00 00 00 0c 6a 50 20 20  0d 0a 87 0a 00 00 00 14  |....jP  ........|

The OpenJPEG application has logic implemented to detect the format by the image header, which may be helpful.

openjpeg/src/bin/jp2/opj_decompress.c at 41c25e3827c68a39b9e20c1625a0b96e49955445 ·…

4. Decode the image

Note that it doesn’t contain any code to destroy resources.

final file = fileName.toNativeUtf8();
final stream =
    _bindings.opj_stream_create_default_file_stream(file.cast(), 1);
final imagePtrPtr = ffi.Pointer<ffi.Pointer<opj_image_t>>.fromAddress(
  ffi.calloc<opj_image_t>().address,
);
if (bindings.opj_read_header(stream, codec, imagePtrPtr) <= 0) {
  throw ArgumentError('Failed to read the header.');
}
final imagePtr = imagePtrPtr.value;
if (bindings.opj_decode(codec, stream, imagePtr) <= 0) {
  throw ArgumentError('Failed to decode.');
}
if (bindings.opj_end_decompress(codec, stream) <= 0) {
  throw ArgumentError('Failed to finalize.');
}

5. Extract color information

You can extract colors from the image after being decoded 💁

final image = imagePtr.ref;
final channels = image.numcomps;
final width = image.comps[0].w;
final height = image.comps[0].h;
for (var h = 0; h < height; h++) {
  final offset = h * width;
  for (var w = 0; w < width; w++) {
    final i = offset + w;
    switch (channels) {
      case 1:
        // Gray
        // image.comps[0].data[i];
      case 3:
        // RGB
        // r: image.comps[0].data[i]
        // g: image.comps[1].data[i]
        // b: image.comps[2].data[i]
      case 4:
        // RGBA
        // r: image.comps[0].data[i]
        // g: image.comps[1].data[i]
        // b: image.comps[2].data[i]
        // a: image.comps[3].data[i]
      default:
        throw ArgumentError();
    }
  }
}

Appendix. Enable logging

If you want to see OpenJPEG logs, set up handlers.

void onMessage(ffi.Pointer<ffi.Char> message, ffi.Pointer<ffi.Void> data) {
  stdout.write(message.cast<ffi.Utf8>().toDartString());
}

Future<Image> decode(String fileName) async {
  ...

  final callback =
      ffi.Pointer.fromFunction<opj_msg_callbackFunction>(onMessage);
  _bindings.opj_set_warning_handler(codec, callback, ffi.nullptr);
  _bindings.opj_set_error_handler(codec, callback, ffi.nullptr);
  _bindings.opj_set_info_handler(codec, callback, ffi.nullptr);

  ...
}

👋