This release adds a new trick and fixes some issues we missed in the initial v0.5 release.
New: Create packages with Linux Executables
I was preparing a demo on how to use Lambda layers to include an executable that can be invoked by a Lambda function when I came across a limitation. It turns out zip archives created on Windows cannot capture the execution permission required for a Linux executable.
At first, this issue confounded me, because none of the zip utilities were showing a difference between a zip archive created on Linux and one created on Windows. Yet, the former would set permissions and the latter would not when deployed by a Lambda layer. I was already familiar with the
ExternalAttributes property on on
ZipArchiveEntry, but setting the desired permissions had no effect.
After hours of researching the issue, I finally discovered
unzip -Z which shows additional metadata about a zip archive on Linux.
~$ unzip -Z /package_FFmpeg.zip Archive: /package_FFmpeg.zip Zip file size: 24394645 bytes, number of entries: 2 -rwxr-xr-x 3.0 unx 70974184 bx defN 19-Feb-23 22:02 ffmpeg -rw-r--r-- 3.0 unx 35821 bx defN 19-Feb-23 22:02 GPLv3.txt 2 files, 71010005 bytes uncompressed, 24394361 bytes compressed: 65.6%
It so happens the zip specification stores which host operating system was used to create the archive. In order for the zip archive to capture execution permission, it must have been created on a Unix system (denoted by
unx). However, when created one on Windows, it shows
fat instead. Why it’s an operating system in one case and a file system in another is probably lost to history…
ZipArchive class in .NET does not allow specifying the host operating system. However,
SharpZipLib does, short of a few fixes which hopefully will be merged soon. In the meantime, λ# CLI uses its own fork of the code.
In v0.5.0.1, the λ# CLI scans
Package files for Linux executables by checking for the ELF header. When detected, the executable is added with the execution permission set and the host operating system declared as Linux, thus allowing the Lambda function to invoke the executable in a Lambda layer. In addition, the λ# CLI checks if an up-to-date zip archive already exists by computing the checksum of the files first. If the checksum matches, the packaging step is skipped, saving valuable time when building and deploying as CloudFormation will also skip the Lambda layer update.
NOTE: The λ# CLI does NOT check if the executable is compatible with the Amazon Linux operating system. It merely checks that it is a Linux executable.
Sample: Create Animated GIFs from Videos with AWS Lambda
You can see the new code in action by deploying the GifMaker-Sample module, which uses the
ffmpeg utility to convert a video into an animated GIF file.
ffmpeg utility is added to a zip archive by using the
Package declaration. Nothing additional is required to capture the proper permissions.
- Package: FFmpegPackage Files: Assets/FFmpeg-Layer/
Then, the package is referenced by the Lambda layer resource declaration.
- Resource: FFmpegLayer Type: AWS::Lambda::LayerVersion Properties: LicenseInfo: GPL-3.0-only Content: S3Bucket: !Ref DeploymentBucketName S3Key: !Ref FFmpegPackage
Finally, the Lambda function just needs referenced the Lambda layer.
- Function: MakeGifFunction Memory: 1024 Timeout: 300 Properties: Layers: - !Ref FFmpegLayer
The λ# CLI takes care of compressing, uploading, and deploying everything for you. The whole process is idempotent and thus well suited for CI/CD pipelines. It’s that simple!
In addition, we included a couple of fixes in this release.
- InvalidCastException when setting Secrets in the parameters file
- Increase memory for S3Writer function in LambdaSharp.S3.IO module