Aperture Flutter — How bad is it?
My experience testing Canon and Zeiss lens apertures
One thing I read a lot is that serious timelapse photographers use fully manual lens. The need to avoid auto-focus and hence focus changing undesirably during a sequence of images is obvious; however, the need for a manual aperture not so clear.
Most modern lenses use an electronic aperture or diaphragm. The aperture is wide open helping the camera auto-focus quickly. Then when the shutter (or depth of field button) is pressed the aperture is narrowed by electrical impulses from the camera to the lens.
How accurate and repeatable is aperture actuation from one shot to the next? Why does this matter?
Aperture flutter is a phrase used to describe the lens missing the desired aperture, going to far or not far enough. It can also describe the aperture changing while the photo is being taken due the blades of the aperture oscillating or fluttering. Aperture flutter causes images in a sequence to vary in exposure and depth of field. Exposure can be corrected easily in post-processing especially if shooting in RAW; however, depth of field cannot be corrected.
I like capturing timelapses and image sequences. I’ve rarely had to worry about exposure variations, although I always apply some post-processing on long sequences. Recently I started using a Zeiss Milvus 35/2 ZE lens and suddenly exposure variations were very much a problem for me.
I did some quick testing and found that the lens was varying 0.15EV between shots at f/4. I emailed Zeiss to ask if this was normal or if my lens had a fault. I got a quick response telling me this was normal: “a variation of +/- 0.15 EV of an instant-return aperture of EF mount lenses is typical an [sic] cannot be avoided or influenced”.
The phrase “cannot be avoided” did not sit with my own experiences. The experimental scientist within me was stirred into action. It was time for some hard data, and so I devised an experiment to test a set of lenses.
Note: skip the next section if you just want to see the results.
Experiment
I wanted to do a thorough test so that I could feel I really understood how much of an issue aperture flutter was.
I setup a Canon EOS 40D pointed at a continuous photographic LED lamp. I borrow a set of lens. Most are Canon L-series so should be equivalent to the Zeiss Milvus in build quality.
For each lens I set the focused at infinity, which kept the lamp out of focus at all apertures. For zoom lens I set them to their maximum focal length.
With each lens I took 20 images at four different ISO settings (100, 200, 400, 800) adjusting the shutter speed to keep the lamp between 0EV and 1EV on the camera spot meter. I repeated this again with an ND filter attached to get a second set of slower shutter speeds.
The point here is to get a range of shutter speeds so that the camera’s shutter can be disregarded as the cause of exposure variations.
To summarise that’s 20 x 4 x 2=160 images per aperture per lens.
In total I took over 5000 RAW images for this experiment.
The capture environment was a separate room in my home, used at night with blackout curtains in place. No one was present in the room while images were captured as the camera was operated remotely. The room temperature was a constant 19C +/- 0.5C. That’s about as controlled as I could reasonably achieve at home without building an enclosure.
I computed the EV value of each image captured using the mean lumiance of 128x128 pixel crop from the centre of each sRAW file and applying a base 2 logarithm. The RAW files were processed using dcraw in linear 16-bit mode.
To determine the EV range I compare the minimum and maximum values within each sequence at same shutter speed and ISO. I focus on the extremes as these will have the most effect on image sequences.
I also took nearly 1000 test shots on a Canon 5DSr under slightly less controlled circumstances. These result set matched those of my main experiment (shown below). This gives me high confidence that the camera is not the cause of any variations.
Results
The graph above shows the range of EV variation (within 20 image sequences) for each lens and aperture combination. The points are the mean variation across all shutter speed and ISO settings for a given aperture, with the error bars indicating the minimum and maximums observed.
Executive summary: “ouch”
The Zeiss Milvus 32/2 ZE seems to have a very unstable electronic aperture. It frequently shows a 1/4-stop variation in exposure, and by implication a equivalent variation in depth of field. For a lens designed in 2015 this is, in my opinion, surprisingly poor.
When it comes to aperture stability the Zeiss Milvus is comfortably outperformed by every other lens I tested, including the much cheaper Canon 50mm f/1.4. To be honest I was surprised by how accurately this data reflects my personal experiences using Canon lenses.
Just to confirm this isn’t a shutter issue (or lamp flicker) I have produced a scatter plot of EV range by shutter speed (below). Note: high shutter speeds generally go with wide-open apertures so tend to show more stability. This confirms that lamp flicker isn’t a major source of variation.
We can look at the image sequences to actually see the variation (flutter) occuring. See the comparative data below showing the EV variation with respect to the mean over 20 image sequences.
This makes it clear that the issue is not one or two spikes, but rather that the Zeiss lens aperture is oscillating between extremes.
Finally I’ve uploaded the worst case variation data here (on Airtable) in case you would like some of the hard numbers. Again, I think the worst case data is the most important to real world impact because a single badly under or over-exposed frame will stand out.
Conclusion
Zeiss told me that I should use the Nikon mount ZF.2 version if I needed to get rid of aperture flutter (for timelapse or stop-motion). The Nikon version of the lens has a fully mechanical aperature, and can be adapted to Canon EF-mount. In the end this is what I did, because I do love the micro-contrast of the Milvus optics.
I’m left wondering what the quality of Zeiss electro-mechanical components is like in the lenses they design for Sony E-mount and other mirrorless cameras.
I also wonder how stable modern electronic apertures are in general?
I don’t think I have really answered whether aperture flutter something we need to worry about. All I can conclude is that it isn’t much of an issue for Canon L-series lenses, and appears to be a major problem for Zeiss Milvus ZE lenses.
I suspect I’ll be doing some more experiments in the coming months…
