Responsive Imaging and Dynamic Media done Right — Part III

19 min readJan 8, 2024

Dynamic Media and HTML 5

Today’s websites are experienced on a multitude of different devices. Not only do screens vary in sizes, but they also vary in resolution, orientation, and aspect ratio.

Having different devices requires us to provide images in different resolutions and formats. HTML5 gives some clues how this can be achieved. But the standard is quite complex and it’s easy to make errors. Also, HTML5 is a bit static, and it does not fully exploit advanced rendering features you can use when you have AEM Dynamic Media.

This is Part III of a four-part series. The series is divided as follows:

Part I: What is Responsive Imaging? Why we need it and what use cases it has.
Part II: How can we use AEM Dynamic Media, to increase page loading performance?
Part III: How can we implement Dynamic Media with HTML 5?
Part IV: Beyond HTML 5: Shrinking the page load time with a custom image loader.

HTML Markup for Responsive Images

You are still here? Great. I am glad you made it so far. In this chapter well focus on the HTML part of responsive imaging. The HTML5 standard provides a couple of new tags to support responsive images. If you are an experienced web developer, you probably have used the not-so-new srcset attribute and the <picture> tag already. This is what we will be working with.

Even if you are familiar with these tags already, I invite you to stay here anyway. This type of markup has a couple of hidden features you might not be aware of. Also, I will argue later, that techniques you use on apps or static websites are not necessarily applicable for sites generated by a Content Management System (CMS).

Images in Fluid and Responsive Layouts

By now we know how to crop and scale individual images. Each image can be referred to by individual URLs. So how does the browser know, which URL to use in a certain layout?

It depends on the use case. To illustrate, I have created a simple layout that covers a few typical use cases. The layout is elastic and responsive.

“Fluid” means all width definitions are in %: If you reduce the size of the browser window, the widths of all elements on the page will shrink proportionally.

“Responsive”: At some point — when we reach the breakpoint for a typical mobile device, we’ll apply a slightly different layout, i.e., multi-column elements will be stacked:

Wireframe for demo page with various use cases

The Use Cases

A: Hero Banner that consumes 100% of the screen width. The size is adapted to the actual screen width. The aspect ratio 3:1 is the same on all devices.

B: Two Small Teaser components displayed side by side. The images take up 50% of the available width and scale proportionally. On mobile devices with narrow screens, the teasers are displayed stacked and consume 100% of the screen width. The aspect ratio 1:1 is the same on all devices.

C: Text-Image component. On Desktop resolutions, the image is displayed right to the text and takes 33% of the viewport width. On mobile devices, the image is displayed on top of the text and the aspect ratio changes from 1:1 to 3:1.

D: Large Teaser with text and image. The image uses 33% of the screen width on desktops. On mobile screens the image is stacked and uses 100% of the screen width and is displayed as 3:1. In contrast to the Text Image component, the height of the image is variable. It is defined by the height of the text block: Longer copy texts lead to a more stretched format. The image can have any aspect ratio and does not conform to any of the pre-defined formats.

Test Environment

On [5] I have prepared a set of simple test pages you can use to follow along and validate my findings.

I recommend using multiple browsers in parallel as they behave slightly differently. I found that Firefox is the best platform to learn the new concepts. Chrome applies a couple of performance optimizations and obscures the core functionality. This can be a bit confusing.

For the experiments, I am using dummyimage.com to generate test images. These images display their intrinsic dimensions.

We call dummyimage.com with dimension parameters like so:

https://dummyimage.com/300x100/000/fff

Which renders a 300x100 pixel image and displaying the dimension as the images content like so:

dummyimage.com displaying intrinsic dimensions

This will come in handy soon.

In production, we would of course use Dynamic Media which provides a similar feature set — but with real images.

Hero Banner

Simple Scaling — Naive Approach

The Hero Banner takes up 100% of the viewport width. The aspect ratio is 3:1. Let’s start with a naive approach to warm up. Let us use a simple <img> tag:

<style>
  .hero{
    width: 100%;
  }
…
</style>

<div class="hero">
  <img src="https://dummyimage.com/300x100/D3C8D9/000" id="hero" class="hero"/>
</div>

The result looks like this:

Image looks blurry when loading at a low resolution,

Not very nice. The image is quite blurry. We are loading an image with an intrinsic size of 300x100, whereas my screen would have been capable of displaying way more pixels.

Note, I also added a few other metrics to the test page via JavaScript. What you see is the total viewport width, the logical dimension of the image and the physical dimension. “Logical” is what the browser uses to calculate the layout. “Physical” means the actual pixel density of the screen. I am currently using a MacBook with a “Retina” display which has twice as many physical pixels as the layout elements. As a result, images and fonts would be rendered “twice as sharp.”[EL3] [KA([4] This is expressed as the device pixel ratio (DPR), which is also rendered by the test page. A higher DPR is not uncommon. Most modern screens have a DPR larger than 1. Entry-level devices may start at 1.5. An iPhone 14 has a ratio of 3.0.

Conclusion: We need a higher resolution image to make the image look crisp and sharp.

Let’s just increase the dimensions of the image to 1500x500:

<div class="hero">
  <img src="https://dummyimage.com/1500x500/D3C8D9/000" id="hero" class="hero"/>
</div>

Image is sharp, but we are wasting bandwidth

See https://ackoch.github.io/image-zoo/001-hero-naive-2.html

Now the image looks super sharp. You might have recognized that in the screenshot above, I reduced the width of the window to 450px. Now, we are loading a 1500px image, even though a 900px image would have been good enough. If you remember from the introduction, the growth of the file size is over proportional to the dimensions: We are wasting a lot of bandwidth here. Especially, if we assume that more narrow screens are more likely to be found on mobile devices, where bandwidth is at a premium.

“srcset” approach

The standard solution is easy: We must provide more than one image — a set of images, a source set:

<img srcset="https://dummyimage.com/300x100/D3C8D9/000 300w,
             https://dummyimage.com/600x200/D3C8D9/000 600w,
             https://dummyimage.com/900x300/D3C8D9/000 900w,
             https://dummyimage.com/1200x400/D3C8D9/000 1200w,
             https://dummyimage.com/1500x500/D3C8D9/000 1500w"
  id="hero"/>

Here we provide a range of candidate images with different resolutions the browser can choose from. The browser cannot know what the actual width of each image is, so we must also add this bit of information to each candidate.

E.g., the expression

https://dummyimage.com/300x100/D3C8D9/000 300w
                                          ^^^^

provides a hint that the image at this URL is 300 pixels wide (300w).

Let’s do our first tests:

Open the page in Firefox (really Firefox).
Disable the cache (Dev Tools / Network / Disable Cache)
Keep the Dev Tools pane open (!)
Change the window size by dragging the window borders.

We can observe that the image being loaded conforms to the physical dimensions required. We can also see that Firefox is always loading the next largest candidate. There is a bit of wasted bandwidth, e.g., loading a 1200px image on a 1040px viewport, but that’s acceptable.

Load different assets at different screen sizes with “srcset”.

Changing Window Sizes on Firefox

Remember that I told you, that Firefox and Chrome behave differently? Here is an example.

Let’s try the same on Chrome:

Open the page in Chrome.
Disable the cache (Dev Tools / Network / Disable Cache)
Keep the Dev Tools pane open (!)
Change the window size by dragging the window borders.

When you start with a small window and gradually increase the size, chrome loads the larger images.

But when you then decrease the size again, the page “sticks” with the largest size. Only when you shift-reload the page Chrome does re-load the smaller version of the image. With the Dev Tools closed, Chrome doesn’t even re-evaluate on reloads.

Chrome does not reload when it can downsample.

See https://ackoch.github.io/image-zoo/001-hero.html

This is not a bug, but a feature. Chrome does not load a smaller version of the image, when it has a larger candidate in memory, it can down sample. That’s good for performance — but can confuse when you try to understand how the new HTML features work.

That’s why I prefer Firefox when working with responsive images. It makes testing a bit easier.

Note: If you have missed the “sizes” parameter. You are right. This is an error I made deliberately to better describe its purpose in the next chapter.

Small Teasers at ½ and ½

A pair of Small Teaser components side-by-side

Let’s apply what we have learned to the two Small Teaser components that should be displayed side-by-side at 50% each:

<div class="flex-container">
  <div class="card">

    <img srcset="https://dummyimage.com/600x600/D5E8D4/000 600w,
                 https://dummyimage.com/800x800/D5E8D4/000 800w,
                 https://dummyimage.com/1200x1200/D5E8D4/000 1200w,
                 https://dummyimage.com/1500x1500/D5E8D4/000 1500w"
         id="card1">

    <p id="dimensions-card1"></p>

</div>

  <div class="card">
    <img srcset="… same as above …" src="…" id="card2">
    <p id="dimensions-card2"></p>
  </div>

</div>

There is also a media query in the style that makes the teasers wrap on screens < 800px:

@media(max-width: 800px){
  .flex-container{
    flex-wrap: wrap;
  }
}

Open the demo page in the browser and vary the window size.

First, naive approach of the Small Teaser component.

See: https://ackoch.github.io/image-zoo/002-small-teaser-naive.html

At first glance, all looks good. But if you look carefully, you’ll see that the images loaded are not aligned to the physical width of the image. The images resolutions are aligned to the physical width of the viewport.

In the side-by-side case, the browser is loading the 1500px images even though the 800px images would have been sufficient. Even in the wrapped case it loads the 1200px images — even though the 800px candidates would have been enough.

Why is this happening?

The browser loads the images as early as possible: When it parses the <img>tag. At this point however, the document is not loaded completely, so the browser cannot fully layout the page and thus cannot know what candidate image to load. It takes the largest candidate it finds to not loose on image quality.

We could argue that it might be better for the browser to wait until the document was loaded and the page was fully rendered and then decide. But that would mean that:

a) loading of the images would start a bit later and the perceived performance could be slower.

b) there would be some re-rendering required if the image is not constraint by the surrounding <div> (like we do with width: 100%).

Loading images early with lesser information is a performance-tradeoff made by the browsers.

In fact, this behavior is according to the HTML5 standard.

I deliberately made an error here to prove a point. Let’s check on the documentation of the srcset in the MDN [6]:

When a srcset contains "w" descriptors, the browser uses those descriptors together with the sizes attribute to pick a resource.

Let me translate: “The browser does not know how wide the images are on the element. We must define that explicitly with a sizes attribute.”

Let’s do that:

<div class="card">
  <img srcset="https://dummyimage.com/600x600/D5E8D4/000 600w,
               https://dummyimage.com/800x800/D5E8D4/000 800w,
               https://dummyimage.com/1200x1200/D5E8D4/000 1200w,
               https://dummyimage.com/1500x1500/D5E8D4/000 1500w"
       sizes="40vw"
       id="card1">
       ...
</div>

The attribute sizes="40vw" means the image takes 40% of the viewport width.

I chose 40% not 50% because the images are not displayed border-to-border, but there is some space left and right and in-between. 40% is a rough estimation. If in doubt, take a larger value to not sacrifice quality. It does not have to be pixel-perfect, because the image candidates — in our example — are in rough 200px steps anyway.

Now the images conform to what is defined in the sizes attribute:

Adjusting Teaser to the fraction of available screen width

https://ackoch.github.io/image-zoo/002-small-teaser-sizes.html

At least in desktop mode. In mobile mode, this still does not look ok, even though the issue is subtle — without printing out the dimensions, this probably would have slipped:

The physical width of the rendered image is 1000px. But the browser loads the 600px candidate: It evaluates the 40vw media query, which translates to 1298px * 40% = 519px. The best candidate is therefore the 600px image.

We need to improve the sizes attribute:

<div class="card">
  <img srcset="https://dummyimage.com/600x600/D5E8D4/000 600w,
               https://dummyimage.com/800x800/D5E8D4/000 800w,
               https://dummyimage.com/1200x1200/D5E8D4/000 1200w,
               https://dummyimage.com/1500x1500/D5E8D4/000 1500w"
       sizes="(max-width: 800px) 90vw,
              (min-width: 801px) 40vw"
       id="card2">
  ...
</div>

The sizes attribute takes a list of media query / width pairs. Here we specify:

Up to 800px in the viewport the image takes 90% of the viewport width (single column plus spacing).
From 801px on 40% is sufficient (2 columns plus some spacing).

Now all cases render with proper intrinsic image dimensions:

Intrinsic dimensions now aligned with mobile view, too.

See: https://ackoch.github.io/image-zoo/002-small-teaser-sizes-responsive.html

Interlude: The CMS Point of View

Let’s take a short break here.

In the last section I explained that the browser requires some additional markup to be able to eagerly load images. From a browser’s point of view, this is a reasonable approach. From an architectural point of view, however, I find this solution sub-optimal. Especial in the context of a CMS application:

a) A component needs to be aware of the context it is rendered into to be able to render the proper media queries. If you want to re-use the same elastic “image” component on a page that covers 1/1 of the viewport width and on another page in a margin column that occupies only 1/3 of the width, you must explicitly configure it for that different context. (In AEM for example you would use component policies). This gets even more complicated, when you re-use the image component in a teaser, that uses only half of the 1/3 viewport width.

b) Media query definitions that usually are kept separated in the CSS now leak into the HTML markup. Media queries from CSS and HTML now need to be aligned — which makes re-using a shared component library on a multi-tenant platform more complex. You cannot change CSS only for a different styling. You’ll have to parameterize the rendering of the components, too.

c) “Bugs” are hard to spot. In our example, it was relatively easy to see if we are using too high or too low a resolution. But only because the images told us their intrinsic dimensions. On photographs, the issues only become apparent when the resolution is way too low. It’s even more difficult to see if we are “overprovisioning” the resolution.

Also, the browser cache adds to the confusion. Testing really is difficult and few testers really have a grasp on the concepts of responsive imaging.

Text Image

We are not finished yet. The next use case, the Text Image component has another twist.

On desktop screens, the image should be 33% wide and have a 1:1 aspect ratio. On mobile device, the image is displayed at 100% in a 3:1 format. Simple downscaling does not help.

We could constrain the height with a surrounding <div> and set the <img> to “object-fit: cover”. But that would still require to transmit the whole picture and throw away 2/3s of it. And the cropping would be a simple center-crop.

Luckily, we know in advance, there are only two fixed aspect ratios. And we know exactly when to apply which. So, we can use the <picture> tag:

<picture class="image">
  <source srcset="
      https://dummyimage.com/100x100/C0CCDE/000&text=square+500 500w,
      https://dummyimage.com/200x200/C0CCDE/000&text=square+600 600w,
      https://dummyimage.com/300x300/C0CCDE/000&text=square+700 700w,
      https://dummyimage.com/600x600/C0CCDE/000&text=square+800 800w,
      https://dummyimage.com/800x800/C0CCDE/000&text=square+900 900w,
      https://dummyimage.com/1000x1000/C0CCDE/000&text=square+1000 1000w,
      https://dummyimage.com/1200x1200/C0CCDE/000&text=square+1200 1200w,
      https://dummyimage.com/1500x1500/C0CCDE/000&text=square+1500 1500w"
    sizes="30vw"
    media="(min-width: 801px)">

  <source srcset="
      https://dummyimage.com/500x166/C0CCDE/000&text=panorama+700 700w,
      https://dummyimage.com/600x200/C0CCDE/000&text=panorama+800 800w,
      https://dummyimage.com/900x300/C0CCDE/000&text=panorama+900 900w,
      https://dummyimage.com/900x300/C0CCDE/000&text=panorama+1000 1000w,
      https://dummyimage.com/900x300/C0CCDE/000&text=panorama+1200 1200w,
      https://dummyimage.com/1500x500/C0CCDE/000&text=panorama+1500 1500w"
   sizes="85vw"
   media="(max-width: 800px)">

  <img src="https://dummyimage.com/900x300/C0CCDE/000&text=panorama+900" 
       id="card3" >

</picture>

The picture tag in this example includes two <source> tags embedded, one for each breakpoint. As in the example before, we list the candidates and provide a hint with sizes, how wide the image will be: ~30% in desktop mode above 800px and ~85% on mobile devices below. The media query is not required in the sizes attribute, as we already have an explicit media query in the mediaattribute. This attribute is used to select either source.

To emulate, different aspect ratios I added the texts “square” and “panorama” to the images. Let’s try it out:

https://ackoch.github.io/image-zoo/003-text-image-art-direction.html

Works like charm… until we drag the window to cover the full screen:

Image width at 30% of the screen not 30% of the content area.

Oops. We configured the image to be 30vw. 30% of the viewport width. But we only need 30% of the main content area. At lower resolutions, the content area and the viewport are equally wide, so there was no problem. Only above 1024px the content area is fixed and does not get any wider. We are unnecessarily increasing the required image’s width by using the wrong reference point: We would need a 1024px * 30% ~ 300px image. But at a 1500px wide viewport we request a 1500px * 30% ~ 500px image (or ~600px vs. ~1000px if you have a 2x retina display, see below).

The problem is that the max-width max-width constraint of the main content area is not reflected in the media query of the source element. This can be fixed by adding yet another source with another breakpoint above 1024px:

<source srcset="https://dummyimage.com/500x500/C0CCDE/000&text=square+600"
        media="(min-width: 1024px)">

See: https://ackoch.github.io/image-zoo/003-text-image-art-direction.html

Image adapting properly to 30% of content area.

This works well on my machine — and probably on yours, too. But remember, some screens can display twice as many pixels in images than the logical resolution of the browser. In the example above, I am assuming a 2x device pixel ratio (1024 x 33% * 2 ≈ 675).

When we want to differentiate different device pixel ratios for fixed-size images, we use a slightly different syntax in the srcset definition:

<source srcset="https://dummyimage.com/300x300/C0CCDE/000&text=square+300 1x,
                https://dummyimage.com/600x600/C0CCDE/000&text=square+600 2x"
        media="(min-width: 1024px)">

Here we explicitly use the 300px image for the low-res screens and the 600px variation for the hi-res screens.

The result can be seen here: https://ackoch.github.io/image-zoo/003-text-image-art-direction-capped-dpr.html

Interlude: How to get parameters

Using <picture> and srcsetrequires a lot of parameters to be defined. How do we get these parameters? It depends. If you have a very(!) well-designed style guide, you might be able to calculate the media queries from there. But I found it more practical to do some introspection on the implemented pages and apply some common sense. We are usually providing image candidates in 100px or 200px steps. So that’s not exact science anyway. If you know that the image takes up about 1/3 of the viewport on a particular breakpoint. And there is some space left, right and in between the elements, you can safely define sizes=”30vw”. With some experimentation, maybe you can bring it down to 28%.

Make sure you know the device pixel ratio of the monitor you are testing on. I.e., if you measure a logical dimension of between 300px and 800px, make sure to provide candidates from 300px for 1x monitors and narrow viewports and a 1600px version for wide screen 2x monitors.

Large Teaser

Image height aligns to height of the text block on desktop screens.

At first glance the Large Teaser looks like the text and image component. However, here we want to always align the height of the image with the height of the text box.

The image uses 33% of the screen width on desktops. On mobile screens the image is stacked and uses 100% of the screen width and is displayed as 3:1. In contrast to the Text Image component, the height of the image is variable. It is defined by the height of the text block: Longer copy texts lead to a more stretched format:

The more text we add, the longer higher the image needs to be.

The image can have any aspect ratio and does not conform to any of the pre-defined formats. We can see that it’s getting dynamically narrower the longer the text gets. Remember, that Dynamic Media’s Smart Cropping requires an exact and pre-defined aspect ratio.

There are two ways to solve that:

Dynamic Cropping without Dynamic Media

Without Dynamic Media you build a wrapper <div> around the <img> to constrain the size of the image. The <img>tag then can be styled with

height: 100%;
width: 100%;
object-fit: cover;
position: absolute;

By default, the image is cropped evenly on the opposing sides. This can be influenced with the object-position parameter:

object-position: 50% 50%;

is a centered crop.

The parameter

object-position: 0% 100%;

places the image at 0% from the left and 100% from the top (i.e. at the bottom), which basically means it crops at the top right. In other words, it assumes the focus is on the south-west quadrant.

This still transmits the full image, but the edges are cropped in the browser. So, we are wasting some bandwidth here.

The object position would have to be defined for each image individually. It does not make sense to define it in the global style sheet. Each image requires different cropping parameters.

In a CMS, you would typically ask the authors to provide these parameters. I.e. with a 3x3 matrix as proposed in Part I. Or you advise the authors to only use images that can safely be cropped on all edges.

Image on the right is better suited for cropping. (Sources: CSS Studio — Adobe Stock; Юрий Маслов — Adobe Stock)

The picture on the left has nothing in the center. In default mode, the browser would crop away the interesting parts of the picture. The image on the right has a lot of background at the edges that can safely be removed.

Dynamic Cropping with Dynamic Media

Even though Dynamic Media cannot apply Smart Cropping on-the-fly, it still brings a few features to improve the authoring experience and the performance:

We can use Smart Pre-cropping to define areas that can safely be center cropped.
We can apply Dynamic Cropping to crop on the server-side. This yields the same visual results, that the (weighted) center cropping in the browser would. Only it is performed on the server. Thus, the transferred binary is smaller, and the page renders faster.

Smart Pre-cropping is not a feature that you’d find in the product description. It is a simple practice you can apply with normal Smart Cropping.

Direct where to dynamically crop by placing the focus via smart cropping, first.

The idea is to create a Smart Crop from the original image first and deliver this to the browser to clip away the rest. By using Smart Cropping (and maybe some manual corrections) first, we can easily find areas that have enough edge to clip.

Dynamic Cropping

Dynamic Media has yet another ace up its sleeve. We can crop server side with URLs like so:

https://techsupporteu.scene7.com/is/image/AEMEMEAPractice/AdobeStock_636568731?fit=crop&wid=300&hei=100

The parameter fit=crop tells Dynamic Media to fit the image into the rectangle defined by width and height and crop away what’s not needed.

If we are using HTML5 only to select from the candidates, this is not particularly useful. We can’t define a static URL — or a range of URLs — that would describe this dynamic cropping we require for the dynamic cropping case. See Part IV of this series for how to fully leverage Dynamic Media’s dynamic cropping capabilities when we are implementing a custom image loader.

Creating the AEM Core Component

We now have pieced all major use cases together.

Why should we bother anyway? Can’t we just use the AEM Core Components?

Unfortunately, not. They only provide limited support for Dynamic Media. Also, the support varies from component to component. A few examples of what is supported:

The Image Component supports Smart Cropping. But it does not allow proportional scaling. It will use the original resolution on all viewport sizes. It does not support art direction or browser cropping. But that’s usually not required for such a component.

The Image List crops in the browser — though this would be an ideal use case for smart cropping. At least it scales down the images.

The Teaser crops in the browser, only. This would have been a good candidate for Smart Pre-cropping, though. It does not support Art Direction for the images, though the formats for desktop and mobile are quite different. Again, only default cropping is applied, wasting bandwidth, and not producing optimal results.

The scaling steps for all components are defined in the policies for the Image Core Component. As this configuration needs to be applied for all components, it creates a very long list of candidate URLs in the srcset. By default, 13 candidates from 100px to 1600px. I would have rather configured an individual srcset for each component. That’s a bit of an esthetic question. The overall HTML increases a bit — but not too much.

I am not the first to realize that the support could be better. In the wcm.io [7] project is an advanced media handler that promises better support for Dynamic Media. I haven’t tried it out, though.