Skip to main content

Responsive Images 201: Client Hints

By Jason Grigsby

Published on September 21st, 2016

Topics

Last year, I wrote a ten-part series called Responsive Images 101 that introduced the new responsive images syntax. Today, I want to continue that series by talking about Client Hints.

As the title indicates, this is an advanced responsive images topic. However, despite being potentially complex to implement, Client Hints can make the day-to-day task of creating responsive images much easier for designers and developers.

Client Hints define a way for browsers to tell the server what the types of content that the browser prefers. It provides this information via HTTP request header fields.

Every request that a browser makes to a server contains HTTP request headers. For example, the headers that Firefox sends to request our home page look like this:

GET / HTTP/1.1
Host: cloudfour.com
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.11; rv:47.0) Gecko/20100101 Firefox/47.0
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
Accept-Language: en-US,en;q=0.5
Accept-Encoding: gzip, deflate
Cookie: _ga=GA1.2.333160365.1465672418
Connection: keep-alive
Code language: HTTP (http)

You can see HTTP request headers in most developer tools in the network pane.

Chrome developer tools showing the HTTP request headers
Chrome developer tools showing the HTTP request headers for Cloudfour.com

Client Hints add additional HTTP request header fields containing information about the the browser. One of the main uses of these additional headers is to send information about the size of images required for the page.

There are three Client Hints that are relevant for responsive images:

  • DPR — Device pixel ratio (e.g., 1x, 2x, etc.)
  • Viewport-Width — The size of the viewport.
  • Width — The width of the image in the page.

Of these three, I find DPR and Width the most useful. As I explained in the Responsive Images 101 series, viewport size can be a poor substitute for the actual size of an image.

Viewport tells us little about the size of an image in the page

What could we do if the browser provided width and DPR information to the server and the server knew what to do with it?

Let’s take a look at a code sample from the Sizes section of the 101 series:

<img src="cat.jpg" alt="cat"
  srcset="cat-160.jpg 160w,
          cat-320.jpg 320w,
          cat-640.jpg 640w,
          cat-1280.jpg 1280w"
  sizes="(max-width: 480px) 100vw,
         (max-width: 900px) 33vw,
         254px">
Code language: HTML, XML (xml)

The reason for the addition of the srcset attribute in the markup is to provide the browser with different image sizes that the browser can request. However, if the server knew the size of the image in the page, it could simply provide an appropriately-sized image to the browser.

That’s what the Width Client Hint provides. It adds the width of the image, calculated using the information in the sizes attribute, to the HTTP headers that are sent to the server.Footnote 1

GET cat.jpg
Accept: image/webp,image/*,*/*;q=0.8
DPR: 2
Viewport-Width: 1024
Width: 508
Code language: HTTP (http)

Therefore, if the browser and the server both support Client Hints, the markup can be simplified to be:

<img src="cat.jpg" alt="cat"
     sizes="(max-width: 480px) 100vw,
       (max-width: 900px) 33vw,
       254px">
Code language: HTML, XML (xml)

Making this change takes all of the complexity of picking responsive image breakpoints out of the hands of designers and developers and puts the onus on the server.

We’ve already made some big improvements in our markup by using Client Hints, but there is more that can be done if we couple Client Hints with another HTTP request header: Accept.

The Accept header is used by the browser to tell the server what types of media the browser considers acceptable for the current request. For example, when Chrome requests an image, it adds the following Accept header:

Accept: image/webp,image/*,*/*;q=0.8
Code language: HTTP (http)

This tells the server that Chrome prefers WebP image formats, but if WebP isn’t available, then send a file with a mime type that starts with image (image/*). If you don’t have that, then send whatever you’ve got (*/*).

The last little bit, q=0.8, tells the server how much the browser prefers the specified format. You can find more on how the quality preference parameter works in the specification.

Adding the Accept header allows us to simplify responsive images markup used to support multiple image formats. Here is the “before” code:Footnote 2

<picture>
  <!-- serve WebP to Chrome and Opera -->
  <source
    media="(min-width: 50em)"
    sizes="50vw"
    srcset="/image/thing-200.webp 200w, /image/thing-400.webp 400w,
        /image/thing-800.webp 800w, /image/thing-1200.webp 1200w,
        /image/thing-1600.webp 1600w, /image/thing-2000.webp 2000w"
    type="image/webp">
  <source
    sizes="(min-width: 30em) 100vw"
    srcset="/image/thing-crop-200.webp 200w, /image/thing-crop-400.webp 400w,
        /image/thing-crop-800.webp 800w, /image/thing-crop-1200.webp 1200w,
        /image/thing-crop-1600.webp 1600w, /image/thing-crop-2000.webp 2000w"
    type="image/webp">
  <!-- serve JPEGXR to Edge -->
  <source
    media="(min-width: 50em)"
    sizes="50vw"
    srcset="/image/thing-200.jpgxr 200w, /image/thing-400.jpgxr 400w,
        /image/thing-800.jpgxr 800w, /image/thing-1200.jpgxr 1200w,
        /image/thing-1600.jpgxr 1600w, /image/thing-2000.jpgxr 2000w"
    type="image/vnd.ms-photo">
  <source
    sizes="(min-width: 30em) 100vw"
    srcset="/image/thing-crop-200.jpgxr 200w, /image/thing-crop-400.jpgxr 400w,
        /image/thing-crop-800.jpgxr 800w, /image/thing-crop-1200.jpgxr 1200w,
        /image/thing-crop-1600.jpgxr 1600w, /image/thing-crop-2000.jpgxr 2000w"
    type="image/vnd.ms-photo">
  <!-- serve JPEG to others -->
  <source
    media="(min-width: 50em)"
    sizes="50vw"
    srcset="/image/thing-200.jpg 200w, /image/thing-400.jpg 400w,
        /image/thing-800.jpg 800w, /image/thing-1200.jpg 1200w,
        /image/thing-1600.jpg 1600w, /image/thing-2000.jpg 2000w">
  <source
    sizes="(min-width: 30em) 100vw"
    srcset="/image/thing-crop-200.jpg 200w, /image/thing-crop-400.jpg 400w,
        /image/thing-crop-800.jpg 800w, /image/thing-crop-1200.jpg 1200w,
        /image/thing-crop-1600.jpg 1600w, /image/thing-crop-2000.jpg 2000w">
  <!-- fallback for browsers that don't support picture -->
  <img src="/image/thing.jpg" width="50%">
</picture>
Code language: HTML, XML (xml)

There’s a lot going on in that example—multiple image formats, numerous image sources, and art direction. It is a bit of an extreme example.

With Client Hints and the Accept header combined, it can be simplified to:

<picture>
  <source media="(min-width: 50em)"
          sizes="50vw"
          srcset="/image/thing">
  <img sizes="100vw" src="/image/thing-crop">
</picture>
Code language: HTML, XML (xml)

Amazing, right?

Notice that the paths for the images in the simplified markup do not have a file extension. That’s because the Accept header will tell the server what image formats the browser accepts and what preference it has. It is then up to the server to pick the best image format to send back.

Some of you may be wondering why you’ve spent so much time learning srcset, picture and the rest of the responsive images syntax if Clients Hints makes it so simple.Footnote 3 But there are reasons why Client Hints are a Responsive Images 201 topic.

At the moment, only Chrome and Opera support Client Hints. They are under consideration by the Microsoft Edge and Firefox teams.

HTML can be used without a server. Perhaps you’re building an eBook or something that works primarily offline. In those scenarios, you may need responsive images markup.

The server needs to know what to do with the Client Hints HTTP request headers. We may eventually see servers like Apache and NGINX add support for Client Hints, but it will take some time because it will require them to not only support the HTTP headers, but also add image processing capabilities.

In the short run, the way to utilize Client Hints is by signing up for an image resizing service that supports the specification. I know that Cloudinary, imgix and ScientiaMobile all support Client Hints.

There may be additional services that support Client Hints that I’m unaware of, so I’ve added a column for Client Hints to the spreadsheet of image resizing services so we can track support. If you know of any services that should be updated, please let me know.

Like many image-related topics, supporting Client Hints on the server sounds easy at first. The browser tells the server the size of the image in the page. The server returns the correctly-sized image.

But doing that would cause all sorts of problems as Eric Portis pointed out on the Cloudinary blog:

To deliver perfectly-scaled resources, our servers must render and store hundreds, maybe thousands of alternate versions of your image, which is both computationally expensive and terrible for your pages’ performance. More, alternate resources means fewer cache hits, and a hit at any level — in the local browser cache, on the CDN, or server-side — will always be significantly faster than a miss.

You probably don’t want small adjustments in the size of the viewport, or a rotation event, to cause the image that has been downloaded to no longer look good. This is one of the side benefits of downloading larger responsive images than needed—resizing the browser doesn’t necessarily mean the image will look bad.

You will also want images to get cached. And you definitely don’t want to pay exorbitant amounts to cache dozens of image sources on your CDN.

This is where the various image resizing services can set themselves apart. If we give them Client Hints, how smart are they about providing the best possible image size and format?

Because browser makers don’t want to add excess HTTP headers to every request unnecessarily, you have to explicitly opt in to Client Hints by adding a <meta> tag:

<meta http-equiv="Accept-CH" content="DPR, Viewport-Width, Width">
Code language: HTML, XML (xml)

You should only list the Client Hints that you’re going to use in the comma-separated list to avoid excess HTTP headers.

Opting in via the <meta> tag presents a conundrum for your markup. When the browser requests the HTML document from the server, the browser won’t supply any HTTP header information that indicates to the server that the browser supports Client Hints because the browser hasn’t yet seen the <meta> tag declaring browser support.

The server can add the <meta> tag into the HTML document that it returns to the browser, but what does it do with the markup for responsive images within that same HTML document? The server doesn’t know yet if the browser will support the <meta> tag request for Client Hints.

I asked Ilya Grigorik, the Google engineer who has led the charge for Client Hints, if browsers don’t declare their support before the HTML is delivered, do you have any suggestions on how to support browsers that support client hints and those that don’t?

Ilya replied:

In the meantime, I guess you can run a simple UA check (Chrome 46+) if you want to customize the HTML markup — yes, I recognize this is a bit ironic given that CH is all about removing the need for sniffing the UA string :-)

Before you throw your keyboard out the window, there are good reasons why Client Hints work this way.

Using HTTP request headers to define what content the server should send back is commonly referred to as Content Negotiation. Browser vendors have a long-standing aversion to Content Negotiation and for good reasons.

The opt-in nature of Client Hints is one of the ways that Ilya has tried to address these concerns and get buy-in from browser vendors. Instead of adding HTTP headers to every request whether or not the server requires it, the opt-in limits it to servers that are ready to use the information.

If the server doesn’t know if Client Hints are supported before generating the markup, how do we know what markup to generate. Assuming we want to support Client Hints, there are two possible solutions:

  • Server-side user agent detection
  • Supply both Client Hints and responsive images markup

I confirmed that if you supply both the normal responsive images markup and the <meta> tag for Client Hints, that the browser will evaluate the responsive images markup and request the appropriate image source using the Client Hints HTTP headers.

So the simplest solution in the short run may be to use both Client Hints and the responsive images markup.

In reality, whatever implementation makes the most sense is likely going to be dictated by whatever image resizing service you’re using. I’d recommend talking to your service provider about what they provide.

A woman in a superhero t-shirt
Photo by Ashley Rose

Client Hints provide a glimmer of a promising future for responsive images. When Client Hints have broad support, the only responsive images syntax you need will be sizes.Footnote 4 This is why I consider sizes to be the superhero of responsive images.

Client Hints also remind us that humans shouldn’t be in the business of processing images or picking image breakpoints. As I stated at the end of my Responsive Images 101 series, “The goal for most organizations should be to centralize image resizing and processing and automate as much of their responsive images as possible.”

Footnotes

  1. The first version of this article listed the value of the width header at 254 which is the size of the image in the page in CSS pixels. Johannes Koch pointed out in a comment that the spec says that it should send the physical width of the image, not the CSS pixel width. I confirmed that Johannes is correct and that the spec changed from CSS pixels to the intrinsic width of the image.  Return to the text before footnote 1
  2. Code examples borrowed from Ilya Grigorik’s great introduction to Client Hints.  Return to the text before footnote 2
  3. A better question might be why I spent days writing a 10-part series on the topic.  Return to the text before footnote 3
  4. Unless you’re doing art direction which still requires <picture> because of the manual editing.  Return to the text before footnote 4

Comments

Johannes Koch said:

Hi Jason,

thanks for you article. Please note, that according to http://httpwg.org/http-extensions/client-hints.html#width the Width cient hint is in physical pixels, not in CSS pixels (unlike Viewport-Width!). So the HTTP request example in ‘Client Hints can mean much simpler markup’ should include

Width: 508

instead of

Width: 254

if the DPR is 2.

Nicolas Hoizey said:

Awesome post, once again!

I always thought using content negotiation was really nice. I used it in the 90’s, mostly for Accept-Language, with great success. But I understand from the WHATWG page you link to that it’s not recommended anymore.

Using Cloudinary on my Jekyll personal site, I have to check how to use Client Hints for images, but how do you “use both Client Hints and the responsive images markup”? Duplicate the HTML code and hide the one that’s not used?

Replies to Nicolas Hoizey

Jason Grigsby (Article Author ) replied:

I’m not sure how Cloudinary will handle it, but say you had markup like this on a web page that also included the Accept-CH meta tag :

<img src="cat.jpg" alt="cat"
  srcset="cat-160.jpg 160w,
          cat-320.jpg 320w,
          cat-640.jpg 640w,
          cat-1280.jpg 1280w"
  sizes="(max-width: 480px) 100vw,
         (max-width: 900px) 33vw,
         254px">
Code language: HTML, XML (xml)

And the viewport was 1024px wide on a 1x screen. The browser would decide that the size of the image in the page was 254px (via sizes) and would most likely decide to download cat-320.jpg (next largest image available).

When the browser sent of the request for cat-320.jpg, the browser would include the Client Hint request headers that were specified in the Accept-CH meta tag. At that point, Cloudinary (or whatever the service was), could choice to sub out the requested image with an alternative one—maybe one sized closer to 254px or maybe the image is smaller in a different image format.

Whether or not any of the existing services handle it that way is unknown to me. All I was commenting on is the browser behavior if the meta tag and the responsive images markup are both present.

Nicolas Hoizey replied:

(not really sure if using the “reply” on my own comment will work, but there is none on your reply…)

Ok, thanks for the explanation. I thought you planned to put both syntaxes from your article. It makes more sense now.

Eric Portis replied:

I’m not sure how Cloudinary will handle it

I am! ? You can do what Jason describes by using the third w_auto parameter, which specifies a fallback Width for Cloudinary to use if it didn’t receive one from the browser.

Here’s an example, building on Jason’s, above.

Note that, IMO, there isn’t really any reason to do this. Once you’ve written out a srcset and figured out how to generate those resources, your image is already responsive, and you’ve already incurred the complexity-cost… what does adding Client Hints on top of that, do?

The main benefit of Client Hints over srcset that it makes responsive images a lot easier, and your markup a lot cleaner, by encapsulating all of the multiple-resource and breakpoints complexity behind a single URL, eliminating srcset. If you’ve already gone through the trouble of writing (or hopefully, automating the generation of!) a srcset, Client Hints don’t bring much to the table.

I would recommend Width/DPR Client Hints to anybody struggling with srcset/breakpoint complexity now… but not to anybody who’s already come to terms with it. In a year or three, if/when browser support has improved, I’d recommend Client Hints to everybody (with a server that can handle ’em).

Morten Pedersen said:

Interesting post Jason, looks very promising.

One of the things I like about the srcset specification though, is that it’s open to interpretation by browser vendors to include network stability and other metrics to determined the most relevant source. This is something we as developers can’t otherwise access in the critical loading step.

In practice I’ve experienced differences in preferred source between browsers and connectivity, but how much they plan to put in to this I don’t know.

Perhaps in time some of these metrics could be added to Client Hints as well.