Why Apple Uses JPEG XL in the iPhone 16 and What It Means for Your Photos

Was just about to see if someone mentioned this video. Really good explanation comparing JPEG XL vs AVIF.

This may be a naive question but can't we have the features of JPEG XL combined with the compression algorithm of AVIF? Why does it have to be one or the other? Size clearly matters, especially for the majority of the world's population who do not have access to super fast internet and TBs of storage. It's definitely a luxury not having to care about file sizes.

You wouldn't want 24MB files with GPS and face detection metadata served raw on the Web anyway.

Of course it’s .dng; .jxl doesn’t carry the metadata needed to process a RAW image because it’s not intended for that.

This is how they sell new phones. The grand majority of new features don't require the latest hardware, but artificially restricting it increases sales of new phones. None technical people can't usually tell the difference between hardware and software clearly enough to understand this nasty trick.

The "CPU" (that is, the generic compute part) cannot encode 4K video in real time, that part is handle by codec-specific HW.

I’d guess hardware acceleration could have something to do with it

It does seem odd to restrict new features to the latest models, especially when older ones still have powerful capabilities.

This makes me think: If JPEG-XL needs more computing power to decompress, does that out weigh the ecological benefit benefit of the smaller file size?

One of the tricks for achieving the target battery life is that photo and video formats are offloaded to dedicated and very power-efficient hardware in all mobile devices. The iPhone 16 is the first to get hardware offload for AV1 and JXL, which is why it'll support these formats.

It's not just software, unlike in the PC world where going from 5W hardware decode to 50W software decode basically doesn't matter.

This, I just don't understand why it seems only latest iPhones got this.

The CPU isn’t used for encoding video

Attaching any significance to being “first” on open standards is a game Apple rarely plays, but which others impose upon them because Apple’s adoption is (rightly or wrongly) seen as the most consequential and/or most newsworthy inflection point.

An annoying oversight is that while my Fujifilm camera is modern enough to shoot HEIF+RAW, Apple Photos only knows to group JPEG+RAW as a single photo. Because Apple did not spend a day of engineering time bringing feature parity for the file format they themselves promoted, it has turned into a bigger feature to match and merge the HEIF and RAW assets after the fact. After several years, I'm growing doubtful they'll ever accomplish it.

I have yet to see whether they did it right with JXL+RAW (or is it DNG+RAW?) but hopefully they will before it becomes available in mainstream cameras.

HEIC is a standard too - it wasn’t a secret internal Apple project…

Thank god they went with a standard because when they went with a standard it wasn’t widespread?

What?

Although it might seem confusing at first glance, having your selling point as "our file sizes are larger!" is so counterintuitive, that I think it's obviously not that!

Bigger is better, it is known.

Not just the original file quality, it's exactly the same image but (losslessly) compressed better.

It would be safe to assume that Apple will eventually add a way to recompress your photo library to JXL… if they weren’t in the business of selling storage and cloud storage. They have in the past released tools to optimize storage so it wouldn’t be completely out of the ordinary, but… I wouldn’t hold my breath.

> but so far I haven’t seen any tooling (in macOS) to re-encode my existing photo library

On Linux you can re-encode every single .jpg to .jxl and they'll decode bit-for-bit, to the original .jpg.

On Debian and derivatives it's the libjxl-tools: the cjxl executable converts to .jxl and the djxl decompresses.

Works flawlessly.

P.S: as a sidenote there's zero reason anymore to serve a .jpg file to a browser when the browser supports .jxl files. That's just a waste of bandwith. (and if your stack cannot serve different files depending on the users' browsers' capacities, it's not much of a stack)

Firefox Nightly is listed as supported on the jpegxl.info web site. My understanding is that it does not fully support the specification yet which is why it is not released to production.

https://jpegxl.info/resources/supported-software.html

Weirdly enough, JPEG-XL support is actually fairly decent in Firefox already and there were patches developed by the community that work well for things such as color profiles, animation support etc. I’ve had them in Waterfox for a few years now - it was a purely “political” decision, if you want to call it that, to stop any more progress until recently.

Man this back and forth is really frustrating.

So Google killed XL a while ago already and I feel like either Microsoft or Mozilla at least considered following suit. After Apple has done heic for a while now I assumed it might go that way regarding a jpeg replacement, but now they did a 180 and switched to xl. I mean good, it's not as patent encumbered, but wtf am I supposed to expect for the future now? Will Google add XL back to chrome? I guess it will take another decade or five until we have a jpeg replacement that's being universally agreed upon, because come on, it would be too easy if we don't get another plot twist where a major player jumps onto something else again for a while.

DNG spec added JPEG-XL as their modern codec. Both lossy and lossless modes are significantly better than the other 3 allowed codecs (JPEG, lossless JPEG, and zip.)

ProRAW files are large, so there's more potential to save space, making ProRAW a more attractive feature to use for a wider audience.

Animated AVIF is widely supported, and can represent GIFs losslessly.

BTW, Chrome vetoed the idea of supporting muted video files in `<img>` like Safari does, so we've got this silly animated AVIF that is a video format that has been turned into a still image format that has been turned back into a video format, which takes regular AV1 video data but packages it in a slightly different way, enough to break video tooling.

You may be waiting a while. APNG and animated WebP are already widely rolled out, but haven't replaced GIF.

Indeed, but Chrome removed support for the format some time ago.

They already adopted it 7 years ago so no.

Most iPhone users would be storing photos in iCloud.

And most professionals are storing their photos in Creative Cloud.

And in both cases the photo data would be replicated on multiple hard drives for redundancy. So it would definitely be more environmentally friendly to have smaller photos.

The more important reason is that cloud storage costs significantly more than local.

If people are going to take credit for crunching something down from 10 MB to 8 MB, I wish they'd also take the blame for massively bloating something into a modern monstrosity out of sheer laziness or betting that profit margins will be higher if they waste user time, storage, and cpu for the sake of releasing the project faster.

> Are they storing all of their images in RAM?

Consider the millions of iPhone users uploading new photos and scrolling their iCloud-hosted photo library. A substantial amount of photo data is in RAM at any given moment, spread across many network devices.

> I would posit that it is possible this format is less environmentally friendly because it takes more compute cycles

I'm curious how much energy it takes to send bits over cellular or wifi, my intuition is that this is orders of magnitude higher than compression or encoding.

> The hard drive on my shelf isn't using any energy

But it is losing its data: <https://news.ycombinator.com/item?id=41244143>

On the other hand, we have industry defined standards to compress TCP headers [1] - incurring a cost to compress/decompress but resulting in lower packet sizes.

[1] https://en.wikipedia.org/wiki/Robust_Header_Compression

What is environmentally (OK, mentally) unfriendly to me is the creation of new formats that various apps, and notably, websites where I post images, can't handle, so I swear a bit, convert the image to jpeg or png, and post.

People are part of the environment, too!

But lo! men have become the tools of their tools. ... Thoreau.

> Are they storing all of their images in RAM?

That part would not matter regardless. RAM is powered at all times, kept being refreshed too. Unless, of course you mean, they'd have to keep buying new machines to store more pictures.

> The hard drive on my shelf isn't using any energy.

When it filled up you put it on a shelf and bought a new one.

This whataboutism we are hearing now about "the energy needed to store your pictures" is just corporate whataboutism, trying to push blame onto consumers.

Let's say you have 2 TB of photos stored. If that's consuming more than 2 watts of electricity on average, the cloud providers have to be quite incompetent (they are mostly not).

2 watts is 1 kWh in 20 days, so 18 kWh in one year. The emissions from 18 kWh in the US is around 6 kg of CO2. This is the equivalent of driving a car for about 20 minutes on the highway, one time per year. Or spending 5 minutes longer in the shower 4 times per year.