Under-display cameras explained: A look beneath the Fold 3's unique screen

We all want great selfies without a hole punch or notch. But there's a delicate balance between display performance and photo quality.

For the past several years, the smartphone industry has been relentlessly marching towards one ideal: An all-screen phone front, uninterrupted by bezels, notches, hole punches, or other blemishes. High screen-to-body ratios have been a hallmark feature of many of the best Android phones for some time now, but the dream is to eliminate anything that's not screen on the front of the handset.

There are trade-offs with each of the technical solutions to this problem. Phones like the ASUS Zenfone 8 Flip eliminate the need for a dedicated selfie camera by installing the rear shooters in a special rotating module — though this results in a fairly bulky handset and eliminates any possible water resistance in the device.

Under-display cameras — or UDCs — are perhaps the most ambitious solution, however. We've seen these in a handful of prototype handsets from various manufacturers, as well as retail products like the ZTE Axon 30 5G. And with the recent launch of the Samsung Galaxy Z Fold 3, under-display cameras are starting to become more mainstream.

This approach puts a camera below the tiny glowing dots that make up the pixels of a smartphone's display. It's similar to a hole-punch camera in that it cuts away the panel's digitizer and backplane to allow a camera to peek through, but this time manages to extend the actual OLED matrix over the top of that hole. Get it right, and the result should be a much less obtrusive gap in the display than a traditional hole-punch selfie shooter.

Pixels need to be big enough to maintain the flow of the display but small enough not to block the camera.

That in itself presents many other compromises. Though to the naked eye they might appear invisible, the individual red, green and blue subpixels that make up each pixel on the phone's display aren't completely transparent, even when the display is off. That means manufacturers need to change the layout or size of the pixels over the display hole to allow light to reach the camera — and therein lies the major trade-off of under-display cameras. If the pixels over the hole look too different — whether they're smaller and spaced further apart, or simply fewer in number — the camera area will be more noticeable. But if they're similar to regular pixels, chances are they'll block too much light or obscure the camera lens.

Different manufacturers have different approaches to under-display cameras. First, let's look at Samsung's, in its latest foldable phone.

Most of the Galaxy Z Fold 3's display uses round subpixels in a traditional diamond matrix pattern — with a cluster of red, green, blue, green subpixels (or R-G-B-G), each representing two pixels. Over the camera hole, the Fold 3 appears to cluster two pixels together in an unusual B-R-G-G-G-G-R-B pattern.

You can see the contrast in pixel density over the Fold 3's camera hole in this macro photo — the density over the hole appears to be around half that of the regular display.

In this shot taken with the display off, you can see why this extra spacing between clusters of pixels is necessary. If the regular display extended over the camera hole, it would be completely obscured by tiny red, green and blue subpixels, which are still there blocking light even with the display completely powered off.

This arrangement of pixels means the resolution is slightly lower over the Fold 3's camera hole, which we could demonstrate by moving an image of a single-pixel-wide white line across the camera hole.

In the first image above, you'll see an entire column of pixels is illuminated through the main display and the hole area. In the second image, we've moved the line one pixel to the left, and it no longer shows over the camera hole, presumably because there are no pixels at the corresponding coordinates due to the space between the clusters of pixels. The effect is like a virtual wire mesh covering that area of the display, reducing the resolution by around half.

This is an extreme example, though. Most of the time, when you're viewing text, photos, or videos, the reduced density over the camera won't be noticeable. While the difference between the two parts of the display is plain to see through a macro lens, in our testing of the Z Fold 3, we've found the difference is much less obvious to the naked eye. Even when viewing things like text up close, your brain tends to fill in the gaps and compensate for the lower resolution in that one area.

Oppo plans on taking a different approach with its new under-display camera, due to arrive in phones sometime next year. While Oppo will use the same subpixel matrix pattern over the camera hole, images shared by the manufacturer show smaller pixels than what Samsung used. Xiaomi's recently announced Mi Mix 4 uses a similar method, shrinking the size of the pixels displayed over the camera.

The difference in image quality between Samsung, Oppo, and Xiaomi phones likely won't be very noticeable to the naked eye. But the interplay between camera sensor size, camera pixel size, and pixel layout on the display is likely to determine where each device falls in terms of the balance between photo quality and display quality.

Whatever the approach to creating a UDC, you'll need to balance photo quality with display clarity.

Whichever approach you take to an under-display camera, though, the pixels over the camera hole will always look at least a bit different from the rest of the display. And the photos from that selfie camera are always going to be at least a bit weaker than they would be from a regular front-facer behind a hole-punch.

There are ways of mitigating the impact on photo quality, of course. Samsung uses larger pixels in the Fold 3's 4-megapixel front-facer — and since it's a foldable with other camera systems that you can use for selfies, it's not such a big deal that the internal selfie camera takes slightly weaker photos.

On the Oppo side, the Chinese firm is talking up its AI image enhancement features, designed to compensate for the light being blocked by the LEDs of the screen. It could use similar processing techniques to counteract those subpixels becoming illuminated by a bright light source while taking a photo.

It's still very early days for under-display camera technologies. If the ultimate dream of a camera that truly disappears into the display is to be realized, it'll be through incremental improvements to OLED and image processing technologies. There's a long way to go, for sure. Still, as with any technology in its infancy, it'd be foolish to write off under-display cameras based on the imperfections of early implementations.