As tools based on machine learning and AI have appeared, most recently Adobe’s Generative Fill feature in Photoshop, photographers seem to bounce between embracing the technology as a new creative tool and rejecting the intrusion of “AI” into a pursuit that values image authenticity and real-world experience.
But while generative AI has stolen all the attention lately, machine learning has long maintained a foothold in the photography field. Here are five areas where you’re probably already benefiting from machine learning, even if you’re not aware of it.
Camera autofocus
 |
| Mirrorless cameras benefit greatly from machine learning and are trained to recognize everything from people to airplanes. |
Your camera generally surveys the scene in an uncomprehending manner, either hunting for contrast or analyzing different perspectives on the scene to assess how misaligned they are. It drives the lens until this contrast is maximized or the two perspectives come into alignment. Depending on the AF mode, the processor is often evaluating one or more focus points or relying on your assistance to specify a focus area.
However, the autofocus system knows nothing about the scene. Increasingly, there’s a separate process going on in parallel. Based on machine learning models, the processor is also trying to interpret the scene in front of the camera and identify subjects (such as faces or objects) in the scene.
We’ve seen this for a while on cameras that feature face and eye detection for focusing on people. Even when the focus target is positioned elsewhere in the frame, if the camera recognizes a face – and by extension, eyes – the focus point locks there. As machine learning algorithms have improved and processors have become faster at evaluating the images relayed from the sensor, newer cameras can now identify other specific items such as birds, animals, automobiles, planes, and even camera drones. Some camera systems, such as the Canon EOS R3 and the Sony a7R V, can recognize specific people and focus on them when they’re in the frame.
Every smartphone photo you capture
Photo Credit: Mike Tomkins
Photos from smartphone cameras shouldn’t look as good as they do. These cameras have tiny sensors and tiny lens elements. It may seem as if the cameras are taking up ever more space on our phones, but remember that there are two, three, or more individual cameras working in tandem to give you approximately the same focal range as an inexpensive kit lens for a DSLR or mirrorless body. The photos they produce should be small and unexceptional.
And yet, smartphone image quality is competing with photos made from cameras with larger sensors and better glass. How? Using dedicated image processors and a pipeline stuffed full of machine learning.
Before you even tap the shutter button on the phone screen, the camera system evaluates the scene and makes choices based on what it detects, such as whether you’re making a portrait or capturing a landscape. As soon as you tap, the camera captures multiple images with different exposure and ISO settings within a few milliseconds. It then blends them together, making adjustments based on what it identified in the scene, perhaps punching up the blue saturation and contrast in the sky, adding texture to a person’s hair or clothing, and prioritizing the in-focus captures to freeze moving subjects. It balances the tone and color and writes the finished image to memory.
 |
| Phone processors, like those from Apple and Google, contain powerful graphics and machine learning accelerator cores, like the ‘Neural Engine’ in the A14 chip. |
Sometimes all this processing is evident, as in the case of people’s faces that look like skin smoothing has been applied or night scenes that look like late afternoon. But in most cases, the result is remarkably close to what you saw with your eyes. It’s possible to divorce the processing from the original image capture by invoking Raw shooting modes or turning to third-party apps. However, the default is largely driven by machine learning models that have been trained on millions of similar images in order to determine which settings to adjust and how people and objects “should” look.
People recognition in software
Before we took advantage of face recognition in camera autofocus systems, our editing software was helping us find friends in our photo libraries. Picking out faces and people in images is a long-solved problem that naturally graduated to identifying specific individuals. Now, we don’t think twice in Google Photos, Lightroom, Apple Photos, or a host of others about being able to call up every photo that contains a parent or friend. Not only can we find every image that contains Jeremy, we can narrow the results to show only photos with both Jeremy and Samantha.
 |
| Google Photos identified all pictures containing Larry Carlson, even photos from when he was younger. |
This technology isn’t limited to photography. DaVinci Resolve can find people in video footage, making it easy to locate all clips containing a specific actor or the bride at a wedding, for example.
Person recognition also enables machine learning-based masking features. Again, because the software knows what a ‘person’ looks like, it can assume that a person prominent in the frame is likely the subject and make a more accurate selection. It also recognizes facial features, enabling you to create a mask containing just a person’s eyes and then apply adjustments to lighten them and add more contrast, for instance.
The humble Auto button
At one point, clicking an Auto button in editing software was like placing a bet: the result could be a winner or a bust. Now, many automatic-edit controls are based on machine learning models. In Lightroom and Lightroom Classic, for example, the Auto button in the Edit or Basic panels pings Adobe Sensei (the company’s cloud-based processing technology) and gathers edit settings that match similar images in its database. Is your photo an underexposed wide-angle view of a canyon under cloudy skies? Sensei has seen multiple variations of that and can determine which combination of exposure, clarity, and saturation would improve your image.
 |
| This unedited raw capture can be edited in several ways in Lightroom Classic using combinations of sliders. |
 |
| Clicking the Auto button in the Basic panel applies light and color adjustments based on millions of similar photos fed into the machine learning models that Adobe Sensei uses. |
Pixelmator Pro and Photomator include ML buttons to apply machine learning to many blocks of controls, including White Balance and Hue & Saturation, to let the software take a first crack at how it thinks the edited photo should appear. Luminar Neo is built on top of machine learning technology, so machine learning touches everything, including its magical Accent AI slider.
What’s great about these tools is that most of them will do the work for you, but you can then adjust the individual controls to customize how the image appears.
Search in many apps
Another lesser-known application of machine learning technologies in photo software is object and scene recognition to find photos and bypass the need to apply keywords. Although tagging photos in your library is a powerful way to organize them and be able to quickly locate them later, many people – advanced photographers included – don’t do it.
Apps already know quite a bit about our photos thanks to the metadata saved with them, including timestamps and location data. But the images themselves are still just masses of colored pixels. By scanning photos in the background or in the cloud, apps can identify things it recognizes: skies, mountains, trees, cars, winter scenes, beaches, city buildings, and so on.
When you need to find something, such as photos you’ve made of farm houses, typing ‘farm house’ into the app’s search field will yield images that include (or evoke) rural buildings. The results aren’t as targeted as if you had tagged them with a “farm house” keyword, but it saves a lot of time narrowing your search.
 |
| A search in the Lightroom desktop app for ‘farm house’ brings up images that do not include those keywords, based only on what Adobe Sensei identified in its scan. |
OK, computer
AI technologies aren’t all about synthetic imagery with seven-fingered nightmare people. Even if you resolve to never create an AI-generated image, when you grab a relatively recent camera or smartphone or sit down to most editing applications, you’ll benefit from features trained on machine learning models.
If they’re working as intended, your attention will be focused on making the image, not the technology.
Author:
This article comes from DP Review and can be read on the original site.
