Editor
Celine Low chevron_right
Trying to squeeze a whole group of people into a selfie is a hassle, familiar to anyone with a smartphone. The end result is trying to take the picture with the phone held horizontally, resulting in an awkward "selfie arm".
But modern smartphone cameras aren't just for taking pictures, they're an important tool for video calls, content creation, and staying connected with others in own lives. Apple takes these factors into consideration and aims to reimagine what an iPhone camera can do.
The company’s new Centre Stage front camera system, first introduced at the "Awe-Dropping" event in September, features two firsts for iPhone: a square sensor and Dual Capture, which lets you record using the front and back cameras at the same time.
As is part of Apple's ongoing philosophy to make technology fade into the background, we looked into the Centre Stage front camera technology and spoke with Jon McCormack (Vice President of Camera and Photos Software Engineering) and Megan Nash (iPhone Product Manager) to understand the motivations behind this.
A differently-shaped sensor
First thing to get out of the way, Apple introduced the industry’s first square front camera sensor on the iPhone. Traditional phone cameras use a rectangular (4:3) sensor, which is optimised for still photography.
"So for the first time ever, we've uncoupled the orientation of your phone and the aspect ratio of your capture," Nash said.
So, a sensor that's equal on all sides allows pictures or selfies to be taken without needing to rotate their iPhones, thus making Center Stage possible. That way, you can hold your iPhone vertically but still capture landscape shots without losing grip on the phone, awkward, off-centered gazes, and losing clarity on the pictures.
"And with this larger sensor, we get more resolution up to 18 megapixels so users have more options to craft and reframe [their shots]," she said.
But, how does the camera know who it should center in a photo?
When you’re on a FaceTime call or taking a picture, the Centre Stage feature uses AI to keep subjects perfectly positioned in the centre of the frame, even if you move around. For group photos, the AI automatically expands the field of view to ensure everyone is included.
McCormack explains that trying to fit people into the frame wasn't the challenge, it was the stability. The camera needs to know what it’s looking at, and movement makes that difficult.
"The first thing we do is stabilise... once the video is stabilised, we can understand what's happening in the frame," he explains.
The AI then focuses on attention. If you're on a call in a busy setting, the phone needs to differentiate between a friend leaning into the conversation and a random person walking past in the background.
McCormack notes the importance of this: "We really want to understand if those folks are part of your story or if they're a random person walking past... Somebody walking by... we're not really going to worry about them until we know that they're a part of your story."
Dual Capture, on the other hand, prioritises visual storytelling
You can also record your reaction in a smaller “picture-in-picture” window and the action happening around you simultaneously.
Watch how Dual Capture works here:
@productnationco What phone’s in his pocket? This was recorded using the Dual Capture mode on the iPhone 17 Pro Max! ð↔️ @apple #iphoneair #iphone17promax #dualcapture #iphone17 #iphone ♬ original sound - ProductNation ð²ð¾
Implementing a square sensor, enabling Dual Capture, and running high-resolution AI framing is an immense technical challenge. McCormack reveals that the idea was explored previously, but the processing power simply wasn't ready.
"We had to wait until Apple silicon came [out] this year because there are so much data to process," Nash confirms, adding that the latest chip was necessary to pull the whole experience together "in a way that becomes so magical and seamless."
And while many competing smartphone manufacturers have moved towards massive, one-inch camera sensors in their main cameras, but Apple continues to focus on a slightly smaller sensor size. McCormack offers his thoughts on the company's approach to their camera designs.
Apple Looks At The Bigger Picture
When Apple designs a camera for their iPhones every year, they consider the "whole package", which includes the device's thickness, battery life, how it fits in your pocket, and how much heat it generates.
McCormack explains that because Apple controls the lens design, the sensor, and the Apple silicon, they have flexibility that others don't. They can focus on software and processing instead of just hardware size.
"Instead of having to create a bigger sensor, we've got other options. We can do better math," he says. "There's Apple silicon that challenges us to do so much more processing to keep the images at an incredibly, world-leading quality without having to introduce other potential compromises into the form factor."
The square sensor might seem like a small change, but there's a whole new way to capture photos and videos on your iPhone thanks to it.
"This is just Apple's approach to building any product that we make. We build it because we think people are going to use it for something great," he said.
Check out the interview here:
@productnationco Here’s why Apple hasn’t used a 1” camera sensor..yet. I got the chance to interview the VP Apple Camera Software Engineer and Apple’s iPhone Product Manager and ask them about their design choices when it comes to camera hardware and processing. What question would you have asked? Lovely opportunity, I couldn’t include the entire interview in this video, so do find the link to the article we’ve done in the bio! @apple #apple #iphone #iphone17 #iphoneair #iphonecamera ♬ original sound - ProductNation ð²ð¾
Stay updated with ProductNation on here, Instagram & TikTok as well.
Read more news on ProductNation: