Qualcomm’s next Snapdragon promises always-on smartphone cameras

• Kryo (Snapdragon 8) CPU: Octo-core architecture: 1 Arm Cortex-X2 (3.0GHz, prime) + 3 Cortex-A710 (2.5GHz, performance) + 4 Cortex-A510 (1.8GHz, efficiency)
• Memory support: Up to 16GB of LP-DDR5 memory, with speeds up to 3200 MHz
• Adreno (Snapdragon 8) GPU: On-device display support up to 4K@60Hz or QHD+(2,880×1,440) @144Hz; external displays up to 4K@60Hz; HDR10, HDR10+ support with Rec.2020 color gamut; Vulkan 1.1 support with HDR gaming; HDR10+, HLG and Dolby Vision support for HDR playback
• Spectra (Snapdragon 8) ISP: 200Mpixel photo capture (200MP still images (108MP/30 fps, single camera; 64MP + 36MP/30fps, dual-camera; 36MP/30 fps, triple camera, all with zero shutter lag); 8K HDR video capture@30 fps+64Mpixel photo capture; 4K video capture @ 120 fps; slow-mo 720p @ 960fps; bokeh engine for video capture, 10-bit HEIF photo capture
• Hexagon (Snapdragon 8): Total Harmonic Distortion + Noise (THD+N) Playback: -108dB
• Connectivity (5G): Integrated x65 5G modem: mmWave (1000MHz bandwidth, 2×2 MIMO), Sub-6GHz (300MHz bandwidth, 4×4 MIMO); LTE support (CBRS, WCDMA, HSPA, TD-SCDMA, CDMA 1x, EV-DO, GSM/EDGE); AI Enhanced Signal Boost
• Connectivity (Wi-Fi): FastConnect 6900 (Wi-Fi 6e/802.11ax, 802.11ac), 3.6 Gbps peak speeds; 4K QAM, OFDMA; MU-MIMO (up to 8×8 MU-MIMO)
• Connectivity (Bluetooth): Bluetooth 5.2, with support for Qualcomm aptX Voice/Lossless/Adaptive specifications
• Qualcomm Sensing Hub
• Power: Quick Charge 5

AI

Artificial intelligence is sometimes difficult to define on the PC, but on the smartphone it’s quickly become synonymous with two things, computational photography and how your phone processes voice commands. With the Snapdragon 8 Gen 1, Qualcomm has improved across both vectors, according to Ziad Asghar, Qualcomm’s vice president of product management of its Snapdragon technology roadmap.

From a performance perspective, Qualcomm’s numbers have improved quadruple the AI performance, with twice the tensor accelerator performance and 1.7X more power efficiency, Asghar said. There will be 8-bit plus 16-bit mixed integer support. What will be more interesting is what the AI engine can do such as video bokeh effects from ArcSoft, improved natural language processing, and a new “Leitz Looks” filter from Ernst Leitz Labs that you’ll be able to apply to photos to give them the look of Leica cameras.

The Snapdragon 8’s Hexagon AI engine can also interact with the integrated 3rd-gen sensing hub, with a couple improvements that could seem cool or otherwise very creepy. Asghar said that Qualcomm is working with Sonde to develop technologies to listen to your voice as you speak to determine if you’re suffering from a health issue such as asthma or the coronavirus. Hugging Face natural language processing will scan your voicemail messages and apply sentiment analysis. Is it really important that your brother speak with you?

Finally, Asghar said, Qualcomm is adding a “completely new experience,” which is the always-on camera system. What that allows you to do is bring in completely new use cases where your phone can unlock just by looking at it and “also prevent unnecessary looks at your display,” Asghar said. Essentially, your phone will be always looking for your face, eliminating the need to “unlock” it.

Qualcomm may be walking a fine line here. Google Glass failed, in part, because users were never sure if they were being recorded or if their behavior was being tracked and there were similar questions asked of Facebook regarding the recent introduction of its own smartglasses. Some laptops have included privacy shutters over their webcams because of concerns, justified or not, that users could be illicitly recorded.

In response to a question about the ethics of an always-on camera, Asghar noted that Qualcomm attempts to “make sure that the data that is there stays on the device.” “As you know, these are capabilities that add to experiences…and [we want] to make sure that we have solutions that are abiding by all the privacy parameters and everything,” Asghar added.

More importantly, Asghar said he believed that the always-on camera would be an OEM-enabled feature, meaning that it will be up to the smartphone OEM to actually enable it as well as provide user controls to manage its use.

Camera

Otherwise, the Snapdragon 8 Gen 1’s camera has added some interesting new features.

Qualcomm is calling its new camera ISP brand “Snapdragon Sight,” representing what the Snapdragon 8 Gen 1’s camera can do. It can take pictures, certainly, but also recognize what the camera is taking photos of. Last year’s Snapdragon 888 offered AI-based auto-exposure and auto-face detection, but the Snapdragon 8’s Spectra Image Signal Processor now performs intelligent face detection. That matters, according to Judd Heape, vice president of product management at Qualcomm, when the camera is asked to unlock the phone for a user whose face is partially obscured by a mask. The camera’s sensor can now map up to 300 “facial landmarks”, tracking your eyes, nose, mouth, cheeks, and so on for animojis and other applications where the camera has to map your facial expression.

Qualcomm

Smartphone cameras powered by Qualcomm Snapdragon 8 processors will also feature dynamic range that has increased by four stops, part of a move from a 14- to an 18-bit ISP. While the 888’s camera ISP allowed users to shoot 8K video, the 8’s ISP will add HDR capabilities to 8K video.

The AI engine and camera engine will work together to perform extreme ultra zooming. Basically, it’s digital zoom with smart interpolation to eliminate graininess. Last year’s Snapdragon 888-powered “night mode” stitched together about six frames to form a composite image. The Snapdragon 8’s camera will combine about 30 frames, delivering “five times better night mode,” Heape said. That’s because each pixel in a night scene is tracked, so the camera can intelligently filter out the pixel “movement” caused by your hands shaking. The result will be a sharper, crisper image, he said. Finally, the company is using AI to eliminate chromatic aberration from wide-angle lenses.

Heape also provided a few more details about the always-on camera will work. Essentially, it will be able to automatically unlock your phone, but also notice when someone may be peering over your shoulder and alert you. We’ve seen this before in the PC space. The “Glance” utility Lenovo bundled with its ThinkPad X12 Detachable Gen 1 tablet, for example, blurs your screen if you’re not there. Dell’s laptops simply detect your presence, then turn on the Windows Hello-enabled webcam.

.

Gaming

Mobile gaming may not be something you think of when you think of mobile phones — well, nothing more than a casual round of Candy Crush. But mobile versions of Player Unknown: Battlegrounds and similar games can sell smartphones, too. Here, processing power and GPUs rule, just as they do with PCs.

Qualcomm is promising three improvements with the Snapdragon 8. First, a new Adreno GPU frame motion engine that will essentially double frames at the same power consumption, or reduce power up to 50 percent at the same frame rate. Second, the new chips will add “desktop-level volumetric rendering,” executives said. Finally, the company is promising a “pro” level version of variable rate shading, which will allow the GPU and display to sync up to provide “tear-free” gaming with fewer visual artifacts.

Modem, trust and security

On the modem side, Qualcomm is bringing together full mmWave and sub-6-GHz enablement, Asghar said. More importantly, Qualcomm is introducing 5G uplink carrier aggregation, where multiple uplink channels will be able to be used simultaneously to improve upload bandwidth. WiFi 6 and WiFi 6e are also being added, as is Bluetooth Low Energy (BLE) and support for CD lossless audio. Finally, Qualcomm is adding something called a “trust management engine,” which is designed to help users secure confidential information.

Qualcomm

Qualcomm has yet to reveal whether updates to its Snapdragon Compute line of PC processors are en route. However, the company’s Snapdragon Tech Summit will run for another day.