MariSilicon X is Oppo’s first custom imaging processor for smartphones

It’s peak chip announcement season, it seems, and Oppo has just unveiled its first piece of custom silicon at its Inno Day 2021 — named MariSilicon X. This is not a fully-fledged SoC like Qualcomm’s Snapdragon 8 Gen 1 or Google Tensor, but rather a custom image signal processor, or imaging NPU, imbued with Oppo’s machine learning smarts.

According to Oppo, algorithms are leading the way for mobile photography innovations yet SoC machine learning number-crunching capabilities are currently the bottleneck to better images. Google would probably agree, having dropped Qualcomm’s Snapdragon in favor of its own custom silicon. As such, the MariSilicon X is built to power Oppo’s AI imaging algorithms for pictures and up to 4K 30fps HDR video.

See also: The best camera phones you can buy

If you’re after some numbers, the MariSilicon X boasts 18 TOPS (trillion operations per second) of int8 or 72 TOPS of int4 number-crunching performance as well as a power efficiency of 11.6 TOPS per watt. By itself, TOPS is a rather meaningless metric because it doesn’t tell you anything about the workload. However, Oppo also gave us a real example. Its AI noise reduction algorithm on the Snapdragon 888 powered Oppo Find X3 Pro runs at just 2 fps and required 1,693mW of power. The same algorithm on MariSilicon X runs at 40fps and consumes 797mW.

That’s 20x the performance and less than half the power, at least in this example. All very impressive-sounding but what does this ISP actually do differently?

On a lower level, the 6nm MariSilicon X combines traditional image signal processing (ISP), a neural processing unit (NPU), and memory components into a single chip built specifically for AI imaging.

MariSilicon X offers a 20-bit HDR imaging pipeline capable of running AI algorithms directly on RAW data rather than later down the compression pipeline. Oppo states that this results in a notable 8dB signal-to-noise (SNR) improvement versus processing after RGB and YuV encoding.

Interestingly, the chipset also offers separate RGB and white pixel data channels for RGBW sensors, such as the one shown off by Oppo earlier this year. Again, this results in an SNR improvement of 8.6dB versus merging this pixel data at the sensor level for delivery down a single pipeline. You can think of this somewhat like the days of separate RGB and monochrome cameras but housed in a single sensor. Oppo calls this is RGBW Pro mode and also claims is a 1.7x improvement in texture quality using this technique.

In a nutshell, Oppo’s ISP applies machine learning image and video processing in RAW data directly from the image sensor, resulting in less noise for better-looking pictures. This massive throughput is enabled, in part, thanks to a dedicated memory subsystem in the MariSilicon X. In other words, data doesn’t have to leave the chip. Oppo developed a custom memory architecture that’s essentially a fast cache for the NPU. The NPU is also paired with its own dedicated external DDR memory with 8.5GB/s bandwidth for lower latency loading and storing compared to traditional system memory. This also ensures that bandwidth isn’t hogged by other smartphone applications.

The net result is a powerful, energy-efficient image and machine learning processor built to augment the broader capabilities of a smartphone SoC with unique computational photography and videography smarts.

New use cases for the Oppo Find series

Ultimately, device use cases are what’s most important. Oppo notes that MariSilicon X will power advanced noise reduction capabilities, improved night-time snaps and 4K videography, real-time live previews, and 4K ultra-dynamic range video.

As we identified earlier in the year, custom image processors with AI smarts are quickly becoming a major smartphone trend, especially in the Chinese market. Vivo and Xiaomi already have their own custom ISPs, while Google Tensor goes a step further with a heavy emphasis on machine learning via a semi-custom full SoC. The common thread is that photography is a major part of modern smartphones and one that’s ripe for differentiation and unique selling points. Particularly where advanced machine learning algorithms are concerned. Get ahead in the photography algorithm space and a brand might just land a must-have smartphone.

Read more: Why custom imaging processors are the next mobile photography battleground

We’ll wait to get our hands on a handset packing MariSilicon X before making any conclusions. The chip will debut in the next-gen Oppo Find flagship smartphone that’s scheduled to arrive in Q1 2022.