Qualcomm is expanding its reference camera module program with three new configurations targeting biometrics and depth-sensing functions in Android-based smartphones, tablets, AR (augmented reality) and VR (virtual reality) headsets, and other devices. While the modules' targeted computer vision tasks tend to be computationally intensive, a next-generation ISP (image signal processor) core optimized for the functions is intended to offload CPU, GPU and DSP resources inside a SoC, delivering higher performance at lower power consumption.
In conjunction with the release of Qualcomm's latest Snapdragon 835 application processor, which the company began sampling to its customers late last year, Qualcomm also unveiled a series of camera modules which it had a hand in designing and are manufactured by unnamed subcontractor partners (Figure 1). As explained by Philip-James Jacobowitz, Qualcomm's product marketing lead for camera and computer vision, while the company remains willing to work with its customers to develop solutions based on a variety of sensor and camera models, standardizing on specific module configurations enables customers to get their products to market more quickly. Doing so also makes efficient use of both Qualcomm's and the customer's engineering resources. And customers who standardize on these particular modules also benefit from resultant high volume cost efficiencies.
Figure 1. Qualcomm's reference camera modules, such as these first-generation models launched last year, are intended to help customers reduce engineering effort, bill-of-materials cost and time-to-market.
The baseline single-sensor configuration, based on a Sony IMX298 16 Mpixel image sensor, is intended for rear-camera conventional photography applications and is found in a number of current smartphones. The dual-sensor "ClearSight" configuration, according to Jacobowitz, boosts low-light performance by blending the outputs of monochrome- and Bayer pattern-filter sensors (from an unspecified supplier), and is today found in products such as the new Essentials smartphone. And the misnamed "Optical Zoom" configuration, again using sensors from an unnamed supplier, leverages camera modules containing fixed-focal length wide angle and telephoto lens, with sophisticated digital interpolation between their outputs to implement zooming effects. It's found, according to Jacobowitz, in devices such as the Oppo R11 and OnePlus 5.
This year's modules focus first on the front camera arrangement (Figure 2). While a conventional visible-light sensor will remain necessary for "selfies" and other conventional front camera applications, according to Jacobowitz, alongside it Qualcomm is advocating the inclusion of an infrared transmitter-plus-receiver cluster intended for iris scanning and other biometrics functions. The infrared approach delivers a two-fold benefit: it allows for operation in dimly lit settings, along with enabling scanning to work even if the subject is wearing sunglasses. At the core of the company's "Iris Identification" module is OmniVision Technologies' OV2281 1920x1080 pixel, 1.12μm, 10-bit IR sensor; the associated IR transmitter supplier and model is unspecified. Qualcomm claims that its iris-scanning algorithm operates in only 40 ms, and includes "liveness detection" features that aren't fooled by 2D images or 3D molds.
Figure 2. Next-generation infrared-based modules target both biometrics and depth sensing applications. An alternative passive stereo rear camera is lower in cost and reliably usable in outdoor environments, at the tradeoff of reduced depth accuracy.
Infrared technology is also at the nexus of one of Qualcomm's two new depth sensing rear camera configurations. Specifically, Qualcomm has selected the structured light depth sensing approach previously leveraged by Microsoft in the Xbox 360's Kinect peripheral and also found in Intel's RealSense camera line, versus the time-of-flight approach employed by Microsoft in the follow-on Kinect for the Xbox One and chosen by Google for its Project Tango AR platform. Jacobowitz touted structured light technology's high accuracy and comparatively compact size and light weight, as well as low module power consumption and cost (Video 1). However, like time-of-flight, structured light can be unreliable when used outdoors and in other environments with high ambient infrared light levels. In such situations, passive stereo, Qualcomm's other new depth-sensing camera configuration, offers a more robust approach. Structured light's other historical shortcoming, which it shares with passive stereo, is that it's more computationally intensive than time-of-flight, which can lead to high power consumption in the absence of hardware acceleration.
Video 1. Qualcomm touts its structured light depth sensing module's ability to capture up to 10,000 points of depth data, with the spacing between points as low as 0.1 mm.
That's where Qualcomm's next-generation Spectra ISP core fits in. The Spectra 180 found in the Snapdragon 835 was architecturally akin to the Spectra 160 used in previous application processors, and primarily relied on lithography shrink- and layout optimization-based performance boosts for its modest uptick in maximum-resolution sensor support and other enhancements. Conversely, according to Jacobowitz, the next-generation Spectra (likely to be referred to as the 2xx series, although final naming is still being determined) is a "ground-up redesign". Its intention (aside from the usual increases in per-sensor resolution, frame rate and the like) is to offload depth analysis, noise reduction, motion-compensated temporal filtering and other electronic image stabilization functions, SLAM (simultaneous localization and mapping) algorithms, and other pre-processing tasks currently handled by the CPU, GPU and Hexagon DSP cores in a Snapdragon SoC, translating into a more performance- and power consumption-optimized heterogeneous processing end result. Support for deep learning acceleration is another key enhancement in this next-generation Spectra ISP architecture, according to Jacobowitz, with more details to follow in the "near future".
To that latter point, with the Spectra 180-based Snapdragon 835, Qualcomm executed a multi-stage rollout: an initial introduction in November 2016, followed by a fuller SoC rollout at the January 2017 Consumer Electronics Show, concluding with initial customers' Snapdragon 835-based system announcements at (and shortly after) the March 2017 Mobile World Congress conference. Jacobowitz indicated, without giving specific dates, that the first next-generation Spectra-based SoC would follow a similar rollout path, beginning "later this year." Along with providing details on the SoC itself, Qualcomm plans to also release additional information on the next-generation Spectra ISP at that time. And the SoC's name? While "Snapdragon 845" has surfaced in recent public documents related to Qualcomm's various legal activities, Jacobowitz indicated that this naming is also not yet finalized.
Add new comment