Vision science studies suggest that the eye is able to discern more than 11 bits of dynamic range for each of the three primary colors – red, green and blue – that typically comprise a given scene. The optical nerve connecting each eye to the brain, on the other hand, is only able to pass roughly five bits' (40 levels) worth of each primary color's data. Yet the brain still is capable of discerning more than 10 billion discrete levels of total color depth, equivalent to that of the 11-bit-per-color (2000+ level-per-color) source. How can such aggressive compression through a bandwidth-constricted intermediary channel be possible? A compression algorithm startup company called Trusight, founded in 2009, thinks it has solved the puzzle. The answer, according to Trusight, centers on luminance.
Two primary lossy-compression schemes currently find use in still and video imaging applications. Intra-frame spatial compression, employed (for example) in JPEG and wavelet schemes, involves DCT (discrete cosine transform) or other frequency transformations of pixel clusters, followed by coefficient quantization (to eliminate information deemed relatively unimportant given the compressed file size/bitrate target) and lossless Huffman encoding. Intra-frame temporal compression (relevant for multi-frame video content) leverages the similarities in successive frames to, identify and code approximate pixel cluster location relationships between frames where possible, versus the individual frames' discrete details.
According to Trusight, typical JPEG spatial compression might reduce an image from 8 MBytes to 800 KBytes with little to no perceived quality degradation, a 10x size decrease translating to roughly 4 bits of dynamic range reduction. And temporal compression results are highly dependent on source material characteristics. So how does Trusight's algorithm work its additional bitrate-slimming magic? Company representatives reiterated several times during a recent briefing that the proprietary approach does not alter the source image's color information; it focuses only on dynamic range reduction (i.e., compression) of luminance
Trusight's technology operates on every pixel in an image, but (unlike with a rudimentary luminance-scaling algorithm) in a selective manner, as the following example images suggest (Figures 1 and 2):
Figure 1. Notice how the landscape in the original image (top) is brightened by Trusight's algorithm, but the sky is left essentially untouched (bottom).
Figure 2. Trusight's Vice President of Engineering, Randall Eike, is obscured by shadows in the original image (top) but revealed in the Trusight-processed version, without washing out his family's facial features and other details in the process (bottom).
Trusight claims that its approach requires only a scant processing burden, unlike the earlier-mentioned spatial and temporal compression schemes. The company goes so far as to call it a "rounding error on current processors." Quantifying the claim, company officials estimate only a 15 to 30 instruction-per-pixel requirement for the algorithm, alternatively translating (in a dedicated-hardware implementation) to 50,000-or-less gates of logic.
Trusight estimates that its approach can reduce transmission bandwidth by 15 to 30 percent, while preserving (if not improving) perceived image quality. The algorithm operates on video in its "raw" RGB or YUV forms, and is therefore compatible with any conventional compression technique. Luminance compression can be implemented at any point in the image generation, processing, transmission and output chain, but should not be redundantly employed at multiple points in the process. Company officials indicate that they have a "secret sauce" to ensure that such processing repetition doesn't occur, but declined to provide specifics.
The inherently brighter images generated by Trusight's approach can also reduce the backlight intensity requirement for subsequent display at the destination. This attribute potentially lowers system power consumption to a notable degree (estimated at 15 to 40 percent for a smartphone or tablet, for example) without a quality penalty, as the following demonstration quantifies:
To date, the company has implemented its algorithm in both C code (for execution on a conventional processor or DSP) and OpenGL routines (for execution on a GPU). An OpenCL implementation is currently under development, with target completion within 90 days. To wit, Trusight has publicly announced an OpenCL partnership with Imagination Technologies (PDF), which CEO Scott Slater claims was done at the urging of Apple (which employs Imagination's PowerVR graphics cores in its ARM-based SoCs).
Trusight is vague on its licensing aspirations, indicating only that for playback devices the fees will be calibrated both to anticipated product volume and battery size-and-cost savings. The license fee for a tablet computer normally using a $12 battery might be $0.25, for example, while for a smartphone traditionally containing a $6 battery it might be $0.10. Similarly, for a server-side Trusight implementation, the encoder-implemented license fee would be commensurate with the anticipated storage and bandwidth savings.
The company's executive team has an impressive pedigree. CTO Kevin Grundy, for example, was the third employee hired by Steve Jobs at NeXT Computer. And Trusight's quality, bandwidth, and power consumption optimization assertions are equally compelling, particularly in conjunction with the company’s claims of scant processing requirements for its algorithm. Real-life implementations to date are few in number and not public. However, if reality matches (or even closely approximates) the hype, and if the company licenses its approach broadly and in a cost-effective manner, the combination of Trusight and a mainstream compression scheme such as H.264 may prove to be a compelling alternative to a next-generation algorithm such as H.265.
If you'd like to test-drive Trusight for yourself, free "Lite" and low-cost "Pro" applications are currently available for iOS-based devices.
Add new comment