Bring out the Best in Pixels · Bring out the Best in Pixels . ... Content-adaptive per-pixel...
Transcript of Bring out the Best in Pixels · Bring out the Best in Pixels . ... Content-adaptive per-pixel...
Bring out the Best in Pixels Video Pipe in Intel® Processor Graphics
Victor H. S. Ha and Yi-Jen Chiu
Graphics Architecture, Intel Corp.
INFORMATION IN THIS DOCUMENT IS PROVIDED IN CONNECTION WITH INTEL PRODUCTS. NO LICENSE, EXPRESS OR IMPLIED, BY ESTOPPEL OR OTHERWISE, TO ANY INTELLECTUAL PROPERTY RIGHTS IS GRANTED BY THIS DOCUMENT. EXCEPT AS PROVIDED IN INTEL'S TERMS AND CONDITIONS OF SALE FOR SUCH PRODUCTS, INTEL ASSUMES NO LIABILITY WHATSOEVER AND INTEL DISCLAIMS ANY EXPRESS OR IMPLIED WARRANTY, RELATING TO SALE AND/OR USE OF INTEL PRODUCTS INCLUDING LIABILITY OR WARRANTIES RELATING TO FITNESS FOR A PARTICULAR PURPOSE, MERCHANTABILITY, OR INFRINGEMENT OF ANY PATENT, COPYRIGHT OR OTHER INTELLECTUAL PROPERTY RIGHT.
A "Mission Critical Application" is any application in which failure of the Intel Product could result, directly or indirectly, in personal injury or death. SHOULD YOU PURCHASE OR USE INTEL'S PRODUCTS FOR ANY SUCH MISSION CRITICAL APPLICATION, YOU SHALL INDEMNIFY AND HOLD INTEL AND ITS SUBSIDIARIES, SUBCONTRACTORS AND AFFILIATES, AND THE DIRECTORS, OFFICERS, AND EMPLOYEES OF EACH, HARMLESS AGAINST ALL CLAIMS COSTS, DAMAGES, AND EXPENSES AND REASONABLE ATTORNEYS' FEES ARISING OUT OF, DIRECTLY OR INDIRECTLY, ANY CLAIM OF PRODUCT LIABILITY, PERSONAL INJURY, OR DEATH ARISING IN ANY WAY OUT OF SUCH MISSION CRITICAL APPLICATION, WHETHER OR NOT INTEL OR ITS SUBCONTRACTOR WAS NEGLIGENT IN THE DESIGN, MANUFACTURE, OR WARNING OF THE INTEL PRODUCT OR ANY OF ITS PARTS.
Intel may make changes to specifications and product descriptions at any time, without notice.
All products, dates, and figures specified are preliminary based on current expectations, and are subject to change without notice.
Intel processors, chipsets, and desktop boards may contain design defects or errors known as errata, which may cause the product to deviate from published specifications. Current characterized errata are available on request.
Any code names featured are used internally within Intel to identify products that are in development and not yet publicly announced for release. Customers, licensees and other third parties are not authorized by Intel to use code names in advertising, promotion or marketing of any product or services and any such use of Intel's internal code names is at the sole risk of the user.
Intel product plans in this presentation do not constitute Intel plan of record product roadmaps. Please contact your Intel representative to obtain Intel’s current plan of record product roadmaps.
Ultrabook™ products are offered in multiple models. Some models may not be available in your market. Consult your Ultrabook™ manufacturer. For more information and details, visit http://www.intel.com/ultrabook
Performance claims: Software and workloads used in performance tests may have been optimized for performance only on Intel® microprocessors. Performance tests, such as SYSmark and MobileMark, are measured using specific computer systems, components, software, operations and functions. Any change to any of those factors may cause the results to vary. You should consult other information and performance tests to assist you in fully evaluating your contemplated purchases, including the performance of that product when combined with other products. For more information go to http://www.Intel.com/performance
Copyright © 2013 Intel Corporation. All rights reserved. Intel, Intel Inside, the Intel logo, Intel Core, and Ultrabook are trademarks of Intel Corporation in the United States and other countries. *Other names and brands may be claimed as the property of others.
Legal
Contents Intel Processor Graphics
Video Processing Pipe Video Processing Modules Control Panel Adaptive Processing
Summary
Q&A
3
• 4th Generation Intel® Core™ Microprocessor, built on 22 nm process technology
• Next Generation Intel® HD Graphics with Microsoft* DirectX*11.1, OpenGL* 4.0, OpenCL* 1.2 support
• Three Simultaneous Display, HDMI, DisplayPort*, with high-resolution up to 4Kx2K
• Significant 3D and Media performance improvement
5
Vertex Processing
Rasterize
Pixel Ops
Multi-Format Codec
Media Processing Display
Command Streamers
Array of Unified Execution Units
Instruction Cache
EU EU EU
EU EU EU
Texture Sampler
Media Sampler
Video Quality Engine
6
Intel Graphics Media Processing
WiDi
Interlaced / Progressive
Chroma Format: 420/422/444
JPG, MPG, H.264, VC1
DVI/HDMI
Display Port
VGA Ethernet/LAN
WiFi
HDD
DVD/BD Network Mobility
Source Format, Content, Quality WebCam
USB
SD, FHD, UHD 4K
BT
7
Multi-format Codec
VQE
Media Sampler
0101010001001011101010101… Denoise
Deinterlace Contrast
Saturation Skin-tone
Gamut Color Correction
Stabilization
Decode MPEG2 / VC1 AVC SVC MVC JPEG / MJPEG
Scaling Sharpness
Bring out the Best in Pixels
Contents Intel Processor Graphics
Video Processing Pipe Video Processing Modules Control Panel Adaptive Processing
Summary
Q&A
8
Video Quality Engine • Video Processing • Color Processing
9
Command Streamers
3D Geometry Processing
Rasterizer
Media FrontEnd
Multi -format CodecVideo Processing Pipeline
VQE
$L3 Cache
EU EU
EU
EU EU
EU EU EU
Array of Execution UnitsMedia
Samplers
TextureSamplers
Pixel Ops
Media Accelerators
Introducing Video Quality
Engine
Multi-Format Codec • Video Decode and Encode • High Performance Parallel Engine
Front-End Thread Dispatch/Management
Media Optimized Execution Units
Video Processing Pipe
Dedicated video processing on newly designed Video Quality Engine (VQE)
Support for an extensive suite of functions for higher quality video at lower power
10
Denoise Deinterlace
Scaler Sharpness Image Stabilization
Skin-tone Gamut Compression Saturation
Contrast ProcAmp
CSC Gamut Expansion
Contents Intel Processor Graphics
Video Processing Pipe Video Processing Modules Control Panel Adaptive Processing
Summary
Q&A
11
De-noise
Spatial and Temporal De-noise Filter Global noise level measurement Content-adaptive spatiotemporal filtering of noise Motion history-based blending of spatial and temporal filter results
Block Interface Input: YCbCr 420/422 Output: YCbCr 420/422
12
Noisy Images Clean Images
De-interlace
Convert interlaced contents to progressive format Edge-adaptive Spatial Interpolation result + Motion-compensated Interpolation result
Motion-adaptive Blending with Temporal Filtering results based on Spatial Temporal Motion Measure (STMM)
Detection of most common cadences
Chroma Upsampling from 420 to 422
13
Skin-tone Processing
Per-pixel Enhancement of Skin-tone Pixels Reproduce the natural skin colors on the display screen Skin Tone Detection identifies pixels with skin-like colors with per-pixel indicator Skin Tone Enhancement modifies the Saturation and Hue of the skin-tone pixels
Block Interface Input: YCbCr444 Output: YCbCr444 with modified CbCr components; Per-pixel skin tone indicator
14
Skin-tone Detection
Skin-tone Definition
Skin-tone Enhancement
Skin-tone Indicator
Original Input Skin-tone Enhanced
Output
Contrast Enhancement
Automatic Contrast Enhancement: Per-pixel mapping of luma to enhance contrast 1. Histogram of luma Y pixel values is generated for the input video frame 2. Piece-Wise Linear Function (PWLF) is generated from luma histogram 3. Pixel values are modified according to the PWLF
Block Interface Input: YCbCr 444 Output: YCbCr 444 with modified Y
15
Histogram
Computation
PWLF
Mapping
Input Image
Enhanced Image
Saturation Enhancement
Per-Pixel Saturation Enhancement Utilize 6 basic colors as primaries/anchors (Red, Green, Blue, Magenta, Yellow, Cyan) Adjust colorfulness (saturation) of pixels while maintaining their color (hue)
Block Interface Input: YCbCr 444 Output: YCbCr 444 with modified CbCr components
16
Input Image Saturation Enhanced
Image
Color Correction
Display proper colors on display screen 1. Inverse gamma correction via PWLF 2. 3x3 matrix multiplication with input/output offset 3. Forward gamma correction via PWLF
Block Interface Input: RGB Output: RGB
17
Input Image
Color Corrected
Image
Image Stabilization
Stabilize shaky video contents captured by handheld camera devices Usage models
Real-time playback: Watch real-time playback of video streams
Offline processing: Stabilize shaky video and save/transmit for video sharing
Block Interface Input: YCbCr 420
Output: YCbCr 420
18
Scaler
Adaptive Video Scaler Advanced scaling method for adaptive scaling mode Content-adaptive per-pixel blending of polyphase filtering and bilinear filtering reduces ringing Programmable coefficient tables available for Y / CbCr channels in horizontal and vertical directions Support for up to 16:1 downscaling and max output picture size of 16Kx16K for
YUV420/422/444/RGB input picture
19
Sharpness Enhancement
Enhancement of Image Sharpness and Details Content adaptive spatial filtering applied to luma channel
User control of sharpness enhancement strength = [0, 63]
Adaptive to Skin Tone Pixels Skin tone information is utilized
Enhance detail information without over-sharpening of the skin regions
Interface Input: YCbCr 444
Output: YCbCr 444
20
Contents Intel Processor Graphics
Video Processing Pipe Video Processing Modules Control Panel Adaptive Processing
Summary
Q&A
21
23
Control Panel • Input Range
– Limited – Full
• Total Color Correction – Saturation Enhancement – Adjustment = [0, 255]
24
Control Panel • Sharpness
– Driver Custom Settings = [0, 64]
• Skin Tone Enhancement – On = [0, 9] – Off
25
Control Panel • Noise Reduction
– Luma only = [0, 64] – Luma and Chroma = [0, 64]
• Contrast Enhancement – On/Off
• Film Mode Detection – On/Off
Contents Intel Processor Graphics
Video Processing Pipe Control Panel Video Processing Modules Adaptive Processing
Summary
Q&A
26
Hardware Statistics
Statistics are gathered on Per-block (16x4) basis for encoder stats Per-frame basis for other stats
16 bytes of encoder statistics data for per-block statistics is available Temporal Variances De-noise: Sum of block noise estimates and Number of blocks per frame
A variety of per-frame data is stored in a linear surface Skin-tone: Ymax = Max luma value, Ymin = Min luma value, Counter = Number of skin pixels Contrast: Luma histogram with 256 bins and 24-bits per bin Gamut Compression: Sum of distances of out-of-gamut-range pixels clipped to 32 bits, Number of
out-of-gamut-range pixels in 29 bits
27
Adaptive Processing & Control
The goal of adaptive processing is to deliver optimized video experience
Hardware statistics and adaptive control logic allow end-to-end optimization and effective sharing of data and information across all processing blocks
28
APC Example
29
Skin-tone Detection
Saturation Enhancement Noise Reduction
Motion Detection
Skin-tone Indicator Global Motion Indicator
Noise Level Estimator
Global Noise Level
Sharpness Enhancement
Brightness Information
Contrast Enhancement
Luma Histogram
Contents Intel Processor Graphics
Video Processing Pipe Video Processing Modules Control Panel Adaptive Processing
Summary
Q&A
30
Summary
Intel is building Media Solutions for Quality, Power, and Scalable Performance
Dedicated HW acceleration for codecs, video and image processing, and analytics
Media processing in 4th Generation Processor Graphics delivers: VQE: A new feature with a full-set of video processing pipe Adaptive Processing & Control for integrated and optimized video pipe behavior Ready for various application scenarios and network/mobility environments
Looking Ahead Future generations of Intel media processing will address further HW acceleration in video
processing, camera image processing, and perceptual computing/machine vision applications Improvement in power and performance fitted to exciting new form-factors
31
Contents Intel Processor Graphics
Video Processing Pipe Video Processing Modules Control Panel Adaptive Processing
Summary
Q&A
32