Multimedia in Every Mobile
Transcript of Multimedia in Every Mobile
A statement about the launching of SMedia
q Color mobile-phone occupy over 60% mobile-phone market in 2004
q Camera phone occupy over 50% of color mobile-phone in 2004q Multimedia become default function of mainstream handheld
device in 2005q Enhanced capability drives data bandwidth and carrier revenueq Multimedia offers handset differentiation opportunityq Consumers will pay for compelling games and contentq SMedia delivers key enabling technologies for multimedia
handheld devices
Founded: 2004 AugInitial Capital: USD 6 Million (approximately)Major Shareholder: UMCEmployees: 60 (approximately)Headquarters: Taiwan/ Hsin-ChuOperating Sites: S&M, Service: Taiwan/ Taipei
SMedia at a Glance
Core Competence
R&D- 100% own IP for core multimedia features- First full hardwired solution of low-power graphics, streaming,
imaging integration in the world- Initial/Endorse industry standard via participating these
committee/organization
Service and Support- Long-term relationship and timely support to customers with time-to-
market product support- Close relationship with alliance partners to synchronize in latest
technology development and industry specifications.
Imaging
VideoGraphics
Audio
uP
LowPower
Innovation
Integraion
Core Value
Integrated core technologies into real innovation
Deliver the valuable innovation to customers completely
Deep service
Customer
Satisfaction
>Real-time feedback >>Efficient logistics >
> Knowledge transfer>> Total solution >
Note:SiS GPU department spun off at Q2’03. So there is no correct statistic data after Q3’03. SiS GPU department combined Trident Inc. as a new company – XGi, which became the top 3 GPU suppliers, and SiS is still the biggest shareholder.
About SiS Multimedia Score
Desktop Graphics Market Share Q2'03
Intel35%ATI
13% Nvidia31%
SiS10%
Matrox1%
VIA/S39%
Others1%
Source: Jon Peddie ResearchDiscrete Desktop Market Share Q2'03
Matrox3%
ATI28%
SiS4%
Nvidia65%
IGC Desktop Market Share Q2'03
ATI<1%
Intel64%
SiS16%
VIA/S317%
Nvidia3%
Polyphonic
Smart message
Multimedia
Inte
ract
ive
Native 2D Gaming
Instant Messaging
SMS EMS MMS
Video Conference
On-line Gaming
Video Broadcast
Native 3D Gaming
3D MMI
WAP Broadcast
Still CameraVideo
Camcorder
4 Development Directions for Feature Phone: Multimedia, Interactive, User-friendly, Personalization
Mobile Phone (Handheld device) Trend
Inverse Screen Size Law
The need for advanced graphicsSmaller screens need more advanced graphics processing per pixel
To make best use of restricted screen real-estate
Advanced graphics techniques provided by state-of-the-art APIsAnti-aliasing, multi-level compositing, rich media, 3D
Interactive3D Gaming Content
Video Processing and synchronization
Perspective windows maximize screen real-estate
Anti-aliased text to makesmall text extremely legible
Translucent Menus don’t obscure lower
layers
Asynchronous SRAM
VideoCapture,Image
Processor
JPEG,MPEG4Codec
2D/3DGraphics
LCDController
Host BusController
Memory Controller
LCDCOMSSensor
Host CPU
MMCController MMC
Asynchronous SRAM
VideoCapture,Image
Processor
JPEG,MPEG4Codec
2D/3DGraphics
LCDController
Host BusController
Memory Controller
LCDCCD/CMOSSensor
Host CPU
MMCController MMC
§ SMedia is integrating the interactive multimedia visual solution for handheld devices to enable new applications that enrich more enjoyment and quality to life.
SMedia Solution
Camera Interface
2D EngineLCD
Interface3D Engine
JPEG decode JPEG encode
MPEG decode MPEG encode
LCD
BBMemory
Power ManagementBattery
RF
Antenna
Melody &Audio Amplifier
(MP3/MIDI… )Speaker &
Vibrator
Lens Sensor
Stacked SRAM
Multimedia Mobile Phone Evolution
Possible Architecture 1
JPEG codec
MPEG4 codec
2D engine
3D engine
Cam
era sensor interface
LCD
interface
RAM
Lens SensorImaging DSP
YUV LCD
Mem
(CPU interface)
Traditional multimedia phone architecture
B.B.
JPEG codec
MPEG4 codec
2D engine
3D engine
CM
OS
sensor interface
LCD
interfaceLens Sensor
LCD
Newer multimedia phone architecture
Imaging DSP
Raw RGB
RGB
B.B.
Possible Architecture 2
RAM MemAdvantage:•Save total system cost, theoretically•From graphic view, it can perform much better quality
Disadvantage:•More bandwidth usage of stack pSRAM for LCD display
Advantage:•Save total system cost, theoretically
Disadvantage:•Need extra effort for calibration•Lack flexibility
Geometry performance with lighting§ Glamo 3D Hardware : minimum 100k
polygons/sec (15 times of the StrongARM200Mhz with 160Mhz FPU)
Rendering performance with texturing§ Glamo 3D Hardware : 5M pixels/sec (80 times
of the StrongARM 200Mhz with 160Mhz FPU)
For Performance
ISCAS 2002 IEEE International Symposium on Circuits and Systems Paper : 2490
Reference data:
a) QVGA 320x240x2 overdraw x30fps/(5x5/2pixels/triangle)=369Kpoly/s
b) QVGA 320x240x2 overdraw x30fps=4.6Mpixels/s
Max 1280k polygons/sec
Max 33M pixels/sec
Why H/W Graphic Accelerator
Why H/W Graphic AcceleratorFor Power Consumption
§ Software solution (executed by CPU)§ Define every picture§ Clean & Redraw every movement on display§ Execute complex instructions to compute simple movement§ Saturation accessing memory
§ Hardware solution§ Only define some components:
§ Object oriented operation to execute a simple instruction to do complex action
Comparison of Performance & PowerPerformance: 4.6 times speed (200x200pixel BLTs x 30 times)
25ms 115ms
PDA with PXA250 + Graphic Accelerator
PDA with PXA250 only
Only initial (around 10ms) need higher power
Power consumption: 85% less power (see cyan-blue portion)
Outstanding MPEG4 CodecFull hard-wired solution à better performance + less power consumption
§ vs E company’s half H/W + half SW
Full support MPEG4 simple profile à better quality & efficiency§ Based on the VM 18 & MPEG-4 reference code:
§ A modified mode decision algorithm for deciding the MB mode. Can achieve over 0.2dB improvement for complex sequence and 1.0dB in scene change.
§ A new rate control algorithm can remain high quality when scene changed.§ Full search in search range
§ I & P frame support§ P frame will have over 5X performance/quality than I frame
§ Special “Scaling-down”process as a partial low-pass filter to reduce the blocking & ringing
§ Rate-control support to meet HVS(Human Visual System) behavior§ Human eyes are more sensitive on low-complexity MB(Macro Block), rate –control
can reduce blocking impact§ vs A company’s H.263 only à Just one of MPEG-4 mode (short-head mode)§ vs A company’s “No rate control”§ vs S & A companies’I frame support only§ vs S & A companies’blocking & ringing result
Ringing & Blocking Example
Why H/W Camera/CamcoderDramatics Power Saving: around 10X~
0
50
100
150
200
250
300
350
400
450
MPEG4 Encode(CIF@30fps)
JPEG Encode(Continous encode
1.3M)
Preview(1.3M@15fps; 60fpsLCD update)
ARM9 core
Glamo core
Technology of Power-saving§ Clock§ Low power PLL for 32KHz clock input (standard RTC clock for the standby usage
of mobile-phone)
§ Active Graphics§ Drawing and rendering more power efficient on dedicated hardware than on
Baseband§ SMedia solution spends less time drawing power by her decades of graphic
experience(SiS experience)
§ Image capture & video recording§ Page-mode processing to minimize memory accessing times which is the major
power consume
§ Screen Refresh§ Display refresh isolated from system & Baseband§ Partial Display make the efficient power saving for whole system
§ ASIC Design for Power§ Stack pSRAM, not SDRAM§ Auto switch-off non-active function§ Low-power process
Best Solution (1)§ Economical system solution§ Add ISP to reduce camera module cost – save around $0.5 for VGA, around $0.5~ $1
for 1.3M model§ Add LCD controller w/ frame buffer to reduce LCM cost – save around $4+ for 176x220
model; save around $10+ for 240x320 model
§ Short developing period§ Strong FAE support with completed documents§ Sample code supply for application support§ Strategy alliance with Game/Middleware/LCD/CMOS/BB vendors to give a total
solution choice
§ Small power-consumption system solution§ With power-saving design and process (~85% less power than one processor solution)§ The most advanced process support by UMC group
§ Follow main stream standard§ Compatible with most of I/F of BB, LCD, and CMOS/CCD sensor§ Follow 3GPP video stream standard§ Follow Java & OpenGL_ES graphics standard
Best Solution (2)§ Fancy features§ Full 2D & 3D graphic acceleration§ Newest MPEG4 (up to CIF 30fps), JPEG codec for imaging & video application§ Fancy imaging processing of digital camera – 16X Continuous Zoom, Flash
synchronization control, Emboss/Sepia/Mono/Negative/Color-Filer effect
§ Best performance§ Best performance of 2D/3D (80X than strong-ARM simulation)§ Full hardwired architecture make the most efficient imaging processing§ Full hardware MPEG4 acceleration make the fastest video compression
§ Flexible architecture§ Support most of CMOS sensor w/ programmable compensation àYUV & Bayer
Pattern input§ Support most of TFT LCD à CPU I/F, RGB I/F§ Brightness Statistics Table output to reserve the flexibility for manufacturers to
make own AE/AF…§ Support the rotation/mirror of Camera & LCD & 2D object to make your design
more varied§ Support extra Java H/W accelerator SIP solution
Strategy Partner
GlamocoProcessor
Driver
SMedia API
CMOS/CCD
S companyO companyB companyP companyA companyM(F) company Baseband
I companyM companyA company
LCD
S companyS2 companyC companyT companyA companyS3 company
SRAM
S companyN companyS2 company
Graphics AP
U company
M companyT company
OS
GameG companyB companyI company
I companyH companyX companyF companyS company
Middleware & authoring tool
Java accelerator
N company
Feature List (1)§ Host Interface & Clock
§ Support 16bits SRAM-like bus§ Support most of B.B./CPU chip (16 bit 80/68 series)§ Input Clock: 32.768KHz or 13MHz
§ JPEG Engine § Fully Compliant with JPEG ISO/IEC 10918 Baseline standard§ JPEG encoding and decoding
§ Display Interface§ Display Resolution: Max. 512x512 75Hz 16/18bpp (RGB565/RB666)§ Support CPU I/F, RGB I/F, LCD I/F§ Support C-STN and TFT of most of LCD module venders§ Display 90o/180o/270o Rotate§ Support 8/9/16/18 bit Output Interface§ Gamma correction§ Partial Display at Power Saving Mode. § (LCM should support)
Feature List (2)
§ MPEG-4 Engine§ Full compliance with ISO/IEC 1446-2 (MPEG4) video simple profile level 0,1,2,3§ Full H/W acceleration § MPEG4/H.263 Encoder and Decoder§ CIF/QCIF 30 FPS§ Provide audio and video synchronization scheme
§ High Performance 2D Accelerator§ Full compliant with J2ME MIDP 2.0 2D requirement§ Built-in an 1T pipelined 16bit BitBlt graphics engine
§ ROP3, Rectangle fill, Font expansion, Line-drawing§ Transparent BitBlt with source and destination keys§ Alpha blended Bitblt, Clipping
§ Support maximum fill rate: minimum 33M pixels per second
Feature List (3)
§ High Performance 3D Accelerator§ Full compliance with OpenGL ES v 1.0 and OpenGL ES v 1.1(proposal7)§ Full compliance with JSR-000184 Mobile 3D Graphics API for J2METM
§ Built-in pipelined 3D graphics engine § Nearest/Bi-linear Perspective Corrected Texture Mapping§ Texture transparency, blending, wrapping, mirroring and clamping§ 1/2/4/8/16-bit texture format§ Support palette texture and compressed texture§ 16-bit Z-buffer, triangle setup engine § Fog, alpha blending and specula effects§ Alpha blended Bitblt, Clipping, dithering
§ Extra 8 lighting support§ Support maximum fill rate: 33M pixels per second§ Support polygon rate: 150~1280K polygons per second
Feature List (4)§ Video Interface
§ Max. Image Resolution : 3 M§ Sensor Interface supports : programmable image size
§ up to 3 Million-pixel CCD/CMOS sensors.
§ Supports Input Formats§ Bayer (Raw Data)§ YUV 4:2:2
§ Gamma Correction by programmable look-up table.§ Lens Shading Correction.§ Bad Pixel Compensation.§ Black Level Compensation.§ Support Flashlight function.§ Preview/MPEG Frame Rate Control.§ Independent Size Scale-down of Vertical or Horizontal direction.§ Support 180o Rotate & Mirror
§ Software Support§ Nucleus, Smartphone, Linux, Symbian
§ Process and Package§ UMC 0.15um low power process§ 3.3~1.8V I/O VDD and 1.5V core VDD supply§ BGA 8mm x 8mm 140 ball 0.6mm ball-pitch
S/W Architecture
Game-related module
Hardware (Processor + Glamo coProcessor)
RTOS Core SMedia Driver SuitSMedia Driver Suit
OS-related Graphics API SMedia 2D/3D API (OpenGL ES)
KVM (CLDC 1.1)
MIDP 2.0 JSR184
3D Graphics Middleware
2D/3D Native Application (C/C++)2D/3D Midlet
First customer sample- 3360 8/E/2004
Nucleus driver 9/M/2004
Application-note release 9/M/2004
EV kit (1) 9/M/2004
SDK (Drivers, APIs, Tools, Sample programs) 9/E/2004
EV kit (2) 10/E/2004
Smartphone 2003 driver (1) 10/E/2004
Smartphone 2003 driver (2) 11/E/2004
Linux driver 12/M/2004
Mass production – 3360 12/M/2004
Symbian driver 2/E/2005
Glamo 3360 Schedule
3360J : CS: 12/E MP: 2/E, 20053260 : CS: 12/M MP: 1/E, 20056330 : CS: 12/E MP: 2/M, 2005