© 2014 NTT DOCOMO, INC. All rights reserved.

How Changing Mobile and Media Technologies is Changing The Way We Create Innovations

Mick Etoh

@mickbean

NTT DOCOMO

1

© 2014 NTT DOCOMO, INC. All rights reserved.

Outline

Big-Data, Big-Server, and Fat-Pipe Solutions

Large-scale Signals from Wireless Infrastructure

APIs for Federated Systems

Definition of Innovation

Progress of Cellphone and Network Technologies

2

© 2014 NTT DOCOMO, INC. All rights reserved.

Who am I?Digital VCR Development (1985)

Artificial Intellligence (1988)

Computer Vision and PR (1991)

Image Coding (1994)

Mobile Multimedia (2000)

Data Mining and NLP (2007-)Speech Recognition(2006) Machine Translation(2014-)

Mobile Network (2002-)

Investment(2012-)

3

© 2014 NTT DOCOMO, INC. All rights reserved.

Connecting the dots….

4

Those experiences gave me the ability of

© 2014 NTT DOCOMO, INC. All rights reserved.5

Those experiences gave me the ability of

Connecting the technologies….

© 2013 NTT DOCOMO, INC. All rights reserved.

What is Innovation?

Joseph Schumpeter‘s “neuer Kombinationen” (Theorie der wirtschaftlichen Entwicklung)

6

© 2014 NTT DOCOMO, INC. All rights reserved.7

“All technologies are combinations of technologies that already exist.” —

• Combinatorial Explosion　 • Creates Economy

W. Brian Arthur

Innovation Example

2.5G Internet connectivity, DOCOMO

1998

Micro Browser, Access Co.

1995

i-mode, DOCOMO

1999New Biz. Concept:

Content Provider Bazar

8© 2014 NTT DOCOMO, INC. All rights reserved.

Wireless Technologies, the EnablersPersonal Storage SNS/Comm. Tools

LTE:～75Mbps2010

+Big data synchronization

Application Market (games, net contents, etc.)

3.5G: ～14Mbps 2006

+Smart Phone + Cloud

Mobile Multimedia

3G: 64-384Kbps 2001+CODEC

i-mode

2G:9.6KbpsX3 1998 +micro browser

9© 2014 NTT DOCOMO, INC. All rights reserved.

© 2013 NTT DOCOMO, INC. All rights reserved.

Very Simplified Formula

Wireless Broadband

Innovation X?

10

❌

© 2014 NTT DOCOMO, INC. All rights reserved.

Distributed Speech Recognition(DSR) in 2006

First Implementation of ETSI AURORA Project (2000-2003)16 Khz Sampling MFCC feature vectors --> 5.6 Kbps

Mobile Terminal

Feature Extraction

Result

Encode

Server

Decode Rec. Engine

Dictionary

11

© 2014 NTT DOCOMO, INC. All rights reserved.

Lessons Learned From the DSR(2006)

12

　Pros• 16Khz sampling outperforms 8Khz Sampling.• Dedicated Feature Extraction Front End outperforms

the legacy IVR (Interactive Voice Response) methods.

Cons• Channel assumption was too narrow (i.e., Kbps-order).• No background noise model (left intact)

Bandwidth is not crucial.

Mobile Issue: Battery

13© 2014 NTT DOCOMO, INC. All rights reserved.

3X energy density evolution

40MHz ARM(2001) Mbps

Broadband

Quad-Core 1GHz+

ARM(2014)

250X Improved

+

© 2014 NTT DOCOMO, INC. All rights reserved.

Thin Client Architecture with Fat Pipe and Big Server since 2007

14

© 2014 NTT DOCOMO, INC. All rights reserved.

LTE-Advanced  Network　（２０１６-）Macro  Cell  Base  Station

Small  Cell  Base  Station

Centeraized  Radio  Access  Network  

15

© 2014 NTT DOCOMO, INC. All rights reserved.

Evolution of Wireless Technologies

16

10  kbps

100  kbps

1  Mbps

10  Mbps

100  Mbps

1  Gbps

10  Gbps

2020

5G

2015201020052000

LTE

LTE-­‐A

HSDPA,  HDRWCDMA,  

CDMA2000

MIMO

Carrier

 

Aggreg

ation/

MU-­‐MI

MO

3D/Ma

ssive  

MIMO

～3.5GHz

～60GHz

CDMA

OFDM/

OFDMA

Small  C

ell

© 2014 NTT DOCOMO, INC. All rights reserved.

Smart Phones as Information Hubs

WiFi

NFC

Bluetooth

Gyro

Microphone

CCD/CMOS Camera

GPS

Accelerometer

Cloud

Wireless Broadband

Smart Phone

17

© 2014 NTT DOCOMO, INC. All rights reserved.

Camera+Snapdragon+Vuforia(FastCV)+Unity

18

Ball-in-a-maze puzzle

See youtube video.

© 2014 NTT DOCOMO, INC. All rights reserved.19

https://play.google.com/store/apps/

details?id=jp.co.sompo_japan.safetysight

iOnRoad

Safety Sight(2014)

http://www.ionroad.com/

© 2014 NTT DOCOMO, INC. All rights reserved.

History of Cellphone Sensors

20

Barometric

Magnetometer

AccelerometerFingerprint

Co-Processors

GPS

2005 2010 2015

Navigation

Power reduction

biometrics

Navigation Compass

altimeter, weather forecast

pedometer, game controler

Mar/03

July/03

April/04

July/06

June/12

May/13

Gyroscope Image StabilizationNov./09

© 2014 NTT DOCOMO, INC. All rights reserved.

Coprocessor solution (iphone 5s)

21

A6 Magnetometer

Accelerometer

Gyroscope

A7 M7 Magnetometer

Accelerometer

Gyroscope

© 2014 NTT DOCOMO, INC. All rights reserved.

Sensor Fusion

22

3 axis gyroscope 2000°/second precision

3 axis accelerometer +-50mg precision

3 axis magnetometer 6° precision

Pressure sensor +/- 10 Pa precision

Ultrasound sensors for ground altitude measurement

60 fps vertical QVGA camera for ground speed measurement

http://ardrone2.parrot.com/ardrone-2/specifications/

© 2014 NTT DOCOMO, INC. All rights reserved.

Flying Cell Phone: Androcopter

23http://diydrones.com/profiles/blogs/androcopter-use-you-smartphone-as-an-autopilot

© 2014 NTT DOCOMO, INC. All rights reserved.

Siri24

Big-Data, Big-Server, and Fat-Pipe Solution

© 2014 NTT DOCOMO, INC. All rights reserved.

“Shabette-Concier” Voice agent

• Launched Mar. 1, 2012

• Over 40 services in it

• Including chatting

• 10 million users

ShabetteVoice

= ConcierConcierge

=How may I help you?

25

© 2014 NTT DOCOMO, INC. All rights reserved.Touch the Concier.Q: “What is the height of Mt. Fuji?”A: “3,766m!”Q: “When is holding schedule of the Tokyo Olympic Games?”A: “It will hold in 2020.” 26

© 2014 NTT DOCOMO, INC. All rights reserved.

Basic Architecture 2010

Logging

VoiceRecognition

TaskRecognition

Logging

Voicetext text contents

Service Providers’ DB

contents

text

Text to speech

27

Fat-Pipe

Big-Servers

© 2014 NTT DOCOMO, INC. All rights reserved.

BELIEF: More data usually beats better algorithms.

Speech

Speech Data

Language Data

Machine Learning Machine Learning

Acoustic Model Lexicon Language

Model

Speech Recognition I want...

Text

Voice Recognition

28

Big-Data

Performance Improvement by Data

Prec

isio

n (%

)

40

50

60

70

80

90

Text Data Amount in Million Lines1 10 100 1000

29© 2014 NTT DOCOMO, INC. All rights reserved.

© 2014 NTT DOCOMO, INC. All rights reserved.

Virtuous Circle Enabled by Machine Learning

Improved Performance

Better User Experience More Data

Machine Learning

30

Signal Processing in　Mobile Comm.Signals From Each Mobile Device about

audio, image, video, and extra sensors for Applications: Information Retrieval, Navigation, etc.

31© 2014 NTT DOCOMO, INC. All rights reserved.

Signals From Gigantic Social Infrastructure about crowds, traffic, earthquake, and weather for

Applications: Large-scale Infrastructure Optimization and Human Security

© 2014 NTT DOCOMO, INC. All rights reserved.

Earthquake Early Warning System(2007)

32

Mobile Telecommunication System

TV Broadcasting

System

© 2014 NTT DOCOMO, INC. All rights reserved.

Earthquake Early Warning System

33

© 2014 NTT DOCOMO, INC. All rights reserved.

PM 14:46, March 11th, 2011

34http://pics.livejournal.com/druzifer/pic/004q2qar

Great Eastern Japan Earthquake

© 2014 NTT DOCOMO, INC. All rights reserved.35

Connecting the technologies

through APIs…

© 2014 NTT DOCOMO, INC. All rights reserved.36

+ Character Recognition,Environment Sensor,Data Storage BOX,Location Information, Mobile Payment, and more.

API as Catalysts

© 2014 NTT DOCOMO, INC. All rights reserved.

Yet another Google Goggle/Amazon Flow

37https://twitter.com/ksasao

© 2014 NTT DOCOMO, INC. All rights reserved.38

API  AggregatorMashape  is  the  largest  API  marketplace,  powering  thousands  of  APIs  and  Applications.  

© 2014 NTT DOCOMO, INC. All rights reserved.39

Rank.25

Rank.34

Rank.39

Rank.43

Rank.16 ICCV2013

Rank.6 Googleglass

© 2014 NTT DOCOMO, INC. All rights reserved.40

Rank.51

Rank.57

Rank.62

Rank.77

Rank.85

Rank.87

© 2014 NTT DOCOMO, INC. All rights reserved.

Take aways for “connecting the dots.”

41

Wireless Broadband

Innovation X?❌

Big-Server, Big-Data, and Fat-Pipe for Smart Thin clients

Virtuous Circle by Machine Learning

Large-scale Signals from Wireless Infrastructure

APIs as Connecters and Indicators of your inventions.

© 2014 NTT DOCOMO, INC. All rights reserved.42

Design of combination

from existing and

foreseeable

technologies

Research for Impact factor

© 2014 NTT DOCOMO, INC. All rights reserved.

Signals from GPS around Tokyo in a normal and non-eventful day. 43

My dream is …to connect the systems.

Innovation X

Speech Dialogue System

Computer Vision System

unknown?