TouchDevelop Chapter 5-7Chapter 5 Audio

Chapter 6 Camera, Graphics and Video

Chapter 7 Sensors

Mengfei Ren

5 Audio

• 5-1 Music• 5-2 Radio• 5-3 Sounds• 5-4 Microphone• 5-5 Media Player on a Home Network

5 Audio

• A smartphone could play music such as MP3 files, audio streamed over the internet or media center

• All the media files could be synchronized to the phone using Zune, or purchased and downloaded via the Marketplace and the Internet

5-1 Music• The smartphone supports three music formats, AAC,

MP3, and WMA.• The TouchDevelop API provides the media resource

for retrieving collections of all the songs, albums, and playlists. However, these collections couldn’t changed by TouchDevelop. [1]

5-1 Methods for Songs, Albums and Playlists

[1]

5-1 Methods for Songs, Albums and Playlists(cont.)

[1]

5-1 Play the Songs• When there is new album or playlist sent to the

player, it creates a queue of songs to be played. The songs will be played in order that they appear in the album or playlist.

• Requesting to play a new song when there is a song playing, the current one will be terminated and the queue be cleared before the new one starts.

• Playing songs is a background event.[1]• Demo 5-1

5-1 Methods for Playing Songs

5-1 Methods for Playing Songs• The song’s volume ranges from 0.0 to 1.0. • The value of 0.0 means mute, however the

value of 1.0 does not mean the maximum which the phone is capable.

• The value of 1.0 is related to the current volume of the cell phone.

• If the phone is under mute mode, the speaker is set to off and there is no any audible sounds.

5-2 Radio• The radio plays as a background activity.• The radio will only stop if the method radio stop is called or if the user shuts off the radio externally to the script. [1] Demo 5-2

5-3 Sounds• The Sound datatype is used for audio

recordings in the WAV format.• The files in WAV format is commonly used

used for short sound clips (30 seconds or less).[1]– This format is commonly used for uncompressed

audio and therefore the files tend to be large. – No method is provided for stopping playback of

the clip before it is finished.

5-3 Methods for Sound Datatype

5-4 Microphone

• API for making a recoding

var snd := senses record microphone• The word “Recording…” and a stop button

will be displayed on the screen.

5-5 Media Player on a Home Network• A media center can transmit videos and music

onto the phone over a Wi-Fi network. The phone could be a remote control for the media player.

• TouchDevelop provides the Media Server datatype for a media center. The script could search and retrieve the songs and videos.

• To play a song from media server:

player play home media (mlink)• Demo 5-3

5-5 Methods for Media Server[1]

6 Camera, Graphics, Video

• 6-1 Camera• 6-2 Static Graphics Drawing and Display• 6-3 Playing Videos from the Internet• 6-4 Media Player

6-1 Camera• To create the picture, the cell phone should at least

have a camera on the back, which is the primary camera.

• To access the camera, TouchDevelop provides the senses service. Also the Camera datatype to retrieve information about the camera and take a quick low-quality picture.

• To show the picture on the wall, there are two methods.

6-1 Ways to Take Pictures• High-quality picture:

senses take camera picture

• Not immediately take the picture. It calls the phone’s built-in software for using the camera and displays a preview image on the screen. Take the picture until the button pressed.

• Low-quality picture:senses front camera preview

• Demo 6-1

[2]Demo 6-2

6-1 Picture Effects[2]

Example: Picture Overlaying

• Demo 6-6

6-2 Static Graphics Drawing

• TouchDevelop provides many methods for drawing lines and shapes.

• The create picture method is to create a new board to draw lines and shapes.

var pic := media create picture(400,200)• Demo 6-3

6-2 Drawing Methods Picture[2]

6-3 Playing Videos from the Internet

• TouchDevelop doesn’t provide access to video files held on the phone. Nor can a script download a video to the phone.

• A script can access and play videos which are streamed from the internet or a media server.[2]

• Demo 6-4

6-4 Media Player• A media center could send pictures to the

phone and stream videos to the phone.[2]

6-4 Media Player (cont.)

• Demo 6-5• TouchDevelop does not provide methods to

stop the video playing from within the script.• To stop the playing video,– Tap the screen and see the pause/continue button– Pause the video and tap the back button to stop the

execution in the script.

7 Sensors

• 7-1 The Sensors• 7-2 Sensor-Driven Events• 7-3 Accelerometer• 7-4 Compass• 7-5 Gyroscope• 7-6 Motion

7-1 Sensors

• GPS(Global Positioning System): obtains the phone’s current location on the map

• Accelerometer: measures gravitational and acceleration

• Compass: returns the direction of magnetic north

• Gyroscope: measures the phone’s orientation in 3D space [3]

7-1 Sensing Methods [3]

7-2 Sensor-Driven Events

• A script can use the event of effect an action, such as pausing the playback of an audio recoding.

• Demo 7-1 [3]

7-3 Accelerometer• The phone measures the force by the

accelerometer when it is shaken or moved, which combines acceleration with gravity.

• The acceleration is in any direction in three dimensional space. The TouchDevelop API use the Vector3 datatype to present the current force in three dimensions.

• Demo 7-2, 7-3

7-3 Accelerometer (cont.)• (0,0,-1) – the phone is

lying flat on the table• (0,-1,0) – the phone is

held vertically with its bottom edge on the table

• The TouchDevelop reports the force in g (gravitational) units.

• Any stationary object is subjected to a force of 1g in the downwards direction

[3]

7-4 Compass

• Report the phone’s orientation with respect to magnetic north.

• Demo 7-4

7-5 Gyroscope• This is a device which reports on whether the

phone is being rotated in any orientation.• Rotation is measured as an angular velocity,

which is rotation speed in degrees per second about an axis in three dimensional space.

• (a, b, c) – the combined speed is • Example: (360,0,0) – a rotational speed that the

X axis in a clockwise direction.

7-6 Motion• The TouchDevelop provides the Motion

datatype to present the current location which is measured by the combination of the various sensor components.

[3]

7-6 Motion (cont.)• The force caused by gravity and the additional

force caused by acceleration is separated. The method just returns a true acceleration without the gravitational force.

• The accurate value of the orientation can be obtained for the time when it was captured. The true orientation is known as its attitude that measured via three methods, pitch, roll and yaw.

7-6 Motion (cont.)• These methods is measured in degrees,

respectively relative to three orthogonal axes. • The axes are diagrammed in the plane.

References

[1] https://az31353.vo.msecnd.net/cpd/kyei-bookchapter5.pdf

[2] https://az31353.vo.msecnd.net/cpd/axwq-bookchapter6.pdf

[3] https://az31353.vo.msecnd.net/cpd/mxnl-bookchapter7.pdf

THANK YOU