INTERACT! with the Flash Platform

SESSION overview

  our DO’S and DONT’S

  interaction with   Sound   Camera   Mobile /Other devices

  Q and hopefully A

  DEMO’s

our DO’S and DONT’S

Wouter Verweirder Belgian Citizen, Lord of Sealand @wouter

our DO’S and DONT’S

  I DO:   Spend a yearly budget of € 200

to lightsaber apps, noble titles, …   Teach hardcore AS3 at HOWEST   Am a Happy Banana   Cheat on Flash Games

  I DON’T   Use the lightsaber apps or noble

titles.

our DO’S and DONT’S

@newmovieclip

our DO’S and DONT’S

  I DO:   Lecture   Write   Organise   Like Happy-Banana!

  I DON’T   Do sports, eat healthy and have a

girlfriend

MAKE SOME NOISE !!

INTERACT! with sound

INTERACT! with sound

  flash.media.sound

  More control about input:   flash.media.Microphone   flash.media.SoundChannel

  More control about output:   flash.media.SoundTransform

INTERACT! with sound

  Microphone   Static getMicrophone() method

  setLoopback() + setUseEchoSurpression()

  Microphone.activityLevel (0-100)

  Capturing Microphone   SampleDataEvent dispatched   Write to ByteArray

INTERACT! with sound

  SampleDataEvent   Contains sample data: event.data

  = ByteArray instance   readFloat() method avances through

available bytes 4 at a time.

9   10   11   12   13   14   15   16   17   18  

readFloat()

soundBytes

next starting point

INTERACT! with sound

  Microphone.rate determines how many samples are used every time the SampleDataHandler is called.

  rate: 44 44,100 khz 8192 samples   rate: 22 22,050 kHz 4096 samples   rate: 11 11,025 kHz 2048 samples

INTERACT! with sound Capture the samples and save in ByteArray:

Create new Sound instance

INTERACT! with sound Fill the Sound with the captured soundBytes

DEMO

INTERACT! with Camera

INTERACT! with Camera

  flash.media.Camera

  Camera.getCamera();

  Video.attachCamera(camera)

  Interactions:   activityLevel + motionLevel motion

detected   Pixel changes

INTERACT! with Camera

  Track pixel changes

  Draw video into BitmapData instance   Use Difference blendmode   Set threshold and give pixels color   Check for color change

INTERACT! with Camera

  flash.media.CameraRoll   CameraRoll.browseForImage()   CameraRoll.addBitmapData()

  Check CameraRoll.supportsAddBitmapData property

  flash.media.CameraUI   CameraUI.launch(MediaType.IMAGE);   CameraUI.launch(MediaType.VIDEO);

  Listen for Event.SELECT, Event.CANCEL   Event.data = MediaPromise instance   Use Loader.loadFilePromise() to load media

promise

INTERACT! with Camera

  flash.media.MediaPromise   = promise to deliver a media object (image or

video)   Access to the actual file: use the File Property

  Eg: promise.file.url, promise.file.nativePath,...

DEMO

INTERACT! with Touch

INTERACT! with Touch

  Native MouseEvents also supported.

  flash.ui.Multitouch   TouchEvent   GestureEvent

  Plan:   Step 1: Set the inputMode   Step 2: Check if MultiTouch is supported   Step 3: Add listeners for user interaction   Step 4: Write handler methods to react on UI

INTERACT! with Touch

  TouchEvent   COOL: pressure property: between 0 and 1   TOUCH_BEGIN, TOUCH_END,

TOUCH_TAP,...

  GestureEvent / TransformGestureEvent   More complex user contact   GESTURE_TWO_FINGER_TAP   ROTATE, SWIPE, ZOOM,...

INTERACT! with Touch

  Multitouch.supportsTouchEvents

  Multitouch.maxTouchPoints

  TouchEvent has property touchPointID

  Dragging : startTouchDrag, stopTouchDrag

DEMO

INTERACT! with Devices

INTERACT! with Devices

  Use devices capabilities als input   Adobe AIR for Android   Adobe AIR for IOS

  flash.sensors.Accelerometer   Using the motion sensor of Device

INTERACT! with Devices

  AccelerometerEvent contains data   AccelerationEvent.accelerationX   AccelerationEvent.accelerationY   AccelerationEvent.accelerationZ

  Send the data to a Server   UDP would be nice…   Flash.net.Socket (TCP/IP)   Create Socket Server with Adobe AIR

INTERACT! with Devices

  CLIENT applications: Socket class

  CLIENT sents commands to the SocketServer

INTERACT! with Devices

  ServerSocket: listen for connections on port

  When client is connected: SOCKET_DATA

INTERACT! with Devices

  Handle the incoming data

DEMO

INTERACT! with Kinect

  I can see, in 3D!

  Kinect data to flash: Sockets

INTERACT! with Kinect

  AS3Kinect – Camera Images   http://www.as3kinect.org/

  TUIOKinect – Blob cursor tracking   https://code.google.com/p/tuiokinect/   http://bubblebird.at/tuioflash/tuio-as3-library/   http://silviopaganini.posterous.com/

openkinect-flash-tuio-udp-flash-bridge

  OSCeleton – Skeleton Detection   https://github.com/Sensebloom/OSCeleton

INTERACT! with Kinect

  as3osceleton   Open-sourced today   Runs on top of OSCeleton & TUIOKinect   Translates joints into cursors using

TUIOKinect bridge

  initial setup

INTERACT! with Kinect

  Event Driven:   UserEvent.USER_JOINED   UserEvent.USER_LEFT   UserEvent.SKELETON_CREATED   JointChangedEvent.JOINT_CHANGED

  Translation of hands to multitouch cursors

LETS PLAY SOME MUSIC !

Questions and hopefully

Answers

THANKS !!

THANKS !! Presenta3on  /  

Source  Code  demos:  

http://factory.happy-

banana.be/ FITC2011