Clementine Requirements Document - Drexel CCI · Project Clementine is an attempt to create a...

O~~ Requirements Document

1

Clementine Requirements DocumentProject Clementine Group

Table of Contents1. Introduction ............................................................................................................................................ 5

1.1. Prior Art ...................................................................................................................................... 61.1.1. HandVu ( http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html ) .................... 61.1.2. Sense8 - World Up™ ( http://www.sense8.com/ ) ........................................................... 6

2. Priority Scheme ...................................................................................................................................... 73. Functional Requirements ........................................................................................................................ 8

3.1. Library ......................................................................................................................................... 83.1.1. Programmer Interface ....................................................................................................... 8

3.1.1.1. Language ................................................................................................................ 83.1.1.2. Operating System .................................................................................................. 8

3.1.1.2.1. Linux ........................................................................................................... 83.1.1.2.2. Windows ..................................................................................................... 8

3.1.1.3. Functionality .......................................................................................................... 83.1.1.3.1. Gesture Specification .................................................................................. 8

3.1.2. Threading .......................................................................................................................... 83.1.2.1. Multi-threading in the library ................................................................................ 93.1.2.2. Thread priority ....................................................................................................... 93.1.2.3. Number of Threads ............................................................................................... 93.1.2.4. Compile time customization .................................................................................. 93.1.2.5. Pre-Compilation Directives ................................................................................... 9

3.1.3. Data Importing ................................................................................................................. 93.1.3.1. Input Frames .......................................................................................................... 93.1.3.2. Data History ........................................................................................................ 10

3.1.3.2.1. Data History Configuration ...................................................................... 103.1.3.2.2. Data History Access Time ........................................................................ 10

3.1.3.3. Data Streamline ................................................................................................... 103.1.3.4. Data Reading Rate ............................................................................................... 10

3.1.3.4.1. Wii Remote Data ...................................................................................... 103.1.3.4.2. Web Cam Data ......................................................................................... 10

3.1.4. Data Transformation ....................................................................................................... 113.1.4.1. Resolving Data Point Collision ........................................................................... 113.1.4.2. Finding Next Value ............................................................................................. 11

3.1.4.2.1. Value Update Distance Tolerance ............................................................ 113.1.4.2.2. Loss of Data Point .................................................................................... 113.1.4.2.3. Lost Point Prediction ................................................................................ 113.1.4.2.4. Lost Data Point Removal ......................................................................... 113.1.4.2.5. Waiting Queue .......................................................................................... 12

3.1.4.3. Data Point Priorities ............................................................................................ 123.1.4.4. Data Point Analysis ............................................................................................. 12


2

3.1.4.4.1. Linear Regression ..................................................................................... 123.1.4.4.2. Circular Regression .................................................................................. 12

3.1.4.5. Raw Data Storage ................................................................................................ 123.1.4.6. Dimensional Smoothing ...................................................................................... 123.1.4.7. Data Extrapolation ............................................................................................... 12

3.1.4.7.1. Future Computation Using Extrapolated Data .......................................... 133.1.4.8. Data Smoothing Example .................................................................................... 133.1.4.9. Derivatives ........................................................................................................... 14

3.1.5. Data Representation ........................................................................................................ 143.1.5.1. Reference Frames ................................................................................................ 14

3.1.5.1.1. Data Access .............................................................................................. 143.1.5.1.2. Data Conversion ....................................................................................... 14

3.1.5.2. Rotation ................................................................................................................ 143.1.5.2.1. Rotational Conversion .............................................................................. 14

3.1.6. Abstract Data Processing ............................................................................................... 143.1.6.1. Gestures ............................................................................................................... 15

3.1.6.1.1. Atomic Gestures ....................................................................................... 153.1.6.1.2. Complex Gestures ..................................................................................... 153.1.6.1.3. Custom Gestures ....................................................................................... 15

3.1.7. Data Exporting ............................................................................................................... 153.1.7.1. Single Point Exporting ........................................................................................ 15

3.1.7.1.1. Positional Data .......................................................................................... 163.1.7.1.2. Velocity Data ............................................................................................ 163.1.7.1.3. Acceleration Data ..................................................................................... 163.1.7.1.4. Gestures .................................................................................................... 16

3.1.7.2. Multi-Point Objects Exporting ............................................................................ 163.1.7.2.1. List of Positional Data .............................................................................. 163.1.7.2.2. Central Position Data ................................................................................ 163.1.7.2.3. List of Velocities Data ............................................................................. 163.1.7.2.4. Average Velocity Data ............................................................................. 163.1.7.2.5. List of Accelerations Data ........................................................................ 173.1.7.2.6. Average Acceleration Data ....................................................................... 173.1.7.2.7. Atomic Gestures ....................................................................................... 173.1.7.2.8. Complex Gestures ..................................................................................... 17

3.1.7.3. Data Export Rate ................................................................................................. 173.1.7.3.1. Guaranteed Updates .................................................................................. 173.1.7.3.2. Fastest Update .......................................................................................... 17

3.1.7.4. Multi-Tiered Exporting ........................................................................................ 173.1.7.5. Multiple Unique Devices ..................................................................................... 18

3.2. Interfaces ................................................................................................................................... 183.2.1. Spatial Tracking Interface .............................................................................................. 18

3.2.1.1. Data format .......................................................................................................... 183.2.1.1.1. Processing of Data .................................................................................... 183.2.1.1.2. Gesture Library ......................................................................................... 18

3.2.2. 3D Interface .................................................................................................................... 193.2.2.1. Initialize Function ................................................................................................ 19


3

3.2.2.2. Process Update .................................................................................................... 193.2.2.3. Compute 3D Position .......................................................................................... 193.2.2.4. Update 3D Position ............................................................................................. 203.2.2.5. Set Error Bounds ................................................................................................. 203.2.2.6. Hardware Device ................................................................................................. 20

3.3. Demos ........................................................................................................................................ 203.3.1. 3D Demo ........................................................................................................................ 20

3.3.1.1. Application .......................................................................................................... 203.3.1.1.1. Tennis ....................................................................................................... 203.3.1.1.2. Other External Libraries ........................................................................... 203.3.1.1.3. Avatar Control .......................................................................................... 21

3.3.2. Spatial Demo .................................................................................................................. 213.3.2.1. Application .......................................................................................................... 21

3.3.2.1.1. Blender ...................................................................................................... 213.3.2.1.2. Gestures .................................................................................................... 213.3.2.1.3. Vertex Selection ....................................................................................... 213.3.2.1.4. Vertex Merging ........................................................................................ 223.3.2.1.5. Face Creation ............................................................................................ 223.3.2.1.6. 3D Transformations .................................................................................. 223.3.2.1.7. 3D Manipulation ....................................................................................... 223.3.2.1.8. Menu Navigation ...................................................................................... 22

3.4. Diagnostic Utilities .................................................................................................................... 223.4.1. Real Time Graphs .......................................................................................................... 22

3.4.1.1. Raw Data ............................................................................................................. 223.4.1.2. Smoothed Data .................................................................................................... 233.4.1.3. Data Accuracy ..................................................................................................... 23

3.4.2. Real Time Representations ............................................................................................. 233.4.3. Analysis Data ................................................................................................................. 23

4. Non-Functional Requirements .............................................................................................................. 244.1. Web Cameras ............................................................................................................................ 24

4.1.1. Framerate ........................................................................................................................ 244.1.2. Resolution ....................................................................................................................... 244.1.3. Frustum Area .................................................................................................................. 24

4.2. Wii Remote ............................................................................................................................... 244.2.1. Communication with computer ...................................................................................... 244.2.2. IR Tracking .................................................................................................................... 24

4.2.2.1. IR Frustum ........................................................................................................... 254.3. Bluetooth ................................................................................................................................... 254.4. Peripherals ................................................................................................................................. 25

4.4.1. Gloves ............................................................................................................................. 254.4.1.1. Glove Dot Positions ............................................................................................ 25

4.4.2. Glasses ............................................................................................................................ 264.4.2.1. Emitters ................................................................................................................ 26

4.4.2.1.1. Emitter Locations ..................................................................................... 264.5. Third-Party Libraries ................................................................................................................. 26

4.5.1. Gesture Recognition Library .......................................................................................... 26


4

4.5.1.1. Inputs ................................................................................................................... 264.5.1.2. Outputs ................................................................................................................. 26

4.6. System Requirements ................................................................................................................ 274.6.1. Hardware ........................................................................................................................ 274.6.2. Language ........................................................................................................................ 27

5. Future Development ............................................................................................................................. 285.1. Application Environment .......................................................................................................... 28

5.1.1. Apple Macintosh ............................................................................................................ 285.1.2. Mobile Phones ................................................................................................................ 28

5.2. Input devices ............................................................................................................................. 285.3. Future Interfaces ........................................................................................................................ 28

5.3.1. GPS Interface ................................................................................................................. 285.3.2. Accelerometer Interface ................................................................................................. 29

5.3.2.1. Hardware Device ................................................................................................. 295.4. Demonstrations .......................................................................................................................... 29

5.4.1. Spatial Demos ................................................................................................................. 295.4.2. 3D Demos ....................................................................................................................... 29

5.5. Developer Expansions ............................................................................................................... 29Glossary .................................................................................................................................................... 31


5

1. IntroductionProject Clementine is an attempt to create a clean, stable programmer's library for use by developers intheir applications. When the library is used, it will allow developers to receive input from a variety ofunique, inexpensive input devices, and translate that input into meaningful commands for the application.The data will be customizable and available to the developer at extremely fast intervals (up to 30 updateevents per second).

Effectively, use of Project Clementine will allow for a basic virtual reality environment that is cheap andwidely available. For the purposes of this project, we define virtual reality as a method of interacting witha computer application with actual, physical actions, be it moving a single finger or the entire body.

The motivation for the project came from our witnessing of a need. Virtual reality is the next step inhuman computer interaction and design, and it has been for over 15 years. Unfortunately, all of the virtualreality peripherals on the market are extremely expensive and unwieldy. These large and expensivedevices are run by third party drivers, and only work in third party applications. This means that if anyonehas a great idea for an application using a new form of virtual reality-like input, they must find a companyin that field and sell their idea to them. These facts have held back the advancement of virtual realitygreatly. Our library would create an avenue through which developers could easily create virtual realityapplications using peripherals that users already have, or are very affordable. This would greatly expeditethe evolution of this exciting new field.

Our end goal is to make a service that is so simple to use that anyone with programming experience canexperiment with it and create fun and impressive applications using technology in ways never thought ofbefore.

This library is meant to be a layer that lies between an input device and the developer's application. At itsbase, it will smooth and interpolate the data coming from the device to a degree that can be specified bythe developer. In its most advanced form, it can take the data coming from the device and reorganize it inways never before seen by the low budget developer.

We will strive to make the input devices for our library as commonplace as possible. We hope to usedevices that are pervasive in not only the developer's environment, but also in the users' that will use thedeveloper's application.

To ensure the accuracy and correctness of our library, this requirements document also specifies theuse of demos that provide an easy way to demonstrate the functionality of the individual portions of thelibrary. The spatial interface, which can be activated with gloves on the users hands, is used to manipulateobjects in a three-dimensional modeling program in the Spatial Demo. The 3D interface is used in theTennis Demo, where a user can both move around the environment by physically moving, and hit tennisballs by swinging a Wii Remote.

Along with these demos, our library will provide a suite of diagnostic utilities to be used by developersin order to view the expected output of simulated input, such as gestures or accelerometers. Not onlywill this provide useful functionality to developers working with the library, it can also be used to easilydemonstrate the nuances of the ability of the library without a full-featured demo.


6

It is important to note that due to the nature of this project, a large portion of the requirements specifiedare not user-level, but programmer-level, and thus it is necessary to define certain programmatic elements(such as UML) in this requirements document.

1.1. Prior Art

Project Clementine aims to provide a well balanced product in relation to quality, functionality andease of use. Many other products offer similar or even superior services in one area but lack in others.

1.1.1. HandVu ( http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html )

The HandVu software system is a gesture recognition library that is available for Windows 2000/XP and Linux. It requires a camera with at least 320x240 resolution but the advantages of theHandVu system are that there is no other special equipment needed in order to capture input fromhand gestures.

However, this ability comes at the cost of flexibility for the application. It can only recognizegestures that are within 15 degrees CCW (and zero degrees CW) of a base orientation. Oursystem will allow for far more flexibility in the position of the hand by using tracking of threeindividual dots on the fingertips. This will also eliminate the need for a calibration stage that isrequired in the HandVu system in order for tracking to begin.

1.1.2. Sense8 - World Up™ ( http://www.sense8.com/ )

Sense8 ™ is a very large and robust product offering client/server solutions, toolkits and evendevelopment environment for 3d/virtual reality. Their latest product, World Up ™ , allows usersto easily create elaborate 3-D environments with minimal programming. It can accept 3-D modelscreated using a myriad of standard CAD applications.

With an estimated starting price of $3500, this development environment by Sense8 ™ is gearedtoward companies with dedicated programmers working to create a 3-D application. Whereas theWorld Up ™ environment provides the tools to create a 3-D world. Sense8's products are gearedtoward high end programmers working for professional companies using expensive hardware.Our library is meant for casual programmers developing custom application using cheap andcommonly avaliable input technology.

http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html

http://www.movesinstitute.org/~kolsch/HandVu/HandVu.html

http://www.sense8.com/


7

2. Priority SchemeID Type Description

1 Critical This is a necessity, multiple important things depend on it,and thus will be implemented first. This priority is referredto informally throughout documentation by use of thedeclaration MUST.

2 ShouldHave

This is more necessary than anything lower than it, buteverything does not depend on it. This priority is referredto informally throughout documentation by use of thedeclaration SHOULD.

3 Like tohave

This is something we would really like to have in our project,but is not necessary for its completion. This priority isreferred to informally throughout documentation by use of thedeclaration MIGHT.

4 If timepermits

This is something that is not necessary, nor do we plan onimplementing it. Only if everything else is perfect will this beimplemented.


8

3. Functional Requirements

3.1. Library

3.1.1. Programmer InterfacePriority: 1

A developer must be able to extend and instantiate classes in our library.

3.1.1.1. LanguagePriority: 1

The developer must write their application in C++.

3.1.1.2. Operating System

3.1.1.2.1. Linux

Priority: 1

The library must be compiled in such a way that applications compiled on Linux mayuse it. The library will be compiled into the .so format. This is a static library that canbe directly included into their final executable.

3.1.1.2.2. Windows

Priority: 2

The library must be compiled in such a way that applications compiled on WindowsXP and above may use it. The library will be compiled into the NT-kernel based .dllformat. This is a dynamically loaded library that must be available to the application atruntime.

3.1.1.3. Functionality

3.1.1.3.1. Gesture Specification

Priority: 1

The library must provide an interface to developers that allows for the creation andspecification of gestures used in interfaces such as the Spatial Interface.

3.1.2. ThreadingPriority: 2

Our library should work in a separate thread from the developer's application. This will keep ourapplication from interfering with the normal operation of the developer's application.


9

3.1.2.1. Multi-threading in the libraryPriority: 2

The library should run in multiple threads. Many portions of our library are serializable andtherefor highly threadable. The following processes should be capable of running in multiplethreads.

• Finding the next value of a data point

• Data Point Analysis

• Providing Data To Subscribers

3.1.2.2. Thread priorityPriority: 3

The priority of the threads in the library should be set lower than that of the developer'sapplication.

3.1.2.3. Number of ThreadsPriority: 3

The number of threads should be scalable to the number of processors in the machine.

3.1.2.4. Compile time customizationPriority: 2

The maximum number of threads should be customizable by the developer at compile time.

3.1.2.5. Pre-Compilation DirectivesPriority: 3

Precompilation directives should be placed in the code to remove threading totally based ona preprocessor variable.

3.1.3. Data ImportingPriority: 1

Data will be read from multiple devices and converted to a standard format so all input devicesare viewed the same.

3.1.3.1. Input FramesPriority: 1

Input from a device will be converted to an input frame. An input frame will have a series ofXY position values, or input points.


10

3.1.3.2. Data HistoryPriority: 1

A history of the data read will be stored for use in extrapolation and smoothing. The amountof back data stored will be specific to rate of input from the device, the higher the rate, themore input. This would yield a constant time in seconds back stored.

3.1.3.2.1. Data History Configuration

Priority: 2

The developer will be able to configure how far back in time, in seconds, to storehistory data.

3.1.3.2.2. Data History Access Time

Priority: 1

History data must be accessible in constant time, pushing a new piece of history dataon to the front must also be constant, and must push the oldest history data value out ofmemory.

3.1.3.3. Data StreamlinePriority: 1

Many of the devices will provide data that the library will not use, so only the necessary datawill be parsed and stored.

3.1.3.4. Data Reading RatePriority: 1

All types of input must have a specified rate of input, or input frames, but they do not haveto always meet their rate. In cases where the rate is not met, data extrapolation will occur tofill gaps.

3.1.3.4.1. Wii Remote Data

Priority: 1

An input point in a Wii remote is defined by the infrared dot data coming from the WiiRemote's data stream. (see Wii Remote Input)

3.1.3.4.2. Web Cam Data

Priority: 1

An input point in a webcam will be defined by a user. (see Web Cameras) Thedeveloper must be able to configure the following properties of dots to be interpreted asinput points

• Dot color (the color of blobs to look for to consider dots)


11

• Dot size (the approximate size of blobs, if within tolerance it is considered a dot)

• Dot size tolerance

• Maximum number of dots

3.1.4. Data TransformationPriority: 1

The data, once aggregated into a common format, will be tracked and transformed.

3.1.4.1. Resolving Data Point CollisionPriority: 2

When two data points overlap, their velocity will be used to determine which data point iswhich after collision.

3.1.4.2. Finding Next ValuePriority: 1

The input frame will only have a series of data points. To determine which data point inthe input frame is the next value of the current data point, the predicted value of the currentworking data point will be compared to the input point. The closest one within a tolerancewill be the next location of the data point.

3.1.4.2.1. Value Update Distance Tolerance

Priority: 2

The update distance tolerance will be a value affected by the velocity of the data pointprior to this reading. The higher the velocity, the higher the tolerance. This will allowfor greater user error acceptance.

3.1.4.2.2. Loss of Data Point

Priority: 1

A data point is lost if no input point values satisfy the update distance tolerance.

3.1.4.2.3. Lost Point Prediction

Priority: 1

To maintain smoothness in a noisy environment, a lost data point will be allowedmultiple update events before it is removed. During this time, the subscribers will beupdated with its extrapolated position.

3.1.4.2.4. Lost Data Point Removal

Priority: 1

When a data point exceeds its time out limit, it will be entered into the waiting queue.


12

3.1.4.2.5. Waiting Queue

Priority: 2

When a data point is lost, it will be entered into a queue. This queue will be sorted bythe data point priorities, then by amount of time out. It will be a stack, meaning themost recent point lost is the first point to be regained.

3.1.4.3. Data Point PrioritiesPriority: 2

Data points will have priorities. Input points will be assigned to data points depending onthese priorities. If no existing data point claims an input point (see Finding Next Value), thenthe unclaimed input point will be assigned to the highest priority waiting data points (seeLost Data Point Removal).

3.1.4.4. Data Point AnalysisPriority: 1

Regression will be used to analyze data points. Multiple types of regression will beavailable. The type of regression that has the smallest variance will be the one chosen to beused by the data point for smoothing and extrapolation.

3.1.4.4.1. Linear Regression

Priority: 1

Linear regression will be available for data point interpretation.

3.1.4.4.2. Circular Regression

Priority: 1

Circular regression will be available for data point interpretation.

3.1.4.5. Raw Data StoragePriority: 1

Raw input data must be stored along with the smoothed and extrapolated data for use infuture computation.

3.1.4.6. Dimensional SmoothingPriority: 1

Smoothing will be done on a single dimensional basis, smoothing in the X dimension of adata point will be independent of the smoothing in the Y dimension of the data point.

3.1.4.7. Data ExtrapolationPriority: 1


13

Extrapolation will be used to predict the next location of a data point. Regression will beused to calculate the values.

3.1.4.7.1. Future Computation Using Extrapolated Data

Priority: 2

Future computation of smoothing and extrapolation will ignore extrapolated data anduse unsmoothed raw data.

3.1.4.8. Data Smoothing Example

The following is a visual example of what smoothing and extrapolation predicting mightlook like.

The graph is a single reading's single dimension, for example, the X value for one dot overtime.

The blue dots represent accepted data_point. The green line represents the line of best fit forthe dots at the time they were read, assume for simplicity's sake that the line of best fit wasthe same for all dots.

The blue lines represent error bounds on either side of the line of best fit.

The red dot represents an input point that was not accepted, due to lying outside of the errorbound.

The yellow dot represents a corrected data point for the error red input point.

The green dots exist where no information from the input device was available. Thus, valueswere predicted using the line of best fit at those times.


14

3.1.4.9. DerivativesPriority: 3

Derivatives of the data will be available up to the second derivative, calculated using theregression type used in data smoothing and data extrapolation.

3.1.5. Data RepresentationPriority: 1

Data from the interfaces will be stored in memory during the library's runtime period. The exactmethod of storage is not within the scope of this document.

3.1.5.1. Reference FramesPriority: 2

Our library must have a method of storing reference frames.

3.1.5.1.1. Data Access

Priority: 2

All data will be tied to the reference frames in which it was observed. A developer maypass a reference frames they wish to view the data in.

3.1.5.1.2. Data Conversion

Priority: 2

The developer will be able to convert data from one reference frames to another usingone function call.

3.1.5.2. RotationPriority: 1

Any rotational data will be represented in quaternions.

3.1.5.2.1. Rotational Conversion

Priority: 2

Conversion between different rotation representations (quaternion, angle axis, Yaw-Pitch-Roll) must be done using one function call.

3.1.6. Abstract Data ProcessingPriority: 1

Abstract data processing will consist of defining gestures using dot tracking that can beinterpreted by the application.


15

3.1.6.1. GesturesPriority: 1

Gestures allow the user to provide information to the subscriber through the use of gloves.Gestures will be organized into two groups: atomic and complex. (see Gesture Specification

3.1.6.1.1. Atomic Gestures

Priority: 1

Atomic gestures must be configurable and detectable. Examples include the movementof a single finger forward and back, or up and down.

3.1.6.1.2. Complex Gestures

Priority: 2

Complex gestures are combinations of simple gestures. A complex gesture requires allsimple gestures to be activated relative a certain amount of time and distance from eachother. For example, if one simple gesture is an index finger moving up and down, andanother simple gesture is a middle finger moving up and down, a complex gesture maybe made that combines both the index and middle finger moving in tandem.

3.1.6.1.3. Custom Gestures

Priority: 3

The library will allow for the developer to create custom gestures at compile time.

3.1.7. Data ExportingPriority: 1

Depending on the interface in use, only a subset of data types will be available for export from thelibrary.

• Single Points 2D

• Single Points 3D

• Multi-Point Objects 2D

• Multi-Point Objects 3D

3.1.7.1. Single Point ExportingPriority: 1

A single points is one data point At compile time, the developer will decide which of thefollowing will be supplied, if they apply to the devices in use.


16

3.1.7.1.1. Positional Data

Priority: 1

Positional data must be made available.

3.1.7.1.2. Velocity Data

Priority: 1

Velocity data must be available.

3.1.7.1.3. Acceleration Data

Priority: 1

Acceleration data must be available.

3.1.7.1.4. Gestures

Priority: 1

Gestures must be made available.

3.1.7.2. Multi-Point Objects ExportingPriority: 1

The developer will decide which of the following will be supplied, if they apply to thedevices in use.

3.1.7.2.1. List of Positional Data

Priority: 1

A list of positional data.

3.1.7.2.2. Central Position Data

Priority: 1

The central position of the list of positional data.

3.1.7.2.3. List of Velocities Data

Priority: 1

A list of velocities parallel to the list of positional data.

3.1.7.2.4. Average Velocity Data

Priority: 1

The average velocity of the list of velocities data.


17

3.1.7.2.5. List of Accelerations Data

Priority: 1

A list of accelerations parallel to the list of positional data.

3.1.7.2.6. Average Acceleration Data

Priority: 1

The average acceleration of the list of positional data.

3.1.7.2.7. Atomic Gestures

Priority: 1

The indiviual atomic gestures loaded into the interface.

3.1.7.2.8. Complex Gestures

Priority: 1

The complex gestures loaded into the interface.

3.1.7.3. Data Export RatePriority: 1

The library will update all subscribers at a rate configurable by the developer. This rate willbe met as often as possible.

3.1.7.3.1. Guaranteed Updates

Priority: 2

Each subscriber must keep track of its most recent update interval. If one or multipleupdate events are missed for a specific subscriber, then that subscriber will be updatedmultiple times on the next pass through. This guarantees a number of calls per second.Each one of the library interfaces implements the Subscriber interface to retrievethese updates, and the developer implements that particular interface to begin recievingdata passed through it.

3.1.7.3.2. Fastest Update

Priority: 2

The library will be capable of providing updates as frequent as 30 times per second.

3.1.7.4. Multi-Tiered ExportingPriority: 3

Regardless of the data types available for export, they will all support multiple levels ofdetail in what may be exported. This support must be done in such a way that only if thedeveloper requests a higher level form of data, will its computation be performed.


18

All positional data, and their derivatives, will be provided in vector form.

3.1.7.5. Multiple Unique DevicesPriority: 2

Since the library will support multiple different devices to be used all at once, any dataexported back must uniquely be identified to its corresponding device.

3.2. Interfaces

3.2.1. Spatial Tracking InterfacePriority: 1

The spatial tracking interface (STI) of the Clementine library is an interface that allows the use ofpredefined and context-sensitive gestures to manipulate an application or operating environment.

3.2.1.1. Data formatPriority: 1

Data provided from the spatial interface will be in the form of gestures.

3.2.1.1.1. Processing of Data

Priority: 1

The interface must process the data in order to interpret it, accounting for relative size(and thus distance from the camera) and position of the relevant data points, beforerequesting a gesture from the gesture library.

3.2.1.1.2. Gesture Library

Priority: 1

Data Points will be passed to the gesture library and the most likely gesture will bereturned (see Non functional gesture library requirements).


19

3.2.2. 3D InterfacePriority: 1

The Three Dimensional Interface allows the use of the user's real-world position to manipulate anapplication or operating environment on a computer with the appropriate hardware.

3.2.2.1. Initialize FunctionPriority: 1

This call follows up the hierarchy into the library. It utilizes the InitializeDefaults()function to set up all of the default values for this instance of Interface3D.InitializeDefaults() is final and can not be overridden by the developer.

3.2.2.2. Process UpdatePriority: 1

ProcessUpdate() is called from higher up in the library. This is dependent on theupdateFrequency variable. If the amount of time specified by updateFrequency haslapsed, it will allow ProcessUpdate() to continue. This function takes in a scene and callsCompute3DPosition() on the left and right data points contained within it.

3.2.2.3. Compute 3D PositionPriority: 1

This function takes in new left and right data points and sends out a vector3 calculatedas a 3d position using the left and right points. This is passed into the abstractUpdate3DPosition() function.


20

3.2.2.4. Update 3D PositionPriority: 1

The implementation is not actually included in our library. Instead, it is implemented by thedeveloper. This is the main conduit through which the developer will receive 3D locationaldata from our library.

3.2.2.5. Set Error BoundsPriority: 2

Sets the boundaries for which data points are tolerated.

3.2.2.6. Hardware DevicePriority: 1

The hardware device for the use of the three dimensional interface must match therequirements as specified in the Wii Remote section of the non-functional requirements.

3.3. Demos

3.3.1. 3D DemoPriority: 1

The Three Dimensional demonstration application will show off the ability of the Clementinelibrary to map the player's real-world position to 3D coordinates in a piece of software.

3.3.1.1. ApplicationPriority: 1

The demonstration application for the 3D interface will need to be able to utilize theuser's real-world position. The position must be represented in a visually appealing three-dimensional rendering of a virtual environment.

3.3.1.1.1. Tennis

Priority: 1

In the demonstration, the player will be placed onto a tennis court and will have tointeract with tennis balls thrown in their direction with a swinging hand motion and 3Dmovement. This will be a demo that will require the use of a graphics library, a basicphysics or dynamics engine, and our 3D input library.

3.3.1.1.2. Other External Libraries

Priority: 1


21

The physics engine will take care of the tennis ball's movement, and the graphicslibrary will be taking that information and displaying it on screen in a visuallyappealing manner. The accelerometer within the Wii Remote will be used to capturethe player's arm movements and inform the game of those movements. This interactionwill be handled by the game itself in the event that the proposed accelerometerinterface within Clementine has not been implemented.

3.3.1.1.3. Avatar Control

Priority: 1

As the user moves around on the 2D plane of the ground below them, the player's onscreen avatar will then adjust according to those movements.

3.3.2. Spatial DemoPriority: 1

The purpose of the STI demo is to provide a working, interactive demonstration of the abilitiesand functionality of the Spatial Tracking Interface built into the library.

3.3.2.1. ApplicationPriority: 1

The application used in the STI demo must be one that allows for the functionality of theSTI to appear intuitive. Since the STI concerns itself primarily with the tracking of gesturesin three-dimensional space, the application will be one that manipulates elements in a three-dimensional (projected two-dimensional) view, and allows for various interactions withthose elements.

3.3.2.1.1. Blender

Priority: 1

In this demo, the application to interface with the library is the three-dimensionalmodeling program "Blender" ( http://www.blender.org/ ). It is, in the projects' ownwords, a "free open source 3D content creation suite, available for all major operatingsystems under the GNU General Public License."

3.3.2.1.2. Gestures

Priority: 1

The application must take input from the library as gestures from the Spatial Interfaceand manipulate the 3D modeling environment in Blender.

3.3.2.1.3. Vertex Selection

Priority: 1

The user must be to able to select and move individual vertices.

http://www.blender.org/


22

3.3.2.1.4. Vertex Merging

Priority: 2

The user must be to able to merge two selected vertices.

3.3.2.1.5. Face Creation

Priority: 1

The user must be to able to create a new face separate from the models being edited

3.3.2.1.6. 3D Transformations

Priority: 1

The user must be to able to select and manipulate a 3D object (translate rotate scale).

3.3.2.1.7. 3D Manipulation

Priority: 2

The user must be to able perform 3D manipulations of objects (extrusion, lathing).

3.3.2.1.8. Menu Navigation

Priority: 2

The user must be able to navigate through menus, and bring up context menus onselected items.

3.4. Diagnostic UtilitiesPriority: 2

Our library will supply diagnostic utilities to aid the user in visualization of the data being received andtransformed by the library. The purpose of the diagnostic utilities is twofold:

• Applications that show developers exactly how the library is responding to certain inputs will beinvaluable during the development process.

• Since the data is very visual, the diagnostic utilities help demonstrate the library and its functionalityin a way the layman can understand.

3.4.1. Real Time GraphsPriority: 2

Graphs of all data will be available. The user will be able to specify exactly what data they wantgraphed as well as what state to graph it in.

3.4.1.1. Raw DataPriority: 2


23

A graph of the raw data will only show the exact input points that are reported by the inputdevices, as they come in.

3.4.1.2. Smoothed DataPriority: 2

A graph of the data points that is smoothed and extrapolated with coloring similar to thegraph provided in the data smoothing example. This graph will also provide indicators ofregression lines and error bounds.

3.4.1.3. Data AccuracyPriority: 3

A graph plots the difference between input points and data points.

3.4.2. Real Time RepresentationsPriority: 3

A representation of the data points on the screen as they would appear to an application would beuseful as well. This utility will simply show the position of data points as white dots on a blackscreen.

3.4.3. Analysis DataPriority: 1

Anything pertaining to the analysis of the data coming in, this includes

• Type of regression used

• Gestures being considered


24

4. Non-Functional Requirements

4.1. Web CamerasConsumer-grade personal computer-compatible cameras, also known as web cameras, are inexpensiveand ubiquitous peripherals that must be supported by the library. They will be used for movementcapturing and computer vision of the user, to be fed into the Clementine library, and thus supported bythe application that utilizes the library.

4.1.1. Framerate

A web camera used with the Clementine library must have a video stream frame-rate of 30frames per second (FPS). Accordingly, programmers using the Clementine library should specifythis requirement to users of their software. The minimum frame-rate of the web camera doesnot guarantee that the library will be able to maintain this frame-rate in the interception andprocessing of data, simply that this frame-rate is required for smooth and effective performance ofthe library.

4.1.2. Resolution

A web camera used with the library must have a video resolution of at least 640 pixels wide by480 pixels high. Accordingly, programmers using the library should specify this requirement tousers of their software.

4.1.3. Frustum Area

A web camera used with the library must have a stable frustum area with a large enough area tosee objects from five to ten feet away. The frustum area of a camera is the trapezoidal solid thatbegins at the camera lens and expands outward to the edges of the web camera's reliable vision.

4.2. Wii RemotePriority: 1

Nintendo's Wii Remote will be used as an input device.

4.2.1. Communication with computerPriority: 1

The wii remote will connect differently than other input devices, it will communicate with thelibrary wirelessly via Bluetooth.

4.2.2. IR TrackingPriority: 1


25

The wii remote will be utilized for 3D tracking and accelerometer based motion tracking. The IRcamera built into the front of the unit will be used to track up to 4 dots and will provide the librarywith the raw data used in 3D tracking.

4.2.2.1. IR Frustum

Below is a rendering of what the frustum for the Wii Remote may look like.

4.3. Bluetooth

Bluetooth is the communication medium used by the Wii Remotes. The computers will either needbuilt-in Bluetooth chips or a separate USB dongle.

4.4. Peripherals

4.4.1. Gloves

Specialized gloves with colored dots on the fingers are used in conjunction with the spatialtracking interface.

4.4.1.1. Glove Dot Positions

Each finger will have a different color so that it is easier to track them. Only three fingerswill be colored. The thumb, index, and middle fingers. The thumb will be green, the indexfinger will be blue, and the middle finger will be red.


26

4.4.2. Glasses

Specialized glasses with infrared(IR) emitters are used in conjunction with the 3D interface.

4.4.2.1. Emitters

The emitters will be a single IR LED and will be powered by onboard batteries.

4.4.2.1.1. Emitter Locations

There will be one emitter on each side of the glasses, to the left of the left eye and tothe right of the right eye

4.5. Third-Party Libraries

4.5.1. Gesture Recognition Library

We will be using a gesture recognition library with out demo applications, which we will requireand expect certain inputs and outputs from. In addition, we require that the library allows us todefine our own gestures.

4.5.1.1. Inputs

• Camera resolution

• Detection area

• Tracking frequency rate

• Multiple points of input

• Sizes of inputs points

4.5.1.2. Outputs

• Reference for object being tracked


27

• Level of recognition of gesture

• Position of gesture

• Scale of gesture

• Rotation of gesture

• Type of gesture

4.6. System Requirements

4.6.1. Hardware

The minimum hardware requirements for a personal computer running an application that utilizesare library are as follows:

• Operating system: Windows XP or Linux kernel version 2.2.5 or Higher

• CPU clock speed: 1.0 gHz single-core

• Memory available: 512 MB RAM

• Display resolution: 1024x768 pixels

• Display depth: 16-bit (16 million colors) or higher

• Video hardware: OpenGL-compatible Graphics Card with 16 MB RAM

4.6.2. Language

Our library will be written in C++ and compiled with the GNU CC compiler.


28

5. Future DevelopmentThere is a good deal of functionality that our project deserves, yet is undoable in our timeframe. If timepermits during the development of the project outlined above, the following enhancements may beimplemented. Nothing below will be addressed before everything outlined above has been completedsatisfactorily.

5.1. Application Environment

The goal of our project is to have two deliverables. One that can be used on a Linux machine, and onethat can be used on a Windows machine.

5.1.1. Apple Macintosh

Unfortunately, we had to leave a significant user base (Mac users) out. We would like to create aversion of the software that can be used on an Apple Mac.

5.1.2. Mobile Phones

Creating a slimmed down version of the library that can be used with mobile handheld deviceswould be an excellent expansion. Many of these devices have a front mounted camera that couldeasily be used by our software. Future devices may also contain gyroscopic and accelerometerequipment similar to the Wii Remote. These devices could be incorporated into our library aswell. Many mobile phones also have onboard GPS. This could be integrated into the future GPSinterface

5.2. Input devices

We would like to support many more input devices then we have time to write support for in the spanof this project. Expanding web camera support to enable users to use any suitable web camera on themarket would be the first step. Beyond that, multitouch screens lend themselves heavily to the spatialinterface.

5.3. Future Interfaces

There are many types of input and new forms of repurposing current input that could be reconfiguredinto new interfaces.

5.3.1. GPS Interface

One interface that we would be very interested in adding to the library would be a GPS interface.This interface would be capable of tracking and smoothing single or multiple GPS enabled


29

devices' data. This interface would be very useful in creating applications similar to the SeniorDesign project last year, Project Inversion.

5.3.2. Accelerometer Interface

The Accelerometer Interface allows the use of the user's real-world movements to manipulate anapplication or operating environment on a computer with the appropriate hardware.

5.3.2.1. Hardware Device

The hardware device for the use of the accelerometer interface must match the requirementsas specified in the Wii Remote section of the non-functional requirements.

5.4. Demonstrations

Along with expansions on the library, more demonstrations could be created to show off the power ofthe library. Demos need to span a gamut of functionality so that they can capture the interest of manycomputer users across disciplines.

5.4.1. Spatial Demos

The Blender Demonstration is the most impressive idea we had for a demo of the spatialinterface, but the interface's applications are endless. Any software that people can write inputplugins for, can benefit from our library. One application would be to modify the X windowssystem on Linux to make windows controllable and resizable using the spatial interface. Otherapplications of the spatial interface include a general Windows human interface device driver.

5.4.2. 3D Demos

The 3D interface can be used in a myriad of different games, as well as other applications. Any3D application can benefit from the knowledge of the location of the user. The simplest methodwould be to implement camera movement based on the users position, evoking the parallaxeffect of 3D viewing. Any 3D modeling or 3D object viewing program, such as Blender, couldbenefit from this as well. The parallax effect would greatly aid anyone looking at a 3D object ona computer screen. It allows the objects and depth to appear real to the user, whereas before theyappeared as 2D projections of objects sitting on a computer screen.

5.5. Developer Expansions

This library will be designed with expansion in mind. The parts of the library that will most likelybe expanded upon by developers are the data input sections and the interfaces. Users may want touse a specific interface along with the data transformation portion of the library, but want to use theirown input devices. Or they may have a novel idea for an interface that doesn't exist, but would besupportable utilizing the current array of input device. In these cases, the developer would simple

http://broadcast.drexel.edu:8080/ramgen/classes/JEFF_SALVAGE/JEFF_SALVAGESenior_Design_Pres_3_256k.rm


30

follow documentation provided along with the diagnostic utilities to create their desired input orinterface addition.


31

Glossary

2D ProjectionThe representation of a 3D environment projected onto a 2D plane.

Angle AxisAn alternate structure for storing rotational data. It is defined by a vector and a rotation about thatvector.

AvatarThe object in an application created by the developer that the user directly interacts with/controls.

BlobA group of similiarly colored pixels found in a frame of web camera input, defined by an area inpixels.

BluetoothBluetooth is a short-distance wireless protocol for portable devices. It is used within the scope of ourproject, as in the Wii Remote, to provide wireless connectivity between devices and a computer.

Circular regressionFinding a circle that most closely represents the trends of a set of independent points of data.

Context-sensitiveAn action that is diffenrent depending on the current area of interaction. For example: a context menu,when requested on a file, shows "Delete", "Rename", "Edit", et cetera.

Data PointA data point is a point being tracked by the engine. Its values are what are reported to the subscriber.

DerivativeThe slope of a line. The rate of change in a function over time.

DeveloperThe target audience for our library. This is the person that will develop custom applications for usersto use.


32

Error BoundsThe amount of error tolerated for a certain type of input. When the input

ExtrapolationEstimating the next value of an input event based on previous data.

ExtrusionExtrusion is an operation performed on an object in three-dimensional modeling. A plane of an object,when extruded, expands out or in from its original location, while its edges are still attached to theconnecting edges of the object.

Gimble lockAn event that occurs when using primitive forms of rotational representation. When rotating 2 ofthe 3 dimensions at the same time, the 3rd will become locked and any rotation on that axis will benullified.

Global Positioning System/ServiceA service that provides the position on the earth of a receiver. It uses a minimum of 3 satelites totriangulate surface position. The civilian service is accurate to within 3-5 meters.

Graphics LibraryA library containing algorithms and classes for displaying 3D environments on a screen.

InfraredInfrared is a form of light invisible to human vision. It is commonly used to provide basicconnectivity, wirelessly, between devices, such as television remotes.

Input FrameAn input frame is a set of input points that represent all of the read points coming from the inputdevice at that time.

Input PointAn input point is an XY value for a point obtained from an input device.

LathingLathing is an operation performed on a two-dimensional plane. The final result of lathing is a three-dimensional shape whose edge entirely around is the same shape as the external edge of the two-dimensional plane.


33

LibraryA library is a collection of classes that can be used by a programmer when writing an application. Alibrary usually performs a common or complex function that many applications can use.

Line of best fitThe line that best represents a grouping of coorelated or semi-random data. It is computed using linearregression.

Linear regressionFinding a line that most closely represents the trends of a set of independent points of data.

Multipoint objectMultipoint Object - an object that contains multiple data points, all statically positioned.

Parallax EffectThe parallax effect is a phenomenon that occurs when an object is viewed from two viewpointscausing the object to appear at its proper depth. In the scope of our project, when an object moves androtates appropriately opposite on screen to the viewers position in the room, an illusion of depth iscreated.

Precompilation DirectiveA preprocessor directive is code that is interpreted as a compiler compiles code. These segments tellthe compiler what sections of code to compile and how to compile them. Examples are #ifndef …#endif.

Preprocessor VariableA variable that can be set in a compilation call, and can be used by percompilation directives.

QuaternionsQuaternions are the standard structure for storing rotational data. They are immune to gimble lock.

Reference frameA reference frame is a set of axes within which a position in space can be measured.

RegressionA statistical method for finding the mean of a data set.


34

SmoothingSmoothing in the scope of this project refers to the removal of outliers from a data set in order to yielda more consistent function, as well as bounding accurate data to within an error range to consider itacceptable data.

STIThe STI, or Spatial Tracking Interface, is a set of functionality that allows programmers to providemanipulation of their application through observable gestures that the user makes. See SpatialTracking Interface for more information.

SubscriberA subscriber is an agent that requests data point values from the library.

ThreadA thread is a process in a computer. A single processor can only process one thread at a time, thoughprocessed threads are switched rapidly in most modern computers. Having multiple threads allows fora long running process to run independent of a quicker one. Multi-core machines take advantage ofthreads by executing multiple threads at the same time.

UserThe target audience of our developer. This is the person that will use the custom application that thedeveloper creates.

VarianceThe variance of a random variable is the average of the squared distance of its values from thestatistical mean.

Wii RemoteA Wii Remote is an input device created by Nintendo for their Wii video game console. It hasseveral buttons for digital manipulation, as well as an accelerometer, for measuring relative velocity,Bluetooth for connectivity, and infrared sensors for measuring contextual position.

Yaw-Pitch-RollAn alternate structure for storing rotational data. It is definded by a rotation ammount in each of the 3axes, X, Y and Z, or Pitch, Yaw, and Roll, respectively. This form of representation is vulnurable togimble lock.

Clementine Requirements Document - Drexel CCI · Project Clementine is an attempt to create a...

Documents

Transcript of Clementine Requirements Document - Drexel CCI · Project Clementine is an attempt to create a...