Open Source Event Processing for Sensor Fusion Applications
-
Upload
guestc4ce526 -
Category
Technology
-
view
3.589 -
download
3
description
Transcript of Open Source Event Processing for Sensor Fusion Applications
Keith McFarlanekeith dot mcfarlane at gmail dot com
AgendaSensor Fusion ConceptsComplex Event Processing ConceptsEsper CEP EngineEventBotNext Steps
Concepts
What is sensor fusion?“…combining…sensory data or data derived from sensory data
from disparate sources such that the resulting information is in some sense better than would be possible when these sources were used individually.”1
1From Wikipedia article on Sensor Fusion
Sensor readings (bumps, ultrasonic distance measurements, rotations, etc.) give only limited views of the world
When taken together, they can be used to build a model of the world they are sensingAnalogous to perception – the lens through which
the robot views the worldOnce perception is realized, the robot can achieve
situational awarenessFinally, the robot can make predictions based on its
model, act according to its goals, and measure the model’s accuracy
Why give sensor fusion special attention?All robotics systems require some level of sensor
data correlationRequires special-purpose code – even if developed
in a general, parameterized way, it is bound to “bake-in” some assumptions
The application of open source software to the sensor fusion problem advances all at onceNo need to “re-invent the wheel” – address new
problems instead of solved problemsShare fusion advances with a broad community
Benefit from the “wisdom of the crowd”Access to cleaner code over time
Everyone contributes fixes and features, everyone benefits
Principles for Dynamic World Modeling1
1. Express real-world primitives as set of properties
2. Observation and model should be expressed in common coordinate system
3. Observation and model should be expressed in common vocabulary
4. Properties should include an explicit representation of uncertainty
5. Primitives should be accompanied by a confidence factor.
1Crowley et al
The Predict-Match-Update Cycle
Transform MatchObservations Update
World Model
Predict
A model for fusing sensor readings into an actionable view of the world
Sensors gather raw data about the world
The data are put into some form that is compatible with the model
The data are matched against / merged with previous predictions about the world
The model is updated with the new data
Predictions are reset based on updated model
Symbolic vs. Numeric MethodsSymbolic fusion entails the development and
maintenance of a schema-based world modelProperties: attributes of entities in schemaCoordinate system: position information should be
consistent with world modelVocabulary: observations must to be mapped to schema
entities and attributesPrecision: attribute value variance may be provided or
calculated using variance from predictionsConfidence: overall trust in world model may be
calculated and represented as a stateNumerical sensor fusion methods apply for math-
intensive domainsVision-based applications fall into this domain, for
example
Concepts
Enterprise Origins: SOA vs. EDAService-oriented Architecture (SOA) is a style of enterprise
software development and governanceEmphasizes discrete, distributed units of stateless software
logic over monolithic applicationsEncourages information hidingWell supported by open source tools (Apache Tomcat,
Glassfish)Event-driven Architecture (EDA) is a supplemental
approach to enterprise software development and governanceAccounts for the real-time nature of many business
transactionsEncourages discovery of business-relevant situationsNot as well supported by open source (or other) tools, but
support is growingComplex Event Processing (CEP) technologies apply very well
to problems in the EDA domain SOA Tools aren’t always a fit
Complex Event ProcessingThe acquisition, transformation, and correlation of
streams of events for use in real-time or near-real-time applicationsMany common business scenarios can be
interpreted as CEP problems Authentication: if ten password denial events occur
inside of ten seconds, lock the user out Fraud detection: if a credit card is used at locations many
miles apart within a short time period, alert the cardholderMost robotics applications could be described as
CEP applications Robots acquire events via sensors, timers, web services,
mobile phones, etc. Robots transform these into some common internal
representation Robots determine what actions to take based on
interpretations of event content, timing, coincidence, sequence, etc.
EventsFor our purposes, an event is an immutable
historical record of some observed occurrenceHistorical: by definition, events have already
happened Important to track time at the proper resolution
Immutable: no changes allowed Generally, it would not be useful to modify events in real
time Better to generate new events representing a
reinterpretation of event meaningObserved: generated externally or through
correlation Simple events represent some single occurrence
Often useful to consider a variety of sourcesIn enterprise applications, events generally originate
from software services (applications, DB triggers,…)
Event conveyance vs. processingSometimes these concepts become entangled – a
frequent point of confusion in business integration circles
Conveyance: the medium of event movementLocal calls and callbacks – e.g., Java listeners
Typical of the Observer design patternNetwork calls – REST or SOAP Web Services, Java RMI,
CORBAMessage queues – JMS-based, MSMQ, IBM MQ Series
Processing: the application of logic to a set of event streamsEngines: Esper, Coral8, Avaya Event ProcessorGoal: glean information from the chaos of event traffic
Complex EventsComplex events represent the detection of a
related set of other simple or complex eventsOften used to identify detected “situations”
E1 E1 E1 E1 E1
E2 E2 E2 E2
E3
Event Queue 1
Event Queue 2
Event Queue 3
C1
Detect event count
C2
Detect event timing
C3Detect event coincidence
Scenario: Detecting Cheap GasGas Prices
Service
Event Processor
“Cheap Gas Nearby” Alert
New Price
New Price
New Price
New Price
New Price New
Price
New Locatio
n
New Locatio
n
New Locatio
n
New Locatio
n
New Location
Rule: My Location within one mile of gas station with price under $2.75 and gas tank low
Mobile phone GPS
Location Service
New Locatio
n
Sends real-time changes in gas prices
for a set of gas stations
Gas Level
SMS
Event Pattern AbstractionA complex event that represents, denotes, or
summarizes the events of which it is composedSome examples:
A “Completed Loan Origination” event might represent the “New Loan Request,” “Credit Verified,” and “Final Approval” events in sequence
A “Human in Path” event might be generated by a car’s navigation system for coincident infrared, radar and vision events
Event processors generally support the ability to create new event streams based upon a detected set of conditions Streams of events may be cascaded for higher levels of
situational understandingSome event processors support the ability to “drill
down” to the events that triggered the abstract event
Sensor Fusion and CEP Conceptual Relationships
Sensor Fusion CEP
Collects observations Receives Events
Requires normalized observation schema
Requires normalized event data
Dynamic world modeling Event stream maintenance
Matching Event correlation/pattern detection
Situational awareness Event pattern abstraction
Symbolic fusion Event schema
Used to model the world using limited data set and predictive feedback model
Used to detect specific situations
Esper: Open Source CEPEsper is an open source CEP platform with a
rich toolset for processing streaming eventsOpen source version is very capable – all core
functions present Event query language Pattern recognizer Support for JavaBeans, Maps, and XML documents High throughput – up to 500,000 events/sec. for a
two-processor system1
Extensible – not difficult to add new input and output event connectors or event types
Commercial version available that includes HA features
http://docs.codehaus.org/display/ESPER/Esper+performance
Esper Container Architecture
Esper Lightweight CEP Container
Event Stream Connectors
and Adapters
Historical Data Access Layer
Events
Container Core
Event Query and Pattern Language Executive
Event Query and Pattern Language
Esper Engines
StatementsStatementsStatements
StatementsStatements
POJOs Output Adapters
Source: http://www.espertech.com
EPL: Event Processing LanguageUsed to specify the event processor’s behavior
when confronted with events or patterns of eventsSQL-like in appearance – helps SQL programmer’s
see analogous relationships May also confuse the issue for those who have trouble
“letting go”Rather than tables, EPL statements are run
against “streamsSQL idea of a row is replaced by the “event”
conceptIt is possible to include data from JDBC data
sources to augment event processing For example, lookup tables could be used to retrieve a
customer’s current balance based on a customer ID delivered with an event
EPL: Familiar Concepts from SQLClause Description
SELECT Indicates an that events should be selected based on a set of criteria. Can select on insert stream, remove stream, or both.
FROM Defines the event stream(s) or named window(s) from which events should be selected
WHERE Allows filtering of events based on a wide range of criteria
GROUP BY Allows stratification of events by a set of characteristics
HAVING Filters groups specified by the GROUP BY clause
INSERT INTO Creates new events in the given named window, allowing events to cascade through the system
An Example EPL Statementselect * from BatteryChargeEvent(charge<400).win:length(5)
The “select *” statement preamble indicates an event retrieval operation inclusive of all event properties
“from Withdrawal” names the event stream from which to pull events
“amount>=200” filters the events that will be accepted into the stream
“win:length(5)” defines the stream as containing the last five matching events inserted
Event StreamsDefined by EPL statement FROM clause, and
may be one of two types: filter or patternFilter-based event streams apply constraints
to incoming events, limiting stream contentExample:
select * from DistanceEvent(distance>0).win:length(10)
Supports standard comparators (=, !=, >, <, >=, <=) and ranges (between, in, not)
Pattern-based event streams look for the specified pattern of eventsExample:
select * from pattern [every left=LeftBumpEvent or every right=RightBumpEvent].win:time(5 sec)
ViewsUsed to derive or aggregate data from an
event streamMay be “staggered” (chained) within a single
EPL statementExample:
select * from DistanceEvent(distance>0).win:length(10).std:groupby(direction)
Some views delivered with Esper:win:length win:length_batc
hwin:time win:time_batch
win:time_length_batch
win:time_accum win:ext_timed
ext:sort_window
ext:time_order std:unique std:groupby win:firsttime
Hardware El CheapoLaptop
Roo232 USB-to-SerialCableRoomba
Sage
Reversed MacMini-DIN 8 M2M Cable
Roomba ConnectorHome-grown Java library that speaks Roomba SCIMulti-threaded for better processor utilizationCommand pattern applied for command encapsulation
Allow for flexibility in both queuing and macro command building
Commands are queued and processed asynchronouslySensors polled with specified frequency
Current default is 10 polls/secondPolling results in generation of event classes that are
sent to subscribed Java listenersRelevant event parameters bundled in event object
The Command Pattern
EventBot Data Flow vs. Fusion Model
Roomba Connector
Event Transformer
Esper CEP
Event Listeners
Roomba
Transform MatchObservation
sUpdate
World Model
Predict
Events Commands
World Model
Code WalkthroughExample: Governing speed via rotation sensors –
a simple sensor fusion exampleFuses multiple readings into better information
about the environment Uses an average of the last N readings to match speed
and adjust power if neededUses this data to adjust in continuous feedback loopElements:
Connector Coding EPL Statements Listener Coding
Runtime
Next StepsTransducer Markup Language (TML) adapter
Allows wide variety of sensor types to be normalized
More complex fusion exampleMore sensors – simple vision? Ultrasonic?RSS event feedRFIDWS-Eventing adapter
Open source availability of tools
email: keith dot mcfarlane at gmail dot com
ReferencesLinks
Esper home http://esper.codehaus.org
http://www.complexevents.comhttp://www.espertech.comhttp://en.wikipedia.org/wiki/Sensor_fusionhttp://en.wikipedia.org/wiki/Gang_of_Four_(software)
PapersPrinciples and Techniques for Sensor Data Fusion
http://www-prima.imag.fr/jlc/papers/SigProc-Fusion.pdf Crowley et al.
Overview of CEP Concepts http://complexevents.com/wp-content/uploads/2008/07/
overview-of-concepts-of-cep.pdf David Luckham