A General Extension System for Event Processing Languages
-
Upload
alexandre-de-castro-alves -
Category
Technology
-
view
1.431 -
download
1
description
Transcript of A General Extension System for Event Processing Languages
<Insert Picture Here>
A General Extension System for Event Processing LanguagesAlexandre Alves - Oracle CEP
Monday, July 11, 2011
DEBS2011
Text
Agenda
• Scenario 1• String manipulation (programming-in-the-small)
• Blending CQL and Java• Architecture• Scenario 2
• Geo-fence (extensibility)• Blending CQL and Spatial• Architecture• Q/A
Monday, July 11, 2011
DEBS2011
Text
Scenario 1:Programming-in-the-small
• Correlate security price changes to real-time event news.
Monday, July 11, 2011
DEBS2011
Text
Scenario 1:Programming-in-the-small
SELECT * FROM news [RANGE 1 HOUR], stock_tick [RANGE 1 HOUR] WHERE /* join-criteria */
Oracle CQLProcessor
Monday, July 11, 2011
DEBS2011
Text
Scenario 1:Programming-in-the-small
• Problem: the NEWS feed is a unstructured (String), hence not trivial to identify its event properties• For example, a naive join criteria is to look for a stock’s
symbol• The problem of parsing a String can be solved using
Regular Expressions. • Regular Expressions and other programming-in-the-
small tasks have long been solved by general purpose programming languages
• Can we leverage this within an event processing language (e.g. CQL)?
Monday, July 11, 2011
DEBS2011
Text
Java Regular Expressions
• Let’s look at a simple solution to the problem using Java:
Matcher matcher = Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message);
if (matcher.find()) { symbol = message.substring(matcher.start() + 1,
matcher.end() - 1));}
Matches three letter symbols, separated by leading and trailing white-spaces
Text
Monday, July 11, 2011
DEBS2011
Text
Blending CQL with Java
CREATE VIEW filtered_news(message, matcher) AS SELECT message, Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher FROM news [RANGE 1 HOUR]
SELECT location, item_description, message FROM filtered_news, stock_tick[RANGE 1 HOUR] WHERE matcher.find() = true AND filtered_news.message.substring(matcher.start() + 1, matcher.end() - 1) = stock_tick.symbol
Monday, July 11, 2011
DEBS2011
Text
Blending CQL with Java
CREATE VIEW filtered_news(message, matcher) AS SELECT message, Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher FROM news [RANGE 1 HOUR]
SELECT location, item_description, message FROM filtered_news, stock_tick[RANGE 1 HOUR] WHERE matcher.find() = true AND filtered_news.message.substring(matcher.start() + 1, matcher.end() - 1) = stock_tick.symbol
Static method invocation
Method invocation
CQL char conversion to Java String
Monday, July 11, 2011
DEBS2011
Text
Blending CQL with Java
CREATE VIEW filtered_news(message, matcher) AS SELECT message, Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message) as matcher FROM news [RANGE 1 HOUR]
SELECT location, item_description, message FROM filtered_news, stock_tick[RANGE 1 HOUR] WHERE matcher.find() = true AND filtered_news.message.substring(matcher.start() + 1, matcher.end() - 1) = stock_tick.symbol
Overload of CQL equality to handle Java String
Overload of CQL ‘+’ operator to handle Java Integer
‘matcher’ is a Java Object, and treated as a complex type by CQL
Monday, July 11, 2011
DEBS2011
Text
Architecture
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
• Is this symbol: • A stream attribute (e.g. message)?• A stream alias (e.g. FROM S1 as A)• A CQL function (e.g. SELECT myfunc())• The l-value of complex type (e.g. myObj.myMethod())• A constructor• A (static) method• A class name
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
Monday, July 11, 2011
DEBS2011
Text
Architecture
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
• To be able to semantically compile symbols, we need to have type information.
• However, CQL is not aware of Java, nor should it be.• There is a need of a generic type-system, and an
extension model for providers to plug-in their own languages
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
Monday, July 11, 2011
DEBS2011
Text
Architecture
CQL Parser
Semantic Analyzer
Code Generator
Code Executor
Type Registry
Java Type-System
Monday, July 11, 2011
DEBS2011
Text
Architecture
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
“Pattern” Type Registry
Java Type-System
methods
fieldsComplexType
Monday, July 11, 2011
DEBS2011
Text
Architecture
“Pattern” Type Registry
Java Type-System
methods
fieldsComplexType
CQL deals with a ‘complex type’ abstraction, and does not know of its ‘Java Language’ binding.
Monday, July 11, 2011
DEBS2011
Text
Architecture
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
Find a “compile” method and its return typemethods
fieldsComplexType
Pattern
Monday, July 11, 2011
DEBS2011
Text
Architecture
Pattern.compile("[.,; ][A-Z][A-Z][A-Z][.,; ]").matcher(message)
Store ‘matcher’ as a stream/view attributemethods
fieldsComplexType
Pattern
methods
fieldsComplexType
Matcher
Monday, July 11, 2011
DEBS2011
Text
Architecture
CREATE VIEW filtered_news(message, matcher)
stream attribute*1 Type
ComplexType
<<is-of>>
Extensible Language
Java Class
<<metadata binding>>
Extension Java Language
‘matcher’ is an attribute of the view of type ‘java.util.regex.Matcher’
Monday, July 11, 2011
DEBS2011
Text
Extensibility
stream attribute*1 Type
ComplexType
<<is-of>>
Extensible Language
Java Class
<<metadata binding>>
Extension Java LanguageHow do we know which ‘language extension’ to use?How to provide new extensions?
Monday, July 11, 2011
DEBS2011
Text
Scenario 2:Spatial Integration
• Targeted marketing for a mobile subscriber• CEP application checks if the location of the subscriber is
within the distance of a registered shop
TextText
Oracle Spatial
id: CHARgeometry: SDO_GEOMETRY
Shop
Monday, July 11, 2011
DEBS2011
Text
Blending CQL with Spatial
CREATE VIEW CustomerLocation-Stream(point, custId) AS SELECT createPoint@spatial(lng, lat) as point, custId FROM Location-Stream
SELECT loc.custId, shop.idFROM CustomerLocation-Stream[NOW] AS loc, Shop as shopWHERE contain@spatial(shop.geometry, loc.point, 2.0d) ‘spatial’ link points to
Oracle Spatial ‘extension’, where ‘contain’ function resides
point is a spatial type
Monday, July 11, 2011
DEBS2011
Text
Architecture
contain@spatial(shop.geometry, loc.point, 2.0d)
CQL link specifies language extension, which are plugged into the system as a CQL cartridge
CQL Cartridge
CQL
Cartridge
<<link>>
Monday, July 11, 2011
DEBS2011
Text
Conclusion
• Blending of CQL with other languages allow for the creation of feature-rich CEP applications while still being highly descriptive• Example:
• String manipulation using Java• Geo-fencing using Oracle Spatial
• Generic frameworks allows for the dynamic plugin of extensions (cartridges)
Monday, July 11, 2011