Using Shapes of Trends in Using Shapes of Trends in Active Data MiningActive Data Mining

Duy LamDuy Lam

Norris BootheNorris Boothe

Shape Querying and Active Shape Querying and Active Data MiningData Mining

Historical time sequences make up a Historical time sequences make up a large portion of data stored in large portion of data stored in computerscomputers

Mining trends in histories usefulMining trends in histories useful Many applications, including Many applications, including

observing trends in stock prices, observing trends in stock prices, online bids, and rule miningonline bids, and rule mining

OverviewOverview

Overview of SDLOverview of SDL SDL languageSDL language Applications to data miningApplications to data mining

A (Very) Simple HistoryA (Very) Simple History

Shape Definition LanguageShape Definition Language

SDL is a shape definition language SDL is a shape definition language used to query the “shapes” of historiesused to query the “shapes” of histories

Small, powerful language that allows Small, powerful language that allows “blurry” matching“blurry” matching

Designed to make it easy and natural Designed to make it easy and natural to queryto query Easily implementableEasily implementable Little non-determinismLittle non-determinism

AlphabetAlphabet

SDL allows you to specify an “alphabet” defining SDL allows you to specify an “alphabet” defining transitionstransitions

Example:Example:SymbolSymbol DescriptionDescription

upup Slightly increasing transitionSlightly increasing transition

UpUp Highly increasing transitionHighly increasing transition

downdown Slightly decreasing transitionSlightly decreasing transition

DownDown Highly decreasing transitionHighly decreasing transition

appearsappears Transition from zero to non-zeroTransition from zero to non-zero

disappearsdisappears Transition from non-zero to zeroTransition from non-zero to zero

stablestable The final value nearly equal to initial The final value nearly equal to initial valuevalue

zerozero Both initial and final value are zeroBoth initial and final value are zero

So with this alphabet we can So with this alphabet we can describe a shapedescribe a shape

Use such a description to query a Use such a description to query a history to produce all subsequences history to produce all subsequences that match the shapethat match the shape

Describing a shapeDescribing a shape

(shape name(parameters) descriptor)

(shape spike() (concat Up up down Down))

Derived ShapesDerived Shapes

anyany allows a shape to have multiple valuesallows a shape to have multiple values

concatconcat shapes can be concatenated together shapes can be concatenated together

contiguouslycontiguously

(any up Up)

(concat down up down up)

Multiple Occurrence Multiple Occurrence OperatorsOperators

Shapes made of multiple contiguous Shapes made of multiple contiguous occurrences of the same shapeoccurrences of the same shape

Resulting subsequences are such that Resulting subsequences are such that they are neither preceded nor followed they are neither preceded nor followed by a subsequence that matches Pby a subsequence that matches P

(exact 5 (any up Up))

(atleast 3 stable)

(atmost 2 (concat disappear appear))

Bounded Occurrence Bounded Occurrence OperatorsOperators

inin permits “blurry” matching by allowing users to state an permits “blurry” matching by allowing users to state an

overall shape without specific detailsoverall shape without specific details

within the specified time period length, we can within the specified time period length, we can have a specified number of occurrences of a shapehave a specified number of occurrences of a shape

can have arbitrary gaps and can have overlapcan have arbitrary gaps and can have overlap

(in 7 (nomore 5 up))

(precisely n P)(noless n P)(nomore n P)

Bounded Occurrence Bounded Occurrence OperatorsOperators

inorderinorder specifies shapes that must appear in a specifies shapes that must appear in a

specific orderspecific order

(inorder P1 P2 ... Pn)

(in 5 (and (noless 2 (any up Up)) (nomore 1 (any down Down))))

Shape Definition ExamplesShape Definition Examples

Shape Definition ExamplesShape Definition Examples

(in 7 (inorder (atleast 2 (any up Up))(in 4 (noless 3 (any down Down))))))

Parameterized ShapesParameterized Shapes

Can parameterize shape definitions Can parameterize shape definitions instead of using concrete valuesinstead of using concrete values

(shape spike(upcnt dncnt)(concat (exact upcnt (any up Up)) (exact dncnt (any down Down))))

(shape doublepeak(width ht1 ht2)(in width (inorder spike(ht1 ht1)

spike(ht2 ht2))))

Advantages of SDLAdvantages of SDL

natural and powerful language for natural and powerful language for expressing shape queriesexpressing shape queries

capability of blurry matchingcapability of blurry matching reduction of output clutterreduction of output clutter efficient implementationefficient implementation

SDL’s Expressive PowerSDL’s Expressive Power

SDL is equivalent to regular SDL is equivalent to regular expressions for regular matchingexpressions for regular matching

several features enchance its several features enchance its effectivesness, howevereffectivesness, however greedy matching and “lookahead” greedy matching and “lookahead”

capabilities help reduce output cluttercapabilities help reduce output clutter

SDL’s Expressive PowerSDL’s Expressive Power

““blurry” matching enables a much more blurry” matching enables a much more natural and compact specification of certain natural and compact specification of certain shapesshapes

For example, if we wanted precisely one For example, if we wanted precisely one occurrence of each aoccurrence of each aii in any order in any order in SDL:in SDL:

regular expressions requires at least exponential regular expressions requires at least exponential size to specify!size to specify!

(and (precisely 1 a1)(precisely 1 a2)...(precisely 1 an))

SDL SummarySDL Summary

SDL is a small, powerful language for SDL is a small, powerful language for naturally and intuitively expressing naturally and intuitively expressing shapes found in historiesshapes found in histories

Equivalent in power to regular Equivalent in power to regular expressions, but much more expressions, but much more effectiveeffective

Permits “blurry” matchingPermits “blurry” matching

Using SDL inUsing SDL inActive Data MiningActive Data Mining

Static Data MiningStatic Data Mining

Discovery of rules forDiscovery of rules for AssociationsAssociations SequencesSequences ClassificationClassification

Entire data set is minedEntire data set is mined Inherent weakness: Rules are not Inherent weakness: Rules are not

staticstatic

Active Data MiningActive Data Mining

Partition into time periodsPartition into time periods Run data mining algorithm on each Run data mining algorithm on each

periodperiod Gather rules into a ‘rulebase’Gather rules into a ‘rulebase’ Create triggers to discoverCreate triggers to discover

Trends in rulesTrends in rules Associations between rulesAssociations between rules

Period 3Rules

Active Data Mining ProcessActive Data Mining Process

Period 1RulesLarge

DataBase

Rule ID

History (support, confidence, etc)

1 2 … n

1

2

…

Period 2Rules

SelectedRules

Active Data Mining Process Active Data Mining Process (cont).(cont).

Rule ID

History (support, confidence, etc)

1 2 … n

1

2

…

ShapeDefinitionLanguage

TriggerDefinitionLanguage

ActiveData

Mining

Active Data Mining Active Data Mining ComponentsComponents

Shape definitions (SDL)Shape definitions (SDL) (shape (shape name(parameters) descriptorname(parameters) descriptor)) Ex: Ex:

(shape spike(upcnt dncnt)(shape spike(upcnt dncnt)

(concat (atleast upcnt (any up Up))(concat (atleast upcnt (any up Up))

(atleast dncnt (any down (atleast dncnt (any down Down))))Down))))

QueriesQueries TriggersTriggers

QueriesQueries For rule selectionFor rule selection Syntax:Syntax:

(query ((query (shape shape ((history-name start-time end-history-name start-time end-timetime))))))

‘‘start’ and ‘end’ specify the end points of start’ and ‘end’ specify the end points of historyhistory

Result: rules that match the desired shapeResult: rules that match the desired shape Ex:Ex:

(shape ramp() (concat Up Up))(shape ramp() (concat Up Up))(query (ramp() (confidence start end)))(query (ramp() (confidence start end)))

(shape upramp(len cnt) (shape upramp(len cnt) (in len (noless cnt (any up Up))))(in len (noless cnt (any up Up))))

(shape dnramp(len cnt) (shape dnramp(len cnt) (in len (noless cnt (any down Down))))(in len (noless cnt (any down Down))))

(query (and(query (and(upramp(5 3) (support start 10))(upramp(5 3) (support start 10))(dnramp(5 3) (confidence start (dnramp(5 3) (confidence start

10))))10))))

Larger Query ExampleLarger Query Example

Results: rules where support is increasing but confidence is decreasing

TriggersTriggers Datastream type functionalityDatastream type functionality ECA (Event Condition Action) model used ECA (Event Condition Action) model used

(Chakravarthy et al. 1989)(Chakravarthy et al. 1989)

Syntax:Syntax: (trigger (trigger trigger-nametrigger-name

(events (events events-specevents-spec))(condition ((condition (shape history-specshape history-spec))))(actions (actions action-specaction-spec))))

Events:Events: Rule creationRule creation History updatesHistory updates

Wave Execution SemanticsWave Execution Semantics Stratified execution of triggers – Stratified execution of triggers –

similar to Datalogsimilar to Datalog

Set ofEvents

Triggersfor thoseEvents

Queriesfor thoseTriggers

Set ofActions/Events

Trigger ExampleTrigger Example Identifying rules where support is Identifying rules where support is

increasing, but confidence is decreasingincreasing, but confidence is decreasing(trigger detect_up(trigger detect_up

(events updatehistory)(events updatehistory)(condition (upramp 5 4) (support (- end 5) end)))(condition (upramp 5 4) (support (- end 5) end)))(actions upward))(actions upward))

(trigger detect_dn(trigger detect_dn(events upward)(events upward)(condition (dnramp 5 4) (confidence (- end 5) (condition (dnramp 5 4) (confidence (- end 5)

end)))end)))(actions notify))(actions notify))

ImplementationImplementation Implemented on AIX systemImplemented on AIX system

Part of IBM’s Quest projectPart of IBM’s Quest project Successfully tested:Successfully tested:

Large set (5 years) of mail order data Large set (5 years) of mail order data (2.9 million records)(2.9 million records)

Large set (3 years) of POS (point-of-Large set (3 years) of POS (point-of-sale) transactions (6.8 million records)sale) transactions (6.8 million records)

Future WorkFuture Work At time of paper…At time of paper…

Integrate constructs into a SQL relational Integrate constructs into a SQL relational systemsystem

Improve incremental computations using partial Improve incremental computations using partial results of current trigger queriesresults of current trigger queries

Since then…Since then… Integrated into the Quest Data Mining SystemIntegrated into the Quest Data Mining System Subsumed into IBM’s data mining products, Subsumed into IBM’s data mining products,

including Intelligent Minerincluding Intelligent Miner Referenced for work in Active Data Mining and Referenced for work in Active Data Mining and

“blurry” pattern matching“blurry” pattern matching

ReferencesReferences ““Querying Shapes of Histories”, by Rakesh Agrawal, Querying Shapes of Histories”, by Rakesh Agrawal,

Giuseppe Psaila, Edward L. Wimmers, and Mohamed Giuseppe Psaila, Edward L. Wimmers, and Mohamed Zait of the IBM Almden Research Center, 1995Zait of the IBM Almden Research Center, 1995

““Active Data Mining”, by Rakesh Agrawal and Active Data Mining”, by Rakesh Agrawal and Giuseppe Psaila of the IBM Almden Research Center, Giuseppe Psaila of the IBM Almden Research Center, 19951995

““The Quest Data Mining System”, by Rakesh Agrawal, The Quest Data Mining System”, by Rakesh Agrawal, Manish Mehta, John Shafer, and Ramakrishnan Manish Mehta, John Shafer, and Ramakrishnan Srikant of the IBM Almden Research Center in Srikant of the IBM Almden Research Center in coordination with Andreas Arning and Toni Bollinger coordination with Andreas Arning and Toni Bollinger of the IBM German Software Laboratory, 1996of the IBM German Software Laboratory, 1996

IBM Almden Research Center Website: IBM Almden Research Center Website: http://www.almaden.ibm.com/software/quest/http://www.almaden.ibm.com/software/quest/