Presented to: SOA Brown Bag #9 By: SWIM Governance Team/ Atif Chaughtai, TIBCO Software Date: July...

Presented to: SOA Brown Bag #9

By: SWIM Governance Team/ Atif Chaughtai, TIBCO Software

Date: July 13, 2011

Federal AviationAdministration Complex Event Processing

Federal AviationAdministration

SOA Brown Bag #9: Complex Event ProcessingJuly 13, 2011

50 million people affected Nuclear power plants in New York and Ohio shut down Air traffic was slowed as flights were halted

Remember This?

2



Why?

Lack of understanding of VAR reserves in the region and the adjacent regions, coupled with the possibility of inadequate dynamic VAR support available from generators. (VAR is shorthand for reactive power, which is the additional power required for maintaining voltage stability when serving certain kinds of load, such as motors, air conditioning, and fluorescent lights.)

Insufficient “visibility” of power flow conditions over the entire region, coupled with inadequate coordination, control and communication of the power system on a regional basis. That information is flowing from 27 distribution feeds, eight re-closure or safety switches and 4,192 transformers.

Insufficient understanding of the potential impact on August 14, 2003 of new power flow patterns caused by increased wholesale power transfers resulting from industry restructuring.

Lack of real-time regional and interconnection-wide power flow models for anticipating changing flow patterns and the formation of new bottlenecks.

•Electric Power Research Institute (EPRI) White Paper Findings:

3



MainframeMainframe DatabaseDatabase

3-Tier3-Tier

OnlineOnline

ESBESB

N-TierN-Tier

Event DrivenEvent Driven

2-Tier2-Tier

BatchBatch

000,000’s000,000’s 000,000,000,000’s000,000,000,000’s000’s000’s

Building Block

Enterprise 1.0

(’60s – ’80s)

Data Processing

Enterprise 2.0

(’80s – 2000)

Client Server

Enterprise 3.0

(2000 – 2020)

Predictive

Software

Velocity

Interactions

4



MainframeMainframe DatabaseDatabase

3-Tier3-Tier

OnlineOnline

ESBESB

N-TierN-Tier

Event DrivenEvent Driven

2-Tier2-Tier

BatchBatch

000,000,000’s000,000,000’s 000,000,000,000’s000,000,000,000’s000,000’s000,000’s

Building Block

Enterprise 1.0

(’60s – ’80s)

Data Processing

Enterprise 2.0

(’80s – 2000)

Client Server

Enterprise 3.0

(2000 – 2020)

Predictive

Software

Velocity

Interactions

Time to

React

Amount of DataHalf Life of Data

5



Transactions

Threats & Opportunities

The Data Advantage

Events

Enterprise 3.0

Enterprise 2.0

6



Understanding The Event CloudUnderstanding The Event Cloud

Enterprise EventsEnterprise Events

7



•Deriving Value from Events

Enterprise EventsEnterprise EventsCEPCEP

Value

8



Pote

nti

al B

usi

ness

Valu

e

Business Event

Result Measured

Root Cause Determined

Corrective Decision Made

Action Taken

Action time

Effective

Real-time

Decisions

Dynamic Business Conditions

Point of Transaction Response to Dynamic Conditions

Business Automation

Enables Fast Response

CEP enables More Effective Decisions

9



Using Real Time Events = A Better Way to Fly

Agenda

• Overview and vision for Complex Event Processing• Real-time customer problems and solutions

• TIBCO BusinessEvents capabilities• Answer your questions

10



Complex Event Processing (CEP): Premises

Problem

There is always a “Plan”. It Never Goes According to the “Plan”.

A “Pattern of Events” is an indicator of “opportunities” (inventory re-allocations).

A “Pattern of Events” is an indicator of “threats” (fraud, inventory shortages).

A “Pattern of Events” can be reactive situations.

A “Pattern of Events” can be predictive (early warning) situations.

Solution

CEP is the foundation technology for detecting patterns in real-time event streams correlated with historical data.

•BusinessOptimization

•BusinessProcessManagement

•Service-OrientedArchitecture

11



Velocity drives need for Event Processing

Perform Trading Analytics

Document Transfer

Handle Call Center Inquiries

Settle Trade/Transaction

Build-to-Order Product

Refresh Data Warehouse

Airline Operations

Track Financial/Risk Position

Supply Chain Updates

Phone/Service Activation

1 month

3 days

6 weeks

5 days

30 minutes

20 minutes

8 hours

1 day

1 day

3 days

1 hour

Mail… express… fax… e-mail

20 seconds

1 day

1 day

5 seconds

30 seconds

10 seconds

5 minutes

15 minutes

1 hour

Business Cycle-Time Improvements

Source: Gartner

12



Operational Intelligence in Transport & Logistics

13



Context-based Decisions in Operations

Early Alert System

AircraftMaintenance

Aircraft OperationsDispatch, Weather

Schedule Planning

Crew Management

Aircraft Communicationsand Operations

AirportOperations

14



Events driving Airline Scenarios

• March Madness Example

• An abnormally large number of “no shows” in one day could mean a mass of extras tomorrow Station alerting for passenger patterns,

monitor check-in patterns and no-show rate

• Flow Rates in/out of airport

• Only X aircraft getting out of airport, but Y>X aircraft are arriving there Prevent/avoid gridlock, compare gate and off-gate

parking capacities + numbers of passengers transferring to which flights

15



The Goal: Total Situational Awareness

Require:• Better visibility into aircraft operations across all

stations

• Apply rules to all inputs and report situations that are out of tolerance

• Display alerts in a meaningful way to inform not confuse

Motivation:

16



Complex Event Processing Solution

Enterprise Messaging

Enterprise Integration

OperationalData

ODSServices

CustomerData

CSTServices

EnterpriseData

Warehouse

EDWServices

OtherData

DataServices

TIBCO BusinessEvents

Match events & rules

Optimizers & Solvers

Aircraft Communicationsand Operations

Dispatch, Weather

17



Aircraft loading 6.25 flights/day/aircraft

544 aircraft

Flight Passengers >86M passengers

Aircraft Event Volume >60K Msgs/Day

Customer satisfaction “Lowest ratio of complaints per passengers boarded of all Major U.S. carriers “

“Diamond Award, at the Air Cargo World awards ceremony, with the highest overall airline score leading all airlines in the Performance and Value categories.”

“Favorite Domestic Airline and ranked #1 in Best Customer Service, Best Airfare Prices, Best On-Time Service, Best Baggage Service, and Best Value Frequent Flier program, among others, in the Reader’s Choice Awards by Smarter Travel.”

...

Results

18



Union Pacific Railways - Challenges

Pre-existing Business Challenges:

Less than half of all network schedules achieved.

Track velocity = 17 mph. Goal is 20 mph; every increase in 1 mph = $10 million saving to the bottom line (each month!).

Customers over-schedule and use crews unwisely.

Appear at capacity but a system of over-schedule.

19



Union Pacific Railways - Benefits

Maintenance & Crew Management

Outsourcer instantly knows where to deliver new crews

Automatically exchange locomotive maintenance order, fulfilment and invoicing data with service providers and business partners

Trains run 1/8th mile per hour faster!

Way Side Detection Real-time Alerting of wayside

diagnostic alerts Service Enabled Maintenance System Pro-actively fix wheels with faulty

bearings

20



TIBCO BusinessEvents v5.0 ®

The 5th Generation Event Platform

21



BusinessEvents - OverviewBE is a framework for messaging based intelligent, distributed agents

Predictive Enterprise

Events Objects

Rules Patterns

Actions

1. SENSE Ability to listen to events and

gather data on demand.

2. REASON Define rules of behavior to reason

on a pattern of objects, events and time

3. INFER Actions

4. SITUATION Visualize

22



State Model

BusinessEvents: State-Oriented CEP Behavior

NewOrderNew

OrderNew

Order

Condition /TimeEvent

State 1 State 2

Event Bus or Source

Low-latency reliable

Message delivery

Event and Data Model

For Information ModelingTemporal Model

For Determining

Time-Dependent InformationPattern Detection Model

For Recognizing Patterns and

Defining Actions

History and Cache

23



Concept Model

Stateful object modelling.

Easily visualize object relationships.

24



State Model / Process Flow

UML-Compliant state model.

State diagram / flow diagram is simple to follow and maintain.

25



Rule

BusinessEvents: Rule-Oriented CEP Behavior

Complex Patterns

NewOrderNew

OrderNew

Order

(Lots ofEvents)

Event Bus or Source


Message delivery



For Determining



Defining Actions

History and Cache

Reaction

26



Inference Rule Features

High Performance Pattern Matching

Large catalogue of functions built in.

27



Decision Manager Features

Rules managed in a decision-table interface.

Export/import to and from Excel.

28



Query

BusinessEvents: Query-Oriented CEP Behavior

Complex Query

NewOrderNew

OrderNew

Order

(Lots ofEvents)

Event Bus or Source


Message delivery



For Determining



Defining Actions

History and Cache

Results Event

29



Query CEP Agent Features

Common query language– Based on SQL/OQL

• Familiar to SQL users.

– Defined dynamically or statically.

– API similar in structure to JDBC.

select city, count(*)

from LoanApp {policy: maintain last 7 days where amount > 350000}

group by city

30



BE Views Provides visibility into real-time business events

Empowers users to make effective real-time decisions and appropriate actions on critical opportunities and issues.

31



Pattern Detection

TIBCO BusinessEvents Thematic Summaryevents

DecisionsProcessesand Views

TIB

CO

B

usin

essE

vent

s

Complex Event

Processing(CEP, ESP)

Decision Management(business rules)

Real-timeDashboards(real-time Inter-

active BI)

Straight Thru Processing

(real-time process eventing)

32



Application to SITS

NODE: NO.: 1SV-4.0 TITLE: SV-4 - SITS System Functions

F1

Manage Security

Airspaces

F2

Manage Flight Security

Information

F3

Monitor Airspace, Tracks, and Trajectories

F4

Monitor and Correlate Security

Reports and Events

F5

Coordinate Event and Incident

Responses

F6

Manage Classes and Rules

F7

Log, Analyze, and Generate Security Reports on SITS

Information F8

Collaborate and Share Information

Airspace definitionsAnd associated

Security constraints

Security Process Functions

Airspace Security,Flight Security,Monitoring, and Response rules

Logged Events

Log entries,Correlated information

User Inputs

Disturbance reports,NORDO reports,

Other threat reports

Events

Security Environmental InformationFlight Risk Profiles

Events

Events

AdjustedAirspace and

Flight Security Levels

Security Foundation Functions

Logging and Collaboration Rules

Logged Response Actions

Connection AwarenessInformation Sharing

Reports,CorrelatedInformation

Rules

User Inputs

User Inputs

Log entries,Correlated information

Reports, CorrelatedInformation Connection Awareness

Information Sharing

Airspace Security,Flight Security,Monitoring, and Response rules

33



Enterprise 2.0 in Action

34



Enterprise 3.0

It's so smart that it knew there was a power outage in one neighborhood 34 minutes before the first resident called the utility.

It's so smart that the number of customer-voltage complaints — about either surges or drops — went from 70 to zero.

It's so smart that it identified a transformer that was overloaded and needed to be replaced — before it got fried.

In the past, the utility knew to replace transformers when they blew and lights went out.

The next step: to collect and share information with about 25,000 homes and businesses that have installed or will install "smart meters.“

35



Sample Event Processing Usage Summary

Adaptive Marketing– Pattern: Capture opportunity with customer while ‘the window is open’.

Telco – SLA (Service Assurance)– Real Time Service Offers and Analytics

Finance– Fraud Detection– Track and Trace Trades/Deals/Settlements– Pre/Post trade exceptions

Logistics - Track & Trace– Track Packages against a “Plan”. Infer package delays in a proactive manner.

Alert customers. Government

– Track and Analyze ‘patterns’ that were otherwise very difficult to detect Dynamic Resource Scheduling

– Real Time Optimization of Resources against a “Plan”.

36



Next Steps

Email us to receive High Consequence Architecture White Paper

For more information or to arrange a meeting to discuss your business problem contact

Atif Chaughtai

[email protected]

(301) 213 6708

37



Questions?

38

Presented to: SOA Brown Bag #9 By: SWIM Governance Team/ Atif Chaughtai, TIBCO Software Date: July...

Documents

Transcript of Presented to: SOA Brown Bag #9 By: SWIM Governance Team/ Atif Chaughtai, TIBCO Software Date: July...