AIA_Error_Handling_And_Recovery
-
Upload
spkumar001 -
Category
Documents
-
view
940 -
download
2
Transcript of AIA_Error_Handling_And_Recovery
<Insert Picture Here>
AIA Error Handling and Recovery
Varsha Bhatia, Principal Product Manager
Rohit Agarwal, Principal Solution Architect
AIA Deep Dive Series
Copyright © 2008, Oracle. All rights reserved.
Agenda
• Key features
• Architecture
• Fault policy
• Notification setup
• Error notification and details
• Extending Error Handling
• Logging
• Enabling Error Handling and Logging
• Targeted new features: 2.x – 3.0
• Guaranteed Delivery and Message Resubmission
Copyright © 2008, Oracle. All rights reserved.
Key Features
Unified Approach
• Across technologies (BPEL, ESB)
• Across integration patterns
Error Notifications
• Issues error notifications to suitable Actor roles, and FYI roles
• Links to Error Console (BPM Worklist) for error details
Error Console
• Centralized UI to access error details that are assigned for resolution or for
informational purposes
• Accessible to administrators, and end-users
Error Logging
• Logs messages non-intrusively in a consistent schema
• Logs can be searched, sorted, and filtered using Oracle Enterprise Manager
Copyright © 2008, Oracle. All rights reserved.
AIA Error Handling Architecture
ESB
AIA Common Error Handler
Async BPEL Process
AIA Error Topic
ESB Error Handler
AIA Logger
ESB Error Topic Catch Block
BPEL BPEL
BPEL Java Action
Error ListenerBPEL Fault
Policy
Error Scenarios
BPEL partner link errors
BPEL non-partner link errors
Enterprise Service Bus (ESB)
errors
3rd Party Errors (e.g. ODI)
Copyright © 2008, Oracle. All rights reserved.
<Action id="aia-ora-retry">
<retry>
<retryCount>3</retryCount>
<retryInterval>2</retryInterval>
<exponentialBackoff/>
<retryFailureAction ref="aia-ora-java"/>
<retrySuccessAction ref="aia-ora-java"/>
</retry>
</Action>
<Action id="aia-ora-java">
<javaAction className="oracle.apps.aia.core.eh.BPELJavaAction"
defaultAction="ora-rethrow-fault">
<returnValue value="REPLAY" ref="ora-terminate"/>
<returnValue value="RETHROW" ref="ora-rethrow-fault"/>
<returnValue value="ABORT" ref="ora-terminate"/>
<returnValue value="RETRY" ref="aia-ora-retry"/>
<returnValue value="MANUAL" ref="ora-human-intervention"/>
Fault Policies for BPEL/ESB Errors
• BPEL Fault Policy files are located at
<SOA_HOME>/bpel/domains/default/config/fault-
policies/AIADefaultPolicy.xml
• ESB Fault Policy file is located at
$AIA_HOME/config
Copyright © 2008, Oracle. All rights reserved.
Fault Message Schema
Enterprise Message Elements
• EBMID
• EBM Name
• EBO Name
• EBOID
• Verb Code
• Business Scope Reference ID
• Business Scope Reference Instance ID
• Enterprise Service Name
• Enterprise Service Operation Name
• Sender Reference ID
• Sender Message ID
• Sender Reference Transaction Code
• Sender Object Identification ID
Fault Specific Elements
• Fault Message Code
• Fault Message Text
• Severity
• Fault Stack
• Faulting Service ID
• Faulting Service Implementation Code
• Faulting Service Instance ID
• Corrective Action
• Reporting Date Time
Copyright © 2008, Oracle. All rights reserved.
Notification Setup
Notification Setup
Error routing mapped to parameters
Preference-based notification to Actor
Role, FYI Role
Copyright © 2008, Oracle. All rights reserved.
Notification Setup Parameters
• Error Code
• Fault code in BPEL, ESBString string for ESB. ESB has a
concept of error codes. Examples: bpelx:bindingFault,
bpelx:remoteFault
• System Code
• Value is obtained from EBM Header. Examples: SEBL_01,
PORTAL_01
• Service Name
• Name of the BPEL process or ESB service that encounters
error. Examples: CreateCustomerPartyProvABCSImpl,
CustomerPartyEBS
• Process Name
• Value is obtained from EBM Header. Examples: Siebel-
Portal-Get-Account-Balance
Copyright © 2008, Oracle. All rights reserved.
Error Notification Role Determination
For every error that occurs, the error handler checks for
matches in the following order:
1. SYSTEM_CODE, ERROR_CODE, SERVICE_NAME,
PROCESS_NAME
2. ERROR_CODE, SERVICE_NAME, PROCESS_NAME
3. SERVICE_NAME, PROCESS_NAME
4. SERVICE_NAME
If none of these values are available, the default roles
are fetched from the AIAConfigurationProperties.xml file.
<Property
name="EH.DEFAULT.ACTOR.ROLE">AIAIntegrationAdmin</Property>
<Property name="EH.DEFAULT.FYI.ROLE"></Property>
Copyright © 2008, Oracle. All rights reserved.
Error Notification and Resolution
Error Notification Example:
Copyright © 2008, Oracle. All rights reserved.
Error Console
Leverages SOA Suite Worklist application
• Error tracking
• Starting point for
further error resolution
• Group assignment:
Actor Role, FYI Role
• Re-assignment
• Identify responsible
persons / groups by
querying AIA setup
Note: 791273.1
Sample script to close out all the tasks
that are pending
Copyright © 2008, Oracle. All rights reserved.
Error Details in the Worklist App
Copyright © 2008, Oracle. All rights reserved.13
Extending Default Fault Message Content
Enriched Fault Content- Add custom content to base fault
message to give more complete
error context: Use xsd:anyType
- Functionally rich information
- More effective Error Resolution
- Applicable for BPEL and Mediator
errors
Depth of Content- Add one, two custom fields or entire
message payload
- View logged content in AIA Log Files
via Enterprise Manager
Enriching fault content
Step1- Implement Custom extension
handler. Implement AIA Interface
Step2- Register class with Error
Handling Framework
Step3- Identify error scenario for which
handler needs to be called within
Setup UI
Copyright © 2008, Oracle. All rights reserved.
Extending Default Fault Message Content
Copyright © 2008, Oracle. All rights reserved.
Extending Default Fault Message Content
Example
15
AIAConfigurationProperties.xml
Copyright © 2008, Oracle. All rights reserved.16
Custom Handling of Errors
Example:- Step 1:
• Indicate custom error handling
for Order Fallout functionality:
ORDER_FO
- Step 2:
• Indicate custom plus default AIA
Error Handling behavior: Error Type
= ORDER_FO, AIA_EH_DEFAULT
- Step 3:
• Implement extension listener to
subscribe to messages with JMS
Header stamp = ORDER_FO. AIA
will ensure default behavior in
additionEnabling Custom Handling of Errors
Step1- Define a unique custom filter
code
Step2- Identify error scenario for which
fault message should be
stamped with custom filter code
in Setup UI
Step3- Implement extension listener
to subscribe to AIA Error topic
for messages with custom
filter code
Copyright © 2008, Oracle. All rights reserved.
Custom Handling of Errors
Copyright © 2008, Oracle. All rights reserved.
Logging Framework
Siebel CRM On Demand
• Trace Logging –
Recording chronological
sequence of events.
- Custom XPath
- Java API
• Error Logging –
Recording of errors. No
specific configuration
required. Logging done
non-intrusively.
• Use Oracle Enterprise
Manager (OEM) as user
interface to search, sort,
and filter logs.
Copyright © 2008, Oracle. All rights reserved.
Log Search Including AIA Attributes
Copyright © 2008, Oracle. All rights reserved.
To set up error handling and logging:
• Set up Error Notifications and Roles
• Set up Trace Log Levels
• Enable Trace Logging in AIAConfigurationProperties.xml
Enabling Error Handling and Logging
Copyright © 2008, Oracle. All rights reserved.
Enable Trace Logging
• Can be done on system level or on service level
• Service level overrides system level
• To be done in AIAConfigurationProperties.xml in
$AIA_HOME/config:
-bash-2.05b$ more AIAConfigurationProperties.xml
<?xml version="1.0" encoding="UTF-8"?>
<AIAConfiguration xmlns="http://xmlns.oracle.com/aia/core/config/V1"
version="1.0">
<!-- System-wide configuration -->
<SystemConfiguration>
<!-- System configuration properties -->
<Property name="TRACE.LOG.ENABLED">true</Property>
[Snip]
• Reload the configuration file afterwards through the admin page
Copyright © 2008, Oracle. All rights reserved.
Targeted New Features 2.x – 3.0
2.x • SOA platform 10.1.3.x
1. Support OC4J JMS
2. BPEL, ESB Fault Policies
3. Error Log + Trace Log
framework
• No B2B Support
• Worklist an integral part of framework
• Log onto BPEL and ESB dashboards
externally
• No customization of notification
• Multiple notifications based on number of
errors
• Manual error resubmission
3.0• SOA platform 11g
1. Support Weblogic JMS
2. Mediator Error Handling
3. Error Log + Trace Log
framework
• Capture Errors from B2B
• Option to decouple or switch off Worklist
• Drill down into EM Control Flow Trace
from email notification
• Customization of email notification content
• Notification aggregation
• Automatic error resubmission
Copyright © 2008, Oracle. All rights reserved.
Guaranteed Delivery and Error Recovery
Copyright © 2008, Oracle. All rights reserved.
Topics
Concepts
AIA Approach
Source Responsibilities
Sample Flows
Configuration Settings
Error Recovery
AIA Recommends
Copyright © 2008, Oracle. All rights reserved.
Guaranteed Delivery - Concepts
What is Guaranteed Delivery
• A message initiated from sender system is persisted until it is successfully delivered to and acknowledged by the receiver
Sender and Receiver are not necessarily the participating applications, they are the logical milestones in an integration scenario
• There could be multiple milestones in a given integration scenario
The work done between 2 milestones is considered as one logical unit of work
• This should be idempotent and could use appropriate compensation handlers to achieve this.
Copyright © 2008, Oracle. All rights reserved.
Guaranteed Delivery – Concepts …
Source
ApplicationTarget
Application
Integration
Milestone
Req
ABCS
EBS Prov
ABCS
Target
Milestone
Source
Milestone
Global Transaction
Copyright © 2008, Oracle. All rights reserved.
Target Milestone is with in Target Application transaction space
Sample Flow 2
Source
ApplicationTarget
ApplicationIntegration
Milestone
Req
ABCS
Prov
ABCSEBS
Target
Milestone
Source
Milestone
Global Transaction
Producer Process
Copyright © 2008, Oracle. All rights reserved.
AIA Approach
AQ-JMS for message persistence
Global Transaction
• Enable all services in one single transaction
• No commit point between two milestones
Error Handling
• Any system error will generate a rollback of all preceding
services
• One notification will be sent to Integration Administrator
Error Recovery
• Manual scripts to recover message from queue and resubmit
Copyright © 2008, Oracle. All rights reserved.
AIA Approach …
Each message with a unique identifier
EBM Header populated for source milestone identifier
Fault Notification shall contain the message identifier and
source milestone identifier of the faulted message
The source milestone identifier will be resolved to a specific
JMS destination (or source application) in AIA
Configuration Properties File
Error Recovery scripts shall make use of these parameters
to recover and resubmit a faulted message
Copyright © 2008, Oracle. All rights reserved.
Source Milestone Responsibilities
It could be Source Participating Application
Publish outbound messages to an AQ-JMS
• One queue for each Entity
Publish a unique message identifier on the JMS header of each
message
• Default MSGID generated in the Header Variable can also be
used for correlation
Copyright © 2008, Oracle. All rights reserved.
Fault Flow
Source
ApplicationTarget
ApplicationIntegration
Milestone
Req
ABCS
EBS Prov
ABCS
Target MilestoneSource
Milestone Fault Notification
Create Task
Global Transaction
Copyright © 2008, Oracle. All rights reserved.
Configuration Settings
Global Transaction
• IDG – Chapter 11 - Enabling Transactions for an Integration Scenario
in Asynchronous Patterns of the Application Integration Architecture
• No commit point between milestones
Error Handling
• IDG - Chapter 14: Implementing Error Handling and Recovery for the
Asynchronous Message Exchange Pattern to Ensure Guaranteed
Message Delivery
• Use bpelx:rollback instead of throw in the catch blocks.
• Java snippet to send notifications for AIA / User Faults.
• Fault policies to contain a filter expression and perform “No Action” in
case of a roll back fault(ORABPEL-02180).
– Filter Expression - <test>$fault.summary/summary[contains(.,
"ORABPEL-02180")]</test>
Copyright © 2008, Oracle. All rights reserved.
Error Recovery – Schema parameters
– EBM Header / EBMTracking
– Fault / FaultNotification / FaultMessage
• SenderResourceTypeCode
– Indicates the type of resource or system
where the rolled back message is stored,
whether Queue, Topic or Resequence
Store
• SenderResourceID
– Identification of the resource/system of
type SenderResourceTypeCode
• SenderMessageID
– Identification of the message persisted in
the resource/system associated with
type SenderResourceTypeCode
Copyright © 2008, Oracle. All rights reserved.
Manual Error Recovery script – FP2.3
Manual ant script at $AIA_HOME/Infrastructure/ErrorHandling/util
• ant –buildfile MessageResubmit.xml
Resubmit Utility API,
• oracle.apps.aia.core.eh.resubmit.util.AIAEHResubmissionUtil class.
• For persistence in a topic:
– public boolean resubmitFailedMessageTopic( String topicName, String
topicTableName, String messageID );
• For persistence in a queue:
– public boolean resubmitFailedMessageQueue( String queueName, String
queueTableName, String messageID);
• For persistence in a Resequencer store:
– public boolean resubmitFailedMessageResequencer( String
routingServiceName, String groupID );
Copyright © 2008, Oracle. All rights reserved.
Auto Error Recovery using EM
EM Targets
• Register all services and Edge applications as “Targets”
AIA Auto Recovery Notification Agent
• Notified when a Target comes back to life (SEBL_01)
• Query AIA_FAULT_TABLE and resubmit all messages
faulted due to this Target
AIA_FAULT_TABLE
• Mapping table for JMS Queue Name, JMS Message ID and
Target instance id
Copyright © 2008, Oracle. All rights reserved.
Auto Error Recovery – auto update Tasks
AIA Auto Recovery Notification Agent
• update the respective Task after message resubmit
• Comments and status as completed
AIA_FAULT_TABLE updated for WF Task ID
• At Task creation time, map a task Id with JMS Message Id
Copyright © 2008, Oracle. All rights reserved.
Source
ApplicationTarget
ApplicationIntegration
Milestone
Req
ABCS
EBS Prov
ABCS
Target MilestoneSource
Milestone
Global Transaction
Update AIA_FAULT_TABLE
Fault Notification
Create Task
Update AIA_FAULT_TABLE
with Task ID
Fault Flow – auto recoveryEM
AIA Auto
Recovery
Notification
Agent
WF
Copyright © 2008, Oracle. All rights reserved.
Some Do’s and Don’ts
Fault Policy Changes Do Not Require a Restart of Oracle BPEL
Server
• http://host:port/BPELConsole/domain_name/doReloadFaultP
olicy.jsp
Fault policies should not have any retry configured in
Guaranteed Delivery Global Transaction scenario
AQ JMS should be configured for max_retries=>0
BPEL Fault policy details
http://www.oracle.com/technology/products/ias/bpel/pdf/1013
3technotes.pdf
Taxonomy of BPEL Faults
http://www.oracle.com/technology/products/ias/bpel/htdocs/o
rabpel_technotes.tn007.html
Copyright © 2008, Oracle. All rights reserved.
AIA Recommends
• Application Services being XA-Compliant is the best
option
• Non idempotent Application Services (non-retryable)
need to have corresponding compensatory services
• ABCS invoking non-idempotent application services
should invoke compensatory services in case of roll back
• Application Services implemented via JCA adapters can
leverage session management to facilitate transaction
coordination
• Requestor, EBS and Provider ABCS have to participate in
same transaction
• Avoid log running transactions by setting the transaction
boundaries based on the business requirements.
Copyright © 2008, Oracle. All rights reserved.
Common Use cases
Integration of AIA Error Handling Framework with
• Trouble Ticket Application
• Monitoring Systems (BAM, OpenView)
• Legacy / 3rd Party application/Custom Integrations
Extending AIA Error Handling
• Custom notifications
• Extending Fault Message
• Turn off notifications only
• Turn off Task Assignments only
Copyright © 2008, Oracle. All rights reserved.
Copyright © 2008, Oracle. All rights reserved.
Copyright © 2008, Oracle. All rights reserved.
Q&A