Case Study of Financial Web System Development and Operations with Oracle WebLogic 12c [CON2820]

Hirofumi Iwasaki(@HirofumiIwasaki)Arshal Ameen(@aforarsh)Financial Services Department, Development Unit, Rakuten, Inc.

2

Speaker Biography (1/2)

Arshal Ameen– Core Engineer.– Development leader for Rakuten Card Member Services.– Team Leader and Trainer.

Carrier– 3 Years as JEE engineer– App developer of iOS, Android and Win8.

Opus, Lectures, etc.– Conferences: JJUG Tokyo(2014)

3

Agenda

1. Java EE with WebLogic and Exadata for Our Financial Systems Overview

2. Starting with GlassFish, Migrated to WebLogic

3. Hard Business Operations, with the Power of the WebLogic and Exadata

4

Agenda

1. Java EE with WebLogic and Exadata for Our Financial Systems Overview

2. Starting with GlassFish, Migrated to WebLogic

3. Hard Business Operations, with the Power of the WebLogic and Exadata

5

Global Rakuten Group

Rapidly Expanding Worldwide from 2010

• E-commerce in 14 countries and regions

• All services and businesses in 28 countries

Rakuten GroupConsolidated GMS

2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013

23.407 52.319 79.134153.004329.106

478.765

832.354

1154.786

1474.570

1861.268

2746.821

3413.582

4146.244

5248.976(JPY B)

US $51.4B

8

Internet Services

(1997)

(2003)

(2006)

(2007)

(2010)

in USA

EC in France

(2009)

(2004)

(2008)

(2008)

(2011)

EC inUK

EC In Austria

In Indonesia

in Brazil

(2005)

(2009)

ECIn Thailand

(2010)

Internet Finance

(2001)(2000) (2005)

In Germany

(2012)

EC in Malaysia

(2013)

(2013)

In Spain

ECIn Singapore

ECIn Japan

Online Books,DVD Store

Online HotelReservation

Pay-per-view Video Service

Portal Site

PerformanceMarketing

In USA

InternetResearch

InternetAuction

ECin Taiwan

Global eBook

Global Video streaming Global Video

Streaming

Global SocialMessaging App

(2014)

Online SecurityBrokerage

Credit Card E-moneyInternetBanking

Rakuten LifeInsurance

IP TelephonyProfessional

BaseballMarriage

Consultant

1997 2014

(2014)

Credit Card PaymentBy Smartphone

(2012)

Point Service

(2013)

Phone Service

Online GolfReservation

Others

Energy Service

Real Café Service

Curation Service

Expanding Business Portfolio

9

Life Insurance

5 Financial Services of Rakuten Group in Japan

Credit Card

E-Money

Bank

Financial Securities

10

Life Insurance

5 Financial Services of Rakuten Group in Japan

Credit Card

E-Money

Bank

Financial Securities

Big 5 Services Each has Huge Transactions 24/7 Non-Stop Services

11

Requirements for Rakuten Financial Systems

Financial Systems Requires1. Rapidly changeable business logic as

business model changes

2. Huge request capacity as business grows

3. Must be transactional

12

3 Big Issues of the Rakuten Card Systems

Credit Card

1. Outdated !

2. Complicated !

3. Difficult!

EOL

13

3 Big Targets for New Architecture

Requirements

Sustainability

FlexibilitySystemTransparency

14

PortabilityResults ofFinancial

Sys

VendorSupport

Community

Excellent Excellent Excellent Excellent

Nothing Good Not Bad Mediocre

Why We Chose the Java EE ?

15

Why We Chose WebLogic 12c

Product Productivity Reliability Scalability Support Price Operation Development

WebLogic Server 12c

Excellent Excellent Excellent Excellent Not Good Excellent Excellent

Product A Excellent Excellent Excellent Excellent Not Good Not Good Not Good

Product B Excellent Excellent Not Good Not Good Not Good Not Good Excellent

16

Why We Chose the Oracle Exadata?

Product Productivity Data reliability

Scalability HA PCI DSS Operation

Oracle Exadata Excellent Excellent Excellent Excellent Excellent Excellent

Product X - Not Good - - - Not Good

Product Y Excellent Excellent - Excellent Not Good Not Good

17

PCI DSS Certification Requirements

We were supposed to be certified with PCI DSS, the card payment industry data security standard. Exadata is the solution.

OS audit DBA audit Standard audit Fine grain audit

Audit target

Instance start and stop, connect with admin or listener

DB operation with admin user

DB operation with login, object operation with DDL/DML, data reference, etc.

CRUD for specific data

Output OS file, listener log OS file OS file, (DBA_AUDIT_TRAIL view in Oracle)

User definition table, (DBA_FGA_AUDIT_TRAIL view in Oracle)

Audit log Time, OS info, DB instance, action, auth info, exit code

Time, DB user, action, auth info, OS user, exit code

Time, user, action, OS user, terminal name, query, etc.

time, DB user, OS user, accessed object name, fine grain audit policy name, query

18

5 Big Issues to Apply New JEE Architecture

2. Education 3. Architecture

4. Environment 5. Test

1.Policies

19

1. Policies: Case of Rakuten

Internal Development First, no outsourcing to external SI vendors. (Group All)

Financial businesses are also the target for the application of this policy.

NO ORDEREducate&Develop

2. Education

3. Architecture

4. Environment

5. Test

1.Policies

Rare Case forFinancial Systems in Japan

In-HouseDevelopment

External Vendors

20

2. Education: Read, Read, Read! 2. Education

3. Architecture

4. Environment

5. Test

1.Policies

RECOMMENDEDfor WebLogic 12c

Good & Only JapaneseEE 6 book

Start from HERE4th Edition

Good PocketReference!

For NetBeans 7with EE 6

Translated

21

2. Education: Online Materials

NetBeans Java EE docsfor Advanced Information

Original Tutorialfor Newbies (Start here!)

2. Education

3. Architecture

4. Environment

5. Test

1.Policies

22

2. Education: Simplify to Learn

Old Architecture

New Architecture

Too difficult to learn many non-standard old technologies

Simple & Easy!

23

Rich Clients(no business logics)

Call

Web Presentation(no business logics)

Business Logic(no presentations)

3. Architecture: Apply EE 6 Specs

Data AccessJPA

EJB

JSF

DBs

Java FXJTA

Automatic Transaction

Messaging

JMS MQ

Connection

RMI-IIOPOther

Servers

EMailMTA

JavaMail

JAX

CallCall

Call

Call

Call

2. Education

3. Architecture

4. Environment

5. Test

1.Policies

There's norich client

24

Core

3. Architecture: Simplified

L7 Balancer

Front Real-time

Front Batch

Web Site B

Web Site A

Internal Site

Reverse P

roxy

Batch Exec

Services (aka A

PIs)

System B

Gateway

Database

System C

Sub Proc

2. Education

3. Architecture

4. Environment

5. Test

1.Policies

25

4. Environment: Ease of Dev.

X X=

Centralized DEV DBLocal Weblogic Server instanceCode and Test with Fast-swap

2. Education

3. Architecture

4. Environmen

t

5. Test

1.Policies

26

4. Environment: Easy Startup 2. Education

3. Architecture

4. Environmen

t

5. Test

1.Policies

3. Install JDK, IDE, App servers -> Build -> Run on the local terminal

2. DownloadCode from Repository

1. Join a project.

4. Refer JIRA ticketsfor tasks

27

5. Test: Full Auto Testing &Validation.

Management Server

1. Auto PULL

2. Auto Build& Test

3. AutoValidate4. Report

HourlyZERO Violations

2. Education

3. Architecture

4. Environment

5. Test

1.Policies

28

Speaker Biography (2/2)

Hirofumi Iwasaki– Group Manager, Technology Manager– Financial Service Department, Development Unit,

Rakuten, Inc.

Carrier– Planning, designing & implementation of huge enterprise systems for financial,

manufacturing and public systems with enterprise middleware, especially Java EE & .NET in Japan for about 16 years.

Opus, Lectures, etc.– Conferences: Java Day Tokyo 2014, JJUG CCC Spring (2014), WebLogic roundtable

(2012-2013), Rakuten Tech Conference (2013) etc.– Magazine: @IT (2005-2010), CIO Magazine (2009), IT Architect (2005-2009),

Web+DB Press (2005), Java World (2001-2004), etc.

29

Agenda

1. Java EE with WebLogic and Exadata for Our Financial Systems Overview

2. Starting with GlassFish, Migrated to WebLogic

3. Hard Business Operations, with the Power of the WebLogic and Exadata

30

Chart of the mid 2011 Java EE app servers

Vendor App Server EE 5 Servers EE 6 Servers

Open Source GlassFish 2.1.1 3.1.1

Oracle WebLogic 10.3 -

IBM WebSphere 7.0 8.0

Red Hat JBoss 6.0 7.0 (partially)

In Mid 2011, We Didn’t Have WebLogic 12c Yet

We wanted to apply Java EE 6 for our new system, but not released. We decided to use GlassFish 3.1.1 until the EE 6 applied WebLogic

(12c) released.

31

Impact of the Migrating within the Project

Non-Interchangeable Development Code– Different container behaviors.– Non Java EE, different each server special APIs.

Scheduled Impacts for Migrating WebLogic Configurations– Cluster configurations for high availabilities.– Other setting adjustments.– Bug checking and applying patches.– Connecting Oracle Enterprise Manager.

GlassFish WebLogic

32

Code Base BEA WebLogic 6.0 based + Improvements

Felix OSGi modular based kernel

Web Container WebLogic Original Tomcat Container

EJB Container WebLogic Original GlassFish Original

Remote Invocation T3, RMI-IIOP, SOAP RMI-IIOP, SOAP

Transaction Processing WebLogic Original GlassFish Original

Persistence Container WebLogic Original TopLink Based

Runtime JRockit, Oracle JDK Oracle JDK

Investigation of the Differences

33

Schedule for Migrating from GlassFish to WebLogic

Java EE 6 Development

with GlassFish

Migrating toWebLogic

(12c)

Developm

entO

perations

Java EE Developmentwith WebLogic (12c)

ProductionRelease

Configuration of Java EE 6 appliedWebLogic (12c) and

Enterprise Manager (12c)

(Dec, 2011)

ProductionRelease

Finally thenew WL wasreleased at

the end of 2011

34

Non-Interchangeable Point: 1. Container Initialization

Single WAR including JSF and EJBs– GlassFish

1. EJB initialize (@Startup)

2. JSF (Servlet) initialize (HttpServlet#init())– WebLogic

1. JSF (Servlet) initialize (HttpServlet#init())

2. EJB initialize (@Startup) Inverse initialization pattern

– Affected for the server initializations.– Absorbed with the wrapper classes

35

New Functions of Moving from GF to WL 12c

* Basically We Didn’t Use These Functions

36

New Function: What WL 12c Knows About GF Config

Glassfish Weblogic Supportcontext-root context-rootsecurity-role-mapping  security-role-assignmentrole-name role-nameprinciple-name principle-namegroup-name principle-namesession-config session-descriptorsession-manager:manager-properties:reapIntervalSeconds invalidation-interval-seconds

session-manager:manager-properties:maxSessions max-in-memory-sessions 

session-manager:manager-properties:directory persistent store dir 

session-manager:manager-properties:timeoutSeconds timeout-secsejb-ref ejb-ref-descriptionejb-ref-name ejb-ref-namejndi-name jndi-nameresource-ref resource-descriptionres-ref-name res-ref-namejndi-name jndi-nameresource-env-ref resource-env-descriptionresource-env-ref-name resource-env-ref-namejndi-name jndi-nameclass-loader container-descriptordelegate prefer-web-inf-classesjsp-config jsp-descriptorcheckInterval page-check-secondskeepgenerated keepgeneratedscratchdir working-dir

* Basically We Didn’t Use These Functions

37

Non-Interchangeable Point: 2. Remote Invocation

Different EJB remote invocation operations. Wrapped for absorbing.

GlassFish (5 properties, no security)

Properties prop = new Properties();prop.setProperty(Context.INITIAL_CONTEXT_FACTORY,“com.sun.enterprise.naming.SerialInitContextFactory”);prop.setProperty(Context.URL_PKG_PREFXIES, “com.sun.enterprise.naming”);prop.setProperty(Context.STATE_FACTORIES“com.sun.corba.ee.impl.presentation.rmi.JNDIStateFactoryImpl”prop.setProperty("org.omg.CORBA.ORBInitialHost", “theservername”);prop.setProperty("org.omg.CORBA.ORBInitialPort", “3700”);Context context = new InitialContext(prop);ARemote remote = (ARemote) context.lookup(“java:global/…”);

WebLogic (4 properties, with security)

Properties prop = new Properties();prop.setProperty(Context.INITIAL_CONTEXT_FACTORY, “weblogic.jndi.WLInitialContextFactory”);prop.setProperty(Context.PROVIDER_URL, “t3://theservername:7001”);prop.setProperty(Context.SECRITY_PRINCIPAL, “weblogic”);prop.setProperty(Context.SECURITY_CREDENTIALS, “thepassword”);Context context = new InitialContext(prop);ARemote remote = (ARemote) context.lookup(“java:global/…”);

38

Agenda

1. Java EE with WebLogic and Exadata for Our Financial Systems Overview

2. Starting with GlassFish, Migrated to WebLogic

3. Hard Business Operations, with the Power of the WebLogic and Exadata

39

active-active cluster to avoid single-point of failure

Non-stop failover

Parallel operation for high performance

Atomic Database Scaling

Stand-by

Real-T

ime

Batch

Old Database New Scaling model (Exadata)

SAN

Active

Fibre Channel switch(1~8Gb/s)

InfiniBandswitch(40Gb/s)

Real-T

ime

Batch

Single point of failure for non-stop service

MySQL limitation for update transaction performance

Batch traffic adversely affects online performance

Shared storage limitation with another service's bad performance affects

5 minutes in fail over

Divide online / batch servers

High performance networking

Independent storage for 24 Hrs / 365 days performance guarantee

Storage

RT group Batch group

・・・ ・・・

Scale-out enabled architecture

Not scalable architecture

×

40

Single Database, Single Schema Strategy

Exadata X3-2

Replication(APB)

Web A

rea

INT

RA

Area

Merged toSingle Exadata

Ultra-huge financialonline transactions

with ACID props.

41

Migration of Application

Front-End (Apache)

Front-End(WebLogic)

Back-End(WebLogic)

Old App Architecture

Back-End

Database

ViewPHP

Actionwith

BusinessLogic

WebService

APIService

DataService

ExternalService

ExternalServices

DMZ(Apache)

StaticHTML,

Images,CSS

ViewFacelet

Exadata

ExternalServices

BackingBean(no

businesslogic)

BusinessLogic

Entity

External

DAO

Entity

TransactionBoundary

TransactionBoundary

BEGIN

COMMIT

WebLogicPlug-In

BEGIN

COMMIT

New App Architecture

BL move to back-end

42

Fast Deployment Operations

WebLogic Server

Single WARfor API

WebLogic Managed Server

Real-Time

Batch

For Management

Deploy

Operatio

n

DeployDeployDeployDeploy

DeployDeploy

WebLogic Management Console

Same WAR, for different servers.

Automatic multi server deploying

43

Non-stop “Production Redeployment”

WebLogic Server

Old Module

New ModuleWebLogicRequest

Dispatcher

Request with current session

Deploy

Requests

Request with

new session

Auto versioning,Non-stop redeployment

WebLogic Managed Server

WAR

Automatic multi versioning, parallel operation

Old modules will be un-deplayed gracefully when all old sessions are invalidated.

44

Managing Servers by Oracle Enterprise Manager (EM)

Introduced Oracle Enterprise

Manager

Easy to FindPerformance &

Status

45

Our Requests for WebLogic and Exadata

For WebLogic,– Appliance of latest Java EE specs ASAP!!

We know the WebLogic is the basement of the Oracle Fusion Middleware, but we want the latest EE for our products.

Yes, we’re waiting the next WebLogic version supporting JEE7.

For Exadata– Make patches easy to operate

Huge costs for updating quarterly update patch. Complicated procedures for non-stop upgrading. Hoping for the next generation updating technology.

46

Questions ?Contact Hirofumi Iwasaki (@HirofumiIwasaki)Arshal Ameen (@AforArsh)

47

Finally We’ve Got It!

Project members (Apr, 2013)

48