Post on 22-Mar-2018
© Strands Inc.
PFM
PERSONAL FINANCIAL MANAGEMENT
Technical Presentation
PRESENTATION TIMELINE 1 ARCHITECTURE &
INTEGRATION
2 PERFORMANCE &
SCALABILITY
3 SECURITY
1 ARCHITECTURE & INTEGRATION
1. ARCHITECTURE & INTEGRATION
COMPONENTS
• PFM DB (database model)
• PFM DB Batch (data loading)
• PFM Web (Java Enterprise Edition 6 application)
OPTIONAL:
• PFM UI (HTML / Javascript / CSS)
• PFM Native Mobile Apps (Android / iOS)
• PFM Back-office (Java Enterprise Edition 6 application)
Strands PFM solution consists of a set of deployable components:
Presentation layer HTML, CSS, Javascript
API
Business Logic Layer Java Enterprise Edition 6
Business Logic
DAO
Struts 2
Spring
Hibernate
Database
Persistence Layer Oracle 11g / 12c
Stored Procedures
Presentation Layer HTML, CSS, JavaScript
UI
View controllers
Model / Collections
1. ARCHITECTURE & INTEGRATION
PERSISTENCE LAYERPresentation layer HTML, CSS, Javascript
API
Business Logic Layer Java Enterprise Edition 6
Business Logic
DAO
Struts 2
Spring
Hibernate
Database
Persistence Layer Oracle 11g / 12c
Stored Procedures
Presentation Layer HTML, CSS, JavaScript
UI
View controllers
Model / Collections
Database Model & Database Batch Processes
PFM DB model: definition of all DB structures needed for PFM.
• Modular
• Parametrized installation
PFM DB batch: all PFM database batch processes, such as data loading process (ETL).
• High performance
• Robustness
• Automation
1. ARCHITECTURE & INTEGRATION
PERSISTENCE LAYERPFM Database MODEL
• DDL contains 2 types of objects:
1. DB structures: Tablespaces, Schemas,
Roles, System Grants…
2. DB objects: Tables, Sequences, Indexes,
Constraints, Object Grants…
• Optimized Stored Procedures for better
performance on data critical processes
• Owner/Access schemas disposition for security
reasons
DBSCHEMA_ADM objects owner
schema
DBSCHEMA_APP access schema
PFM application
PFM DB Batch
read write
execute
DBA deploys and specific access
1. ARCHITECTURE & INTEGRATION
PERSISTENCE LAYERPFM Database BATCH
The data load is the main batch process:
• Source load
‣ Loads data into the DB (original format)
‣ Categorizes
‣ Prepares data (indexes…)
• Online load
‣ Moves data into online model
• Consolidation
‣ Goals, budgets, statistics…
PFM DB
CONSOLIDACONSOLIDAONLINE MODEL
CONSOLIDATICONSOLIDATIRELATIONAL FILES (optional)
CONSOLIDATION SOURCE load and categorization
PFM application
ONLINE LOAD
PL/SQL transformation
CONSOLIDACONSOLIDASRC_
source model
CONSOLIDATICONSOLIDATIPRE_SRC source model
1. ARCHITECTURE & INTEGRATION
PERSISTENCE LAYER: REAL TIMEWeb services are used to provide the PFM required financial data. The web service transactions are made on SOAP over HTTP. The services can be push or pull
PFM Customer
PFM Web Service
PFM Web Container
PFM
PFM Web Container
PFM EAR
PFM Server
PFM DB Servers
Core Systems
SOAP / HTTPS HTTPS
JDBS
Transaction
Management Service
PERSISTENCE LAYER: REAL TIMEStrands has implemented a realtime AMQP or JMS based solution, which subscribes to all the messages PFM is interested in.
Whenever any transaction is published by the bank’s messaging system, Strands will handle it using our core
abstract classes for loading transactional data in our database, and doing all the post-processing needed
(populate aggregation tables as needed, alerts triggering etc…).
Queue Transactions Message Consumer
Update PFM Update Stats etc.
Client Banking SYSTEM
Queue
PFM Real Time
1. ARCHITECTURE & INTEGRATION
Real time: Message queue
1. ARCHITECTURE & INTEGRATION
AGGREGATIONManual & Automatic
Strands has generic manual aggregation. Specific
parsers for manual aggregation can be built for
convenience and robustness
Strands can implement automatic aggregation with a
variety of options:
• Web Services (OFX, FinTS, ...) for specific banks
• Web Scraping for specific banks
• In both cases, login credentials would be stored in the
PFM database with an encrypted table
We work with partners that build aggregation (Yodlee,
Intuit, Fiserv, Eurobits, ...)
Strands Data Model
Multi-Entity Model
Aggregation Engine
OFX FinTS robot 1 robot 2 robot n…
1. ARCHITECTURE & INTEGRATION
PERSISTENCE LAYERCategorization
• Categorization can be seen as a Classification
Problem that can be solved with Machine Learning.
The problem can be formulated as: classify
transactions into a given set of categories.
• The Strands categorizer is generic, meaning it works
without having to know what transaction data is being
used.
• Transactions metadata like description, sign, MCC…
Categorised
Transactions
Machine Learning
Algorithm
Classification Rule Un-categorised
TransactionsCategorised Transactions
LABELED SET OF EXAMPLES
NEW EXAMPLE PREDICTED
CLASSIFICATION
1. ARCHITECTURE & INTEGRATION
PERSISTENCE LAYERCategorization
INITIAL STEPS
GO LIVE
MAINTENANCE
• Create taxonomy
• Identify Transaction Data: find relevant metadata fields from the transactions
• Make Rules
• Order Rules
• Monitor: validate categorization success, important metrics:
‣ % uncategorized: top occurring metadata among uncategorized transactions
‣ % recategorized: top occurring rules among recategorized transactions
• Improve
ALWAYS MORE THAN 90% OF AUTOMATIC CATEGORIZATION
1. ARCHITECTURE & INTEGRATION
BUSINESS LOGIC LAYERPresentation layer HTML, CSS, Javascript
API
Business Logic Layer Java Enterprise Edition 6
Business Logic
DAO
Struts 2
Spring
Hibernate
Database
Persistence Layer Oracle 11g / 12c
Stored Procedures
Presentation Layer HTML, CSS, JavaScript
UI
View controllers
Model / Collections
Java Application Server with standard libraries such as Struts and Hibernate
KEY FEATURES:
• Modular
• Serves both PFM UI and PFM Native Mobile apps
• REST like API to access PFM services
• JSON or XML responses
• Standard technology for financial environments
LIBRARIES:
• JDK, servlets
• Request mapping and routing
• Dependency injection, transaction management
• Object-relational mapping
PFM DB
1. ARCHITECTURE & INTEGRATION
BUSINESS LOGIC LAYERrequest
HTTPSresponse
HTTP Server
Application Server
Authentication Filter
STRUTS 2 Filter
net Worth API saving Goals API budgets API
…
COMMON API
Business Logic Business Logic Business Logic Business Logic
Data Access Data Access Data Access Data access
JDBC JDCB JDCB JDCB
i18n i18n i18n i18n
Hibernate Hibernate Hibernate Hibernate
SSO
WS
1. ARCHITECTURE & INTEGRATION
BUSINESS LOGIC LAYER: INTEGRATION
HTTP Server
Apache HTTP Server
IBM HTTP Server
Oracle iPlanet Web Server
Database
Oracle 10g
Oracle 11g
IBM DB2
Application Server
Tomcat
WebSphere
WebLogic
JBoss
Operating system
Linux
Unix variations
(like AIX and Solaris)
Load Balancer
Web Server nWeb Server 1
Load Balancer
Application Server 1
Application Server m
DB Server DB Batch
Database Storage
…
…
1. ARCHITECTURE & INTEGRATION
BUSINESS LOGIC LAYER: INTEGRATION INTERNET
ZONEBUSINESS
ZONERESOURCES
ZONECUSTOMER
ZONE
eBANKING Presentation XBF
Webserver (Apache)
eBanking Business JBF
Host
PFM Database
Webserver (Apache)
Browser
Smartphone
PFM Portal
PFM AppServer (Tomcat)
HTML, JSON ,PFM
PFM REQUESTS
XML? LOGIN ACCOUNT LIST, DEFAULT CUSTOMER BATCH JOB
JBF, PFM REQUEST
JDBCPFM JSON
BANKING JSON
PFM AJAX
HTML
eBANKING PORTAL HOST
1. ARCHITECTURE & INTEGRATION
PRESENTATION LAYERFront-end application that uses cutting-edge technologies
KEY FEATURES:
• Modular
• MVC extended approach
• Modular and event-driven
• Supports desktop (including IE8), tablet and mobile
• Responsive design
• Name-spaced to avoid code collisions (both JS and CSS)
• Accessibility
• Custom builds with the minimum JS and CSS required to display a subset of widgets
• It uses Strands UI, a custom framework used in all Strands Front-end applications
Presentation layer HTML, CSS, Javascript
API
Business Logic Layer Java Enterprise Edition 6
Business Logic
DAO
Struts 2
Spring
Hibernate
Database
Persistence Layer Oracle 11g / 12c
Stored Procedures
Presentation Layer HTML, CSS, JavaScript
UI
View controllers
Model / Collections
1. ARCHITECTURE & INTEGRATION
PRESENTATION LAYER
LIBRARIES:
• MVC
• Dependency management
• DOM manipulation
• Template engine
• Charts
Presentation layer HTML, CSS, Javascript
API
Business Logic Layer Java Enterprise Edition 6
Business Logic
DAO
Struts 2
Spring
Hibernate
Database
Persistence Layer Oracle 11g / 12c
Stored Procedures
Presentation Layer HTML, CSS, JavaScript
UI
View controllers
Model / Collections
DEVELOPMENT LIBRARIES:
• Build tools
• Styles
1. ARCHITECTURE & INTEGRATION
• Easily adaptable to any Look & Feel through LESS
• Internationalization and localization
• Configuration options including validation rules, formats, type of charts…
• Everything is thought and ready to be customized
1. ARCHITECTURE & INTEGRATION
PRESENTATION LAYER: CUSTOMIZATION
Case Studies playlist Case Studies playlist
1. ARCHITECTURE & INTEGRATION
PRESENTATION LAYER: OLB INTEGRATION
Javascript API:
• Exposing a single JS object to interact with.
• Standalone widgets or entire platform
• Inline (Preferred) or Iframe
• Simple methods for loading widgets.
• OLB and PFM communications through events.
Backend Connection:
• Direct access to PFM API
• Access handled through a Bank Proxy
Custom Builds:
• Custom builds: aggregation + compression
• Minimum JS and CSS required to display a subset of widgets
<link rel="stylesheet"
href="pfm.min.css" type="text/css">
<script src="pfm.min.js"></script>
<script>
require(["pfm/pfm"], function(PFM) {
PFM.show( widgetId,
placeholderId );
});
</script>
1. ARCHITECTURE & INTEGRATION
NATIVE MOBILE• REST like API to access PFM services
• JSON responses
Mobile Native Apps
Web Widgets
Inte
rne
t
HTTPS
HTTPS
Mobile API - PFM Core PFM Database
1. ARCHITECTURE & INTEGRATION
NATIVE MOBILEIntegration with delegation (Preferred)
Strands PFM Native Library
Mobile Banking Application
Authentication Proxy
Strands PFM API
Re
qu
est
(De
leg
atio
n P
atte
rns)
Re
spo
nse
(D
ele
ga
tio
n P
att
ern
s)
Secured Request (HTTPS + Signature)
Secured Response (HTTPS + Signature)
Authenticated Request
Response
1. ARCHITECTURE & INTEGRATION
NATIVE MOBILEIntegration with SessionID
Mobile Banking Application
Strands PFM Native Library
Authentication Proxy
Strands PFM API
Initia
lise Lib
rary w
ith
Se
ssion
ID
Secured Request (HTTPS + Signature)
Secured Response (HTTPS + Signature)
Authenticated Request
Response
2 PERFORMANCE & SCALABILITY
2. PERFORMANCE & SCALABILITY
KEY ASPECTS
• Intensive use of AJAX calls, highly asynchronous compared to other web solutions.
• Standalone widgets that can live out of the context of the application platform itself.
• Strategies to improve the performance and scalability:
‣ Cache: using a customizable combination of eh-cache, Hibernate and custom caching.
‣ Memory: using PL/SQL stored procedures and native queries when required to avoid intensive use of Java heap.
• Capacity, load, volume, endurance and regression testing as part of regular development process.
• Assured the performance and endurance in close collaboration with IBM Innovation Center
LIVE IMPLEMENTATIONS:
• AVERAGE 1.3M users (up to 5M)
• Batch processing of 2 million transaction in 1 hr.
• More than 800M transactions (2 years)
PERFORMANCE TESTS:
• 1.8M total users and up to 20k concurrent users
• More than 1 Billion transactions (1year)
2. PERFORMANCE & SCALABILITY
HORIZONTAL SCALABILITYBest practices for horizontal scalability have been defined in partnership with the IBM Innovation Center Barcelona
IBM Rational Performance
Tester
WorkStation SUSE Linux 2.6
Load Balancer (LPAR1)
IBM HTTP WEB
Injector (RHEL 6.1)
testUserMS1
testUserMS2
testUserMS6000000
…sticky session
distribution
CLUSTER
IBM POWER 770 AIX 6.1
node 01 Was 7
APP SERVER (LPAR1)
ORACLE DB SERVER (LPAR5)
node 03 Was 7
APP SERVER (LPAR3)
node 02 Was 7
APP SERVER (LPAR2)
node 04 Was 7
APP SERVER (LPAR4)
3 SECURITY
3. SECURITY
KEY ASPECTS
• Security policies follow ISO-27002
• Secured development practices:
• Cross-site scripting (XSS)
• Injection checks
• Data validation
• Security assessments follow OWASP & CVE
• Code assessments by external company
THANK YOU
LEARN MOREABOUT STRANDS:
WEBfinance.strands.com
BLOGblog.strands.com
TWITTER@StrandsFinance