Core System Replacement: A Case Study of Citibank

1

Core System Replacement: A Case Study of Citibank

www.celent.com

© 2002, Celent Communications. Any reproduction of t

J U L Y 2 0 0 2

Octavio [email protected]

Celent Communications183 State StreetFifth FloorBoston, MA 02109USA

Tel.: +1.617.573.9450Fax: +1.617.573.9455Email: [email protected]

www.celent.com

T A B L E O F C O N T E N T S

C o r e S y s t e m R e p l a c e m e n t : A C a s e S t u d y o f C i t i b a n k

E X E C U T I V E S U M M A R Y . . . . . . . . . . . . . . . . . 3

C O R E B A N K S Y S T E M S . . . . . . . . . . . . . . . . . 4

I N F L E X I B I L I T Y . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

T H E F A I L U R E O F C O R E S Y S T E M M I G R A T I O N P R O J E C T S . . . . . . . . . . . . . . . . 4

C I T I B A N K ’ S C O R E S Y S T E M - - C O S M O S . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

R E P L A C I N G C O S M O S . . . . . . . . . . . . . . . . . . 7

S E L E C T I N G T H E N E W A R C H I T E C T U R E . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

S O F T W A R E - - F L E X C U B E F R O M I -F L E X . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

H A R D W A R E - - H E W L E T T - P A C K A R D S U P E R D O M E . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

F I N A N C I A L A N A L Y S I S . . . . . . . . . . . . . . . . . 1 4

C O N C L U S I O N S . . . . . . . . . . . . . . . . . . . . . . . . . . 1 7

N E X T S T E P S A T C I T I B A N K . . . . . . . . . . 17

L E S S O N S F O R O T H E R B A N K S . . . . . 18

his report by any means is strictly prohibited.

Core Systems Replacement: A Case Study of Citibank AUGUST 2002 3

E X E C U T I V E S U M M A R Y

This case studies examines the migration of Citibank’s core banking applications, called Cosmos, in Europe, Asia, Africa and Latin America onto a more modern systems architecture.

In the 25 years that had elapsed since Cosmos’ introduction, numerous versions of the software have evolved with each country making its own modifications to the underlying Cosmos source code. As a result, Citibank is forced to maintain dozens of versions of the same application, greatly increasing the bank’s IT expenses.

Starting in 1999, Citibank embarked on a project, first in Europe and shortly afterwards in Asia to replace the wholesale banking functionality offered by Cosmos through a migration onto a common platform built in a more modern environment than the classical IBM mainframe technology (Cobol, batch, VSAM, MVS) that Cosmos runs on.

Citibank selected an Hewlett-Packard hardware architecture with top-end Superdome servers running a banking application called Flexcube provided by an Indian software vendor called i-flex. Citibank created four regional centers for hosting the application and database -- London for Western Europe, Warsaw for Eastern Europe and Africa, Singapore for Asia-Pacific and Silver Springs, Maryland for Latin America.

Currently, 20 countries are live with the new application with more to follow in Europe and Asia. In Latin America, Citibank is starting to migrate both wholesale, as well as retail banking functionality onto the new application -- a move which, if successful, is likely to be mimicked by other Citibank regions. To date, there are no plans to migrate the core applications running in Citibank’s largest market, the USA. However, if the Flexcube migrations continue to be successful, we believe that it is highly likely that an attempt will be made at some stage in the future to migrate Citibank’s core systems in the USA as well.

In terms of the financial impact of the core system migration outlined above, Citibank has been reticent about providing any detailed numbers. However, the migration has been self-funding from the beginning and we estimate that Citibank is spending on the order of US$35 million annually on this migration with cost savings ultimately approaching US$100 million annually.

© 2002, Celent Communications. Reproduction prohibited. www.celent.com


C O R E B A N K S Y S T E M S

I N F L E X I B I L I T Y

Virtually all major banks in the world have at one time or another wrestled with their core banking systems. By core systems, we mean those applications responsible for the crucial task of processing and posting transactions, of performing deposit accounting, maintaining loan accounts, keeping securities positions, clearing payments, etc. These systems form the backbone of a bank’s technical infrastructure, and constitute a mission critical element in the financial services industry. Simply put, without core banking applications, most banks would instantly come to a grinding halt.

At the same time as being mission critical, core banking systems are also largely antiquated, frequently having been built in the 1970s using the technology that was current at the time. Most banks continue to use mainframe-based applications, written in Cobol or even older programming languages. While most core systems excel at their task -- processing large volumes of transactions rapidly and without fault -- they have not shown themselves to be particularly flexible. Frequently, it is difficult, if not impossible, to extract informational content from core systems. Building new functionality into the systems, be it new products, extracing information for external events such as Basle II, accessing customer information for cross selling, frequently brings with it considerable expense, because these core systems were not designed to be readily changed.

Indeed, over time so much additional capability has been heaped on these core systems, and they must exchange data with so many other applications, that any change in the core systems architecture can bring with it unforeseen consequences. A small change in a program in the core application can cause another bank application to crash. In short, making changes to core systems is difficult, expensive and downright dangerous.

Despite these shortcomings, banks have been forced to continue to maintain and develop new functionality into their core systems -- at considerable expense. Indeed, a majority of most banks’ IT budgets is dedicated to maintaining their existing legacy core systems, including minor upgrades, small pieces of new functionality, as well as necessary maintenance.

T H E F A I L U R E O F C O R E S Y S T E M M I G R A T I O N P R O J E C T S

The problems and expense associated with core systems has, of course, lead countless banks to try to replace their legacy systems with more modern applications. Applications that are database-centric (allowing easy retrieval of information), modular (allowing changes to be



made to one part, without repercussions in others), flexible (allowing new products and functionality to be readily added), and so on.

Large banks embarking on core systems replacement exercises have aimed to radically reduce their maintenance and hardware expenses and to create systems that afford much greater flexibility for the future. With the widespread use of client/server technologies in the late 1980s, numerous financial institutions decided that the time had come to retire their mainframe-based, Cobol applications. The seductive promises of lower operating expenses, of drastically reduced hardware expenditures, of greater flexibility, lead a string of banks to attempt to replace their core systems.

Virtually all of them experienced bitter disappointments. Creating the same level of functionality in a new environment from scratch proved too great a challenge. Frequently, banks did not have the necessary technical skills, and some of the technologies were undoubtedly not as mature as various technology vendors had lead their bank customers to believe. However, the most difficult hurdle was catching up with the capabilities of the existing core systems. The new generation of client/server applications had to first offer exactly the same set of features as the existing mainframe systems. However, these mainframe systems were not static targets -- new modules were constantly being added. As a result, the new generation of client/server applications being developed could never reach their goal. For many banks, these failed core system migration projects ran into the hundreds of millions of dollars. Celent estimates that Citibank, the bank examined in this case study, has spent in excess of US$1 billion over the course of years in various attempts to replace Citibank’s core banking system, an application called Cosmos.

C I T I B A N K ’ S C O R E S Y S T E M - - C O S M O S

Citibank has first deployed its current core banking system, Cosmos, in 1972. This application was built in typical IBM-mainframe environment and was designed to be a self-contained banking system. This was particularly important at the time, because Citibank was rapidly expanding into new markets around the world, and these new ventures needed to have an application which could quickly support their banking activities. Each country in which Citibank operated was given access to the source code underlying Cosmos, allowing them to make the necessary changes and adjustments for the country being entered. This meant that each country could readily alter the application without regard for other regions. By the mid-1980s, Cosmos was running in over 90 countries, each with their own little variations, and in some cases with major technological differences.

In essence, Citibank was maintaining numerous core systems -- one in each country. The cost of this rapidly began to mount. Making system-wide modifications was proving to be excessively expensive. For example, making the various versions of Cosmos Y2K-compliant cost the bank more than a staggering US$400 million. A figure that was inflated, because the



same basic changes had to be made on each of the many versions of Cosmos in production. This clearly drastically increased the cost of Y2K at Citibank, which also had to have one of the longest moratoriums during 1999 on new system development of any major bank.

While Y2K underscored the problems associated with Cosmos, this was by no means the first time that Citibank had recognized the issue. Starting in the early 1980s, Citibank tried on numerous occasions to create a new, unified application, built on more modern and flexible technologies. Each time, the bank failed. In the process, we estimate Citibank spent close to US$1 billion attempting to replace Cosmos. This failure was by no means because of Citibank’s lack of technical skills -- many other top banks had attempted to migrate their core systems during the mid-1980s and virtually all of them failed too. In fact, we are not aware of any top 100 bank in the world that was able to successfully migrate its core banking applications onto a new platform in the 1980s or 1990s. Certainly, some banks had succeeded in replacing isolated components of their systems, but a wholesale replacement of the bank’s core systems was, and continues to be, exceedingly rare. This rarity is precisely what makes this case study of Citibank so unique and so interesting.



R E P L A C I N G C O S M O S

In the 25 years that had elapsed since Cosmos’ introduction at Citibank, three basic versions of the system had evolved, each running on a different technical architecture. These three versions included:

• IBM Cosmos. This version of Cosmos runs in classical IBM main-frame environment, namely, ES9000 MVS as the hardware and oper-ating system, VSAM as the data storage mechanism, Cobol as the programming language, running in a batch mode. This version of Cosmos is typically used in larger countries in Europe and Asia, need-ing considerable scalability.

• HP Cosmos. While similar in terms of functionality to the above ver-sion, this version of Cosmos runs on Hewlett Packard hardware with HP-UX as the operating system. Usually, this version of Cosmos has been used in smaller countries within Europe, the Middle East and Latin America. The system has not been used in Citibank’s subsidiar-ies in Asia.

• Integrated Branch Banking System - IBBS. This version of Cos-mos is the one most frequently used in the Central European and African market, and also runs on HP-UX.

While the above three versions are the basic technical architectures, each country also has its own modifications, meaning that on top of the technical architectures outlined above, there are myriad other application versions. In Europe, for example, Citibank had no fewer than 18 different versions of Cosmos running, 11 of these within Euroland. It was here, in Europe, that Citibank’s migration plans for Cosmos started. A time-line for the project is shown in Figure 1 on page 8.

The impetus within the European market for starting with a core system replacement was not only the fact that Citibank had 18 separate versions of Cosmos running in Europe, but it was also the increasing integration of financial markets within the European Union that lead Citibank to decide to handle all of Europe with a single application. Various directives from the European Commission on creating a single financial market had paved the way for Citibank to handle all of Europe with one institution, regulated in one single European country. Previously, Citibank had had subsidiary banks in every individual country within the European Union. With the creation of a single market for banking in the mid-1990s, Citibank planned to centralize its European banking activities in the Untied Kingdom, creating



Citibank International plc in London. Other European countries would be branches of this bank in the UK.

This elimination of separate banks in Europe paved the way for creating a new technology architecture as well. In order to maximize efficiencies, Citibank planned to centralize new account processing in London, cash processing in Dublin, and securities processing in Germany and Italy. In order to support this move, Citibank would migrate to a new, database-centric core system, with the central servers being located in London.

S E L E C T I N G T H E N E W A R C H I T E C T U R E

A new proposal for the replacement of Cosmos was circulated within Citibank in late-1999 as part of the so-called “Single Europe Initiative” outlined above. The aim was to begin replacing Cosmos in Europe and various other regions outside of the USA after the Y2K conversion had been completed. For the various reasons previously outlined, Citibank wanted to migrate to a single system for all of its operations, and, concurrently, wanted to implement a database-centric approach to deploying applications. Citibank had been considering one of three alternatives to replace the multiple versions of Cosmos that were in circulation around the globe. These three alternatives were:

Figure 1: Project Time-Line

Source: Citibank, Celent

1999 2000 2001 2002

Proposal to replace Cosmos after 2000 is circulated within Citibank

Decision made to replace Cosmos using Flexcube running on HP-UX starting in Europe for wholesale banking, Feb 2000

Pilot project started in Finland and UK with wholesale banking, Sept 2000

Asian pilot project started in Singapore

Singapore live, May 2001

Taiwan, Hong Kong, Macau, China live on wholesale banking

Turkey live on wholesale banking

Paraguay live on both retail and wholesale banking – first country to use new retail and wholesale application

Vietnam, Brunei live on wholesale banking

1999 2000 2001 2002

Proposal to replace Cosmos after 2000 is circulated within Citibank

Decision made to replace Cosmos using Flexcube running on HP-UX starting in Europe for wholesale banking, Feb 2000

Pilot project started in Finland and UK with wholesale banking, Sept 2000

Asian pilot project started in Singapore

Singapore live, May 2001

Taiwan, Hong Kong, Macau, China live on wholesale banking

Turkey live on wholesale banking

Paraguay live on both retail and wholesale banking – first country to use new retail and wholesale application

Vietnam, Brunei live on wholesale banking



• Updating Cosmos. Rather than replace Cosmos outright, migrating all of Citibank’s regions onto a common, single version of Cosmos was considered. This alternative was rejected because it would have required too many of the key components of the Cosmos architecture would to be replaced, namely VSAM, Cobol, as well as the batch-ori-ented nature of Cosmos.

• Build new application from scratch. The second alternative consid-ered was to build a new application from scratch. Ultimately, this alter-native was also rejected, because of the considerable amount of time that would be necessary to build the new system.

• License vendor-provided application. Citibank had traditionally built most of its core applications and other systems in-house. Never-theless, the bank considered the option of licensing an existing core banking application from an external software vendor.

S O F T W A R E - - F L E X C U B E F R O M I - F L E X

Confronted with the above choices for deploying a new core banking system, Citibank decided to use a vendor-provided application. This approach was considered to not only reduce the risk of the migration, but was also estimated to be the most effective way to ensure that the migration happened quickly with a minimum of expense. Citibank considered core banking solutions from a number of vendors. Ultimately, the decision was made to use an application called Flexcube from an Indian vendor called i-flex. While Citibank retains a significant equity stake in i-flex, the decision to use the Flexcube application was made entirely independently of the bank’s ownership stake in the firm.

Ultimately, Flexcube was considered to best fit Citibank’s requirements. The modules that Citibank chose to initially deploy revolved around wholesale banking, and more specifically, commercial lending, foreign exchange and trade finance. The basic functionality that Flexcube is used to deliver includes commercial loan origination and accounting in Europe, with the UK adding investment products. In terms of electronic delivery, Citibank did not select i-flex’s solution, but instead is relying on Citibank’s existing e-banking platform called CitiDirect.

While in Asia and Europe, Citibank has only used Flexcube’s wholesale banking functionality, in Latin America the bank has started to deploy both Flexcube’s wholesale, as well as the retail banking modules. In essence, Citibank in Latin America, starting with Paraguay and the Dominican Republic, is replacing Cosmos entirely, not just in wholesale banking. An overview of the countries deploying the new core banking system is shown in Figure 3 on page 11.



H A R D W A R E - - H E W L E T T - P A C K A R D S U P E R D O M E

For a variety of reasons, Citibank selected Hewlett-Packard (HP) to provide the hardware for this new implementation. Citibank had already gained significant experience with HP hardware and used HP’s version of Unix, HP-UX, extensively throughout the bank. This meant that Citibank already had strong experience maintaining and operating an HP-UX environment. This background, coupled with the fact that Citibank had been happy with HP’s track record at the bank, meant that alternatives to HP were never very seriously considered.

In selecting which HP hardware to use, Citibank considered a number of options, including HP’s rp8400 (which supports up to 16 CPUs) and Superdome servers (up to 64 CPUs). While the rp8400s were felt to be able to provide enough processing capacity initially, Citibank wanted to keep its options open in terms of future deployments. As a result, Citibank opted to use HP’s top-end server, the Superdome. 2 Superdomes were implemented in London for

Figure 2: New Wholesale Core Banking System -- Citibank Europe

Source: Celent

Other Core SystemsCore banking OS/390Cosmos


Wholesale core banking server

2 HP Superdome serversHP-UX operating systemOracle database mgt

Citibank International plc London, United Kingdom




Country Connectivity

Server


Server

Commercial Lending

Commercial Lending

TradeFinanceTrade

FinanceInvestmentProducts

InvestmentProducts

SpecialLeasingSpecialLeasing

MQ Series for connectivityMQ Series for connectivity


Server


Server


Server


Server

Citi

Dire

ctC

itiba

nk

Who

lesa

le e

-ban

king

appl

icat

ion

Citi

Dire

ctC

itiba

nk

Who

lesa

le e

-ban

king

appl

icat

ion

Finland, Austria, Norway, Luxembourg, UK, Denmark using commercial lending and trade finance

Poland and Turkey using all wholesale banking offerings

UK also using investment products

Sweden, Spain, Belgium, Portugal and France due to go live shortly N

ew a

pplic

atio

n










Server


Server

Commercial Lending

Commercial Lending

TradeFinanceTrade

FinanceInvestmentProducts

InvestmentProducts

SpecialLeasingSpecialLeasing

MQ Series for connectivityMQ Series for connectivity


Server


Server


Server


Server

Citi

Dire

ctC

itiba

nk

Who

lesa

le e

-ban

king

appl

icat

ion

Citi

Dire

ctC

itiba

nk

Who

lesa

le e

-ban

king

appl

icat

ion

Finland, Austria, Norway, Luxembourg, UK, Denmark using commercial lending and trade finance

Poland and Turkey using all wholesale banking offerings

UK also using investment products

Sweden, Spain, Belgium, Portugal and France due to go live shortly N

ew a

pplic

atio

n



the European hub, with 2 additional servers for the Asian hub in Singapore. The hub in Warsaw is currently upgrading to a Superdome architecture as well.

As part of the deployment, HP provided Citibank with Superdome servers with a fairly limited number of processors enabled -- between four and six processors per server were deemed to be adequate initially. However, HP delivered the machines with 12 processors in each one, allowing Citibank to readily expand processing power when needed. Under this scheme, called Instant Capacity on Demand (ICOD), Citibank only pays for six processors in each server. The other six are not enabled and are actually the property of HP. Once Citibank needs the additional capacity, it can be readily switched on without delay. Citibank is then simply obligated to pay HP for the additional CPUs. Rather than involving HP or Citibank technicians and having to take servers offline and install additional CPUs and RAM, a simple financial transaction is all that occurs. From a technical perspective, nothing has changed other than additional CPUs being available.

A high level overview of the systems architecture is shown in Figure 2 on page 10. While this chart shows the European implementation, the Asia implementation is essentially similar. In London and Warsaw, Citibank decided to use Citrix (a product that allows Windows applications to be served up through a web browser) to deploy the application. In Asia and Latin America, the decision was made to rely on Oracle Application Server to deploy the application. The hardware configuration in Singapore also uses two HP Superdome servers. The connectivity to other Citibank core systems is achieved using IBM MQ Series, which is Citibank’s standard tool for inter-application messaging.

Figure 3: Countries Migrated to New Core System in Western Europe


Currently livePlanned in 2002

Hub in London


Hub in London



Figure 4: Countries Migrated to New Core System in Central and Eastern Europe and Africa


Figure 5: Countries Migrated to New Core System in Asia-Pacific



Hub in Warsaw


Hub in Warsaw


Hub in Singapore


Hub in Singapore



Figure 6: Countries Migrated to New Core System in Americas



Hub in Maryland


Hub in Maryland



F I N A N C I A L A N A L Y S I S

Understandably, Citibank has been rather reluctant to share details of the financial impact that the core system migration has had. Nevertheless, Celent has made some informed estimates based on information Citibank has made actually made public. Some of the information that Citibank and i-flex have provided is:

• A total of 280 individuals have been engaged in the migration project. A breakdown of this number is given in Figure 7 on page 15;

• The migration effort in Europe has been self-funding, since Citibank has already been able to reduce system maintenance and upgrade expenses;

• Citibank spent approximately US$9.5 million on i-flex consultants in 2001, and about US$5.8 million on software licenses for the core banking application (according to i-flex annual reports).

Based on the above numbers, Celent has extrapolated the total expenses shown in Figure 8 on page 15. Currently, we believe that Citibank is spending close to US$35 million annually on this migration. By 2003, we believe that Citibank should be able to start to wind down spending, saving especially on external staff, which are mainly provided by i-flex. This, of course, assumes that Citibank does not expand the scope of the project from wholesale banking in Europe and Asia to include other lines of business and other regions. An attempt, for example, to migrate the versions of Cosmos running in the US would carry with it significantly higher costs.



Figure 7: Employees Dedicated to Citibank Migration Project

Source: Citibank

Figure 8: Total Expenses for Cosmos Migration in Europe and Asia

Source: Celent Estimates

Number of Individuals involved in Citibank Migration Project

I-f lex developers

35%

Citibank IT testers and

analysts29%

Citibank business analysts

36%

Total : 280 employees

Annual Spending on Cosmos Migration Effort

05

10152025303540

2000 2001 2002 2003 2004 2005

US$

milli

ons Softw are Licenses

Internal Staff

External Staff

Hardw are



Our estimated annual cost savings for the migration away from the Cosmos system onto the new core banking application is shown in Figure 9 on page 16. We believe that Citibank will, ultimately, be able to save close to US$110 million annually through reduced maintenance costs, as well as more rapid development times. As a net result, we estimate that Citibank will be able generate in excess of US$80 million annually through the migration from multiple versions of Cosmos to a single version of Flexcube (See Figure 10 on page 16).

Figure 9: Cost Reduction Figure 10: Net Cash Flows

Source: Celent Estimates Source: Celent Estimates

Annual Cost Reduction Through Cosmos Migration Effort

020406080

100120

2000 2001 2002 2003 2004 2005

US$

milli

ons

Annual Cash Flow Through Cosmos Migration Effort

0

20

40

60

80

100

2000 2001 2002 2003 2004 2005

US$

milli

ons



C O N C L U S I O N S

Citibank has shown that it is possible to replace mission critical core banking applications without spending an inordinate amount of money and without risking key systems. We believe that the use of a vendor-provided software application was crucial in Citibank’s successful migration of many of the bank’s core wholesale banking applications in Europe and in Asia. Creating a new application from scratch would not only have been too costly and time consuming, it would have come with the considerable risk of failure. Failure that Citibank had experienced on numerous prior occasions when attempting to replace its core systems.

N E X T S T E P S A T C I T I B A N K

To date, Citibank has focused on replacing its core wholesale banking applications in Europe and Asia. A lot of the activity has centered on countries that are, from Citibank’s perspective, smaller regions. Many of the largest markets have not yet been impacted, including Germany, Japan and most importantly for Citibank, the USA. Whether or not Flexcube is rolled out to all Citibank regions remains to be seen. However, the success so far leads one to the conclusion that this is very likely for Citibank’s non-US operations.

The USA is an entirely separate decision point for Citibank. This market is not only distinguished by its size for the bank, but also by the fact that within the USA, Citibank operates three different versions of Cosmos (or Citichecking, as it is called in the USA). These factors certainly add to the complexity and risk of migration. Another factor is that politically it may well prove to be more difficult to make a major shift in the US market. We believe that it will be some time before Citibank considers migrating its core banking applications in the USA, but that eventually this too will happen.

While Europe and Asia have used Flexcube’s capabilities in the wholesale banking arena, two Latin American countries has successfully deployed both retail and wholesale banking modules: Paraguay, which went live with Flexcube in June 2002. The Dominican Republic will follow in September 2002. Clearly, migrating both retail and wholes banking systems onto a single application will have considerable benefits and we expect to see the Latin American approach exported to other regions in due course, once Citibank has a proof point for such a combined migration.



L E S S O N S F O R O T H E R B A N K S

Clearly, the choice of a centralized, regional architecture, using HP’s Superdome servers and i-flex’s Flexcube are interesting for other banks to observe. However, direct lessons from a financial perspective are decidedly more difficult to make. The simple reason for this difficulty is the fact that Citibank is very unusual in terms of the number of countries that the bank is active in and very unusual in terms of the number of different versions of its core banking software that were in production.

Most other banks would have far fewer versions of their core banking software running, except for those institutions who made numerous acquisitions over the years without migrating their acquisitions onto a common platform. With fewer versions of their core systems, the financial case for migration becomes less compelling. As a result, it is necessary to carefully evaluate the financial impact of a migration in order to gauge whether the risk and expense involved can be warranted. In Citibank’s case, with over 90 separate versions of its core banking applications in production, the case was clear and compelling. For other, smaller banks, active in fewer countries, the case for migrating core systems may not be so clear cut. However, what Citibank has shown is that, done correctly, with a judicious choice of technologies, core systems can be replaced, even at the largest financial institutions in the world.


Core System Replacement: A Case Study of Citibank

Documents

Transcript of Core System Replacement: A Case Study of Citibank