7/28/2019 Oracle Database Assessment Draf

1/17

1

Oracle Database KT AssessmentSession

TFS / HCL IMS Transition

7/28/2019 Oracle Database Assessment Draf

2/17

2

Agenda

Introduction

Operational procedures

Overview of production database clusters

Database environment overview

Database backup policy

Possible gaps and improvement suggestions

Volumetric Data

Summary

7/28/2019 Oracle Database Assessment Draf

3/17

Introduction

Toyota Financial Services (TFS), is a subsidiary of Toyota Motor Corporation

(TMC).a Japanese corporation.

The Scope for this engagement includes Database Services ( Oracle, MSSQL

and Netezza Databases) Management.

Database Management Services will comply with Toyota service

management policies, processes and procedures as prescribed by Toyota(e.g., Incident and Problem management, configuration management,

change management, release management, asset management).

Service Delivery Locations

This team will be centralized and will be based out of HCL, India and

TMCC locations

HCL Global Delivery Centers

Noida/Hyderabad, India.

Cont

3

7/28/2019 Oracle Database Assessment Draf

4/17

4

DatacentersTCP, TCXdatabase servers are hosted on TCP (Phoenix) andTCX (Chandler) locations.

EnvironmentsProduction, Staging, Testing, Development

Support Features

Installation/De-commissioning: Instance/database installation, De-commissioning & upgrades

Security : Implementing security standards, enforce adherence andcontinuous audit.

Backups: Ensure database backups performed through Netbackup inaccordance with the plan.

Monitoring: Continuous monitoring and attending to alerts generated andreceived through HPOV to ensure minimal disruption of services.

Reporting : Monthly Reports are generated to provide various server anddatabase information to TFS.

7/28/2019 Oracle Database Assessment Draf

5/17

5

Operational Procedures :

NER process IRT Process

Incident Management

Problem Management

Change Management

Configuration Management

7/28/2019 Oracle Database Assessment Draf

6/17

6

NER process : Used to build Oracle Database instance as required by

Application Teams. The NER process is used during creation of new

environment across development, test, stage and production.

IRT Process : Used to maintain consistent database parameters and System

settings across production database instances. Before any CI moves into

Production and gets Operational it has to undergo Infrastructure Readiness

Testing (IRT).

Incident Management : Used to report any disruption of service or deviation

from agreed standards.

Problem Management : Used for managing persistent issues.

Change Management : Alter or Deploy database objects on production andnon production environments

Configuration Management : To ensure accuracy of database server relatedinformation in CMDB. CI is updated in CMDB to ensure proper implementation

during the course of IRT.

7/28/2019 Oracle Database Assessment Draf

7/17

7

7/28/2019 Oracle Database Assessment Draf

8/17

8

7/28/2019 Oracle Database Assessment Draf

9/17

9

Overview of Prodcution DB Cluster

7/28/2019 Oracle Database Assessment Draf

10/17

10

The cluster may be of active/passive type. The cluster software used for

Oracle Farms is VERITAS Clusters

No RAC is used for Oracle databases in TFS environment

DB_MON Schema captures all system related information through scheduled

jobs which are correlated for reporting purposes by ops team.

Scheduled backups performed through Netbackup.

The database size for different applications varies in the range of 1.2 GB to 3.5

TB.

The total number of supported Oracle database instances are 305

Configuration item ( CI )provides details of Oracle databases currently inoperational status.

DBA passwords have to be changed every 90 days and are controlled by AD.

7/28/2019 Oracle Database Assessment Draf

11/17

Database environment overview

Currently there are 305 Oracle Databases distributed across the two data

centers

11

Support Hours: 24 x 7

7/28/2019 Oracle Database Assessment Draf

12/17

Oracle Database Backup policy

All of the oracle production databases are backed up using the Oracle RMAN

technology and using VERITAS Net Backup.

The backup strategy or all Oracle production databases are as follow.

Every Sunday Full backup. Few of the databases full backup is configured on

Saturday.

Monday, Tuesday, Thursday, Friday and Saturday incremental and

Wednesday Cumulative backup.

Oracle sub-production database backups are not scheduled and are

performed only on adhoc basis.

The default retention of Full backups has been defined for 90 days

All other backup (incremental, cumulative and archivelog backups) is

retained till 21days

12

7/28/2019 Oracle Database Assessment Draf

13/17

13

Possible Gaps and Improvement Suggestions

NO High Availability : There is no RAC implemented in TFS environment . Hence , risk is involvedas far as availability is concerned .

Support Gap : Currently 90% of the databases are of 9i and 10g .

Here , we have an issue in having vendor support for 9i and for

10g as well in future.

Oracle ASM features are not used in TFS environment

No Oracle Data Guard or Oracle Standby databases are used for database replication. Only

storage and hardware level mirroring are used where database replication is requited

No documented procedures/learnings - There should be a defined DR Procedure and/or

Checklist for Databases which will be handy during DR tests.

Increase automationJobs should be implemented for frequent requirements like databaselevel information gathering etc for monthly reports in order to save time and avoid human error.

Implement AuditsImplement Audits in order to track any changesimplemented by groups provided higher privileges.

Implement Proactive Monitoring - There should be proactive database file utilization monitoring

and DBA team should get notifications similar to the database drive alerts in order to minimizeinterruption of services.

7/28/2019 Oracle Database Assessment Draf

14/17

Volumetric Data per month

Average number of NERs

Average number of incidents

Average number of Change requests

14

7/28/2019 Oracle Database Assessment Draf

15/17

Summary

Database user creation, deletion, modification has to go through Sec1 process in

business systems

Netezza Appliance Database access request come to DBA queue, but they aretransferred to Insight Team who manage those requests

Backup and restore methods: RMAN, NetBackup, Oracle Export/import, DataPump. Sometimes, Storage Snapshot, Clones as per requested.

TFS wants to virtualize all Oracle database instances as much as possible

OEM is used for Oracle Database Administration, but not any management packs andother features like email alerts, integration with ticketing tools are being used

Each database host has to follow a naming convention mentioned in Installationstandard document

Service account request application as well as database access request is done withLAN ID

15

7/28/2019 Oracle Database Assessment Draf

16/17

Any object level change requirement in Prod & Stage and Training Environment donethrough an RFC, which is raised by ADS team

HPOV monitoring agents are installed by Unix team

DBA Team usually are part of DR activity, they do not initiate DR

Patches for bug fixes usually applied after confirmation by Oracle Support

All databases have profiles created for userthere are 4 profiles created in databasesaccording to the user type and role required

There are a few SOX compliance databases, TFS conducts compliance test by SOX

16

7/28/2019 Oracle Database Assessment Draf

17/17

17

Thank You