Software Architecture Document v0.51.doc

Architectures for Software Systems(17-655)

Software Architecture for DBAuditor2

April 22, 2008

Team Metis

Jinhee Cho

Seung Ick Jang

Taekgoo kim

GwanPyo Do

Final Project 1/52 Metis


Document Information

Title Software Architecture DocumentBased on Team AoBSaved in Physical storage place

Document Revisions History

S. No Revision Date Author Comments1 0.1 3/2/08 Taekgoo Kim Initial Creation2 0.2 4/3/08 Gwanpyo Do,

Seung Ick JangModify Introduction and Quality Attributes Scenario

3 0.3 4/9/08 Taekgoo Kim, Jinhee Cho

Modify Introduction and Quality Attributes Scenario; Create C&C View and module View

4 0.31 4/11/08 Taekgoo Kim Modify Quality Attributes Scenario.5 0.36 4/12/08 Taekgoo Kim,

Jinhee ChoModify C&C View; Modify Module view description

6 0.4 4/13/08 Jinhee Cho, Seung Ick Jang, Gwanpyo Do, Taekgoo Kim

Modify C&C View and description, Modify module view and description.

7 0.41 4/18/08 Jinhee Cho, Seung Ick Jang, Gwanpyo Do, Taekgoo Kim

Quality Attribute Scenarios

8 0.5 4/24/08 Gwanpyo Do, Taekgoo Kim

Modify Context Diagram, Architectural Drivers and Architectural Decision

9 0.51 4/27/08 Taekgoo Kim, Jinhee Cho

Modify module view and C&C view, Add allocation view

Final Project 2/52 Metis


Table of Contents

1. Introduction.............................................................................................................................71.1. DBAuditor2 System................................................................................................................71.2. Business Context......................................................................................................................72. Architectural Drivers..............................................................................................................92.1. Functional Requirements.......................................................................................................92.1.1. Project management........................................................................................................92.1.2. Project/Job Status............................................................................................................92.1.3. Query Generation and Execution..................................................................................92.1.4. Test Report Generation..................................................................................................92.1.5. Wizard..............................................................................................................................92.2. Constraints...............................................................................................................................92.3. Quality Attribute Requirements..........................................................................................102.3.1. Performance...................................................................................................................102.3.2. Portability.......................................................................................................................102.3.3. Usability..........................................................................................................................112.3.4. Availability.....................................................................................................................112.3.5. Modifiability: change of query sets..............................................................................112.3.6. Modifiability: change of specification..........................................................................112.3.7. Modifiability: addition of new benchmark service....................................................113. Architectural Decisions.........................................................................................................123.1. Architectural Style................................................................................................................123.1.1. Client-Sever....................................................................................................................123.2. Tactics.....................................................................................................................................143.2.1. Performance: Resource Demand.................................................................................143.2.2. Modifiability: Localize modifications..........................................................................153.2.3. Modifiability: Defer binding time................................................................................153.2.4. Usability: Separation of UI...........................................................................................153.2.5. Usability: Support User Initiative................................................................................163.2.6. Portability.......................................................................................................................163.3. Other decisions......................................................................................................................163.3.1. Project profile repository..............................................................................................163.3.2. Comparison[TBD-figure for comparison]..................................................................174. C & C Architectural View....................................................................................................184.1. Overall C&C View................................................................................................................184.2. Server.....................................................................................................................................214.2.1. TestExecuter..................................................................................................................224.3. Client......................................................................................................................................245. Module Architectural View..................................................................................................265.1. Representative Module View of the Server........................................................................275.2. Detailed Module View: Sever...............................................................................................285.3. Representative Module View of Client................................................................................305.4. Detail Module View: Client..................................................................................................305.5. Detail Module View: EventBusPackages............................................................................326. Allocation Architectural View.............................................................................................346.1. Deployment view...................................................................................................................346.2. Implementation veiw.............................................................................................................357. Mapping between Architectural Views...............................................................................368. Future Works........................................................................................................................379. Appendix................................................................................................................................389.1. Acronym.................................................................................................................................389.2. Functional requirements......................................................................................................389.3. Prioritized Quality Attribute Scenarios..............................................................................419.4. References..............................................................................................................................45

Preliminary Report 3/52 Metis


[1] http://dogbert.mse.cs.cmu.edu/mse2008Korea/AoB/repository/documents/AoB_SRS_v1.0_20080302.doc......................................................................................................45[2]

http://dogbert.mse.cs.cmu.edu/mse2008Korea/AoB/repository/misc/Communication_Experiment.doc.........................................................................................................................................45[3] Clements, Bachmann, Bass, Garlan, Ivers, Little, Nord, Stafford, Documenting Software Architectures – Views and Beyond, Addison-Wesley, 2003......................................45[4] Heng Chen, Myung-Joo Ko, Neel Mullick, Paulo Merson, “SEI ArchE System, Architecture & Design documentation for the ‘Architecture Expert & Design Assistant’, 2004........................45



Table of FiguresFigure 1 overall context diagram..........................................................................................................8Figure 2 Top level runtime view of the legacy DBAuditor....................................................................12Figure 3 Top level runtime view of DBAuditor2..................................................................................13Figure 4 The location of project profile..............................................................................................16Figure 5 C&C View of the system......................................................................................................19Figure 6 C&C view of the server.......................................................................................................21Figure 7 Detail View of TestExecuter.................................................................................................23Figure 8 C&C View of the client........................................................................................................24Figure 9 Top-level module view of Server..........................................................................................27Figure 10 Detailed module view of Server..........................................................................................28Figure 11 Top-level module view of Client.........................................................................................30Figure 12 Detailed module view of Client...........................................................................................30Figure 13 ServerEventBusPackage.....................................................................................................33Figure 14 ClientEventBusPackage.....................................................................................................33Figure 15 Deployment view..............................................................................................................34Figure 16 Package structure for implementation..................................................................................35

Table of TablesTable 1 Java RMI vs. TCP/IP socket...................................................................................................14Table 2 Description of Overall C&C view...........................................................................................20Table 3 Description of C&C view of the server...................................................................................22Table 4 Description of TestExecuter...................................................................................................23Table 5 Description of C&C view of the client....................................................................................25Table 6 Description of Top-level module view of server.......................................................................27Table 7 Description of detail module view of server.............................................................................29Table 8 Description of top-level module view of client.........................................................................30Table 9 Description of detiail module view of client............................................................................32Table 10 Mapping table between C&C view and Module view.............................................................36Table 11 Project Management............................................................................................................38Table 12 Project Profile.................................................................................................................... 38Table 13 Project Status.....................................................................................................................39Table 14 Query Generation and Execution..........................................................................................39Table 15 Test Report......................................................................................................................... 39Table 16 Job Status........................................................................................................................... 39Table 17 Wizard............................................................................................................................... 40Table 18 Configuration.....................................................................................................................40Table 19 Connection Control.............................................................................................................40Table 20 Environment...................................................................................................................... 40Table 21 Quality Attribute Scenario 1.................................................................................................41Table 22 Quality Attribute Scenario 2.................................................................................................41Table 23 Quality Attribute Scenario 5.................................................................................................42Table 24 Quality Attribute Scenario 6.................................................................................................42Table 25 Quality Attribute Scenario 7.................................................................................................43



Table 26 Quality Attribute Scenario 8.................................................................................................43Table 27 Quality Attribute Scenario 9.................................................................................................43



1. Introduction

1.1. DBAuditor2 System

The main goal of this project is to enhance a previously developed automated DBMS benchmark

tool, DBAuditor. The will-be developed tool, the DBAuditor2, needs to provide benchmark project

management which stores configurations and project histories. Also, testers can search the history

using filtering. The tool creates reports. Testers can make their own report format, and can choose

specific graph type among various available types to make a resource monitoring graph. The usability

must be improved by offering batch job, benchmark wizards, predefined queries of each DBMS, and

easy method of inputting day/time. The last objective is provision of detailed user guide including

basic usage of DBMS. In addition to these functionalities, the performance should be high enough to

be competitive in the DBMS testing market.

1.2. Business Context

The DBAuditor2 project was created by the Telecommunication Technology Association (TTA).

TTA is an IT standards organization, and its services include providing one-stop services for the

establishment of IT standards as well as provides testing and certification for IT products. DBMS

testing tool tests various kinds of DBMS based on several Transaction processing Performance

Councils (TPC) standards.

TTA Software Quality Evaluation Center (SQEC) had been using TeamQuest; it is a commercial

DBMS testing tool. However, the tool is too heavy to test simple TPC standards because it provides

too many features; some of them are useless for the organization. TTA wanted to have lighter and

simpler tool; still it should have core important DBMS testing functionalities. According to that needs,

DBAuditor2 is developed last year.

Even though DBAuditor2 meets some of the needs, still the tool is not easy to use for testers in

TTA. Thus, the DBAuditor2 project is created to enhance mainly usability. However, the organization

faced some problems related to performance in terms of CPU usage and latency while using

DBAuditor2. Now, performance becomes one of main purpose of the DBAuditor2 project.



1. Request for benchmark

Tester Client

DBAuditor2 Client

2. Test execution

4. Execute TPC-C/TPC-Hbenchmark test

DBMS5. Test result

7. Test report

8. Report benchmark result

DBAuditor2 Server

Target System Boundary

App Server/Client application

DBMS DBMS under testDocument delivery

UI Interaction

Query execution/return

Legends

Network communication

3,6. server-client communication

Figure 1 overall context diagram

Figure 4 Figure 1 shows the context diagram of the DBAuditor2. A client requests a DBMS

performance benchmark test to TTA. A tester in TTA requests for performance test for a DBMS under

test when TTA accepts the request. The tester interacts with the client application of DBAuditor2.

Once the tester finishes his request, the client application communicates with the server of

DBAuditor2. The server executes queries for testing the DBMS under test. When the server gets the

return of the request, it reports the results to the tester. The tester refers to the report, and makes the

benchmark result document. The document is delivered to the client.



2. Architectural Drivers

2.1. Functional Requirements

Here are key functional requirements that the DBAuditor2 shall support. For more detail of

functional requirements, refer to SRS documents [1].

2.1.1. Project management

- Create/delete/modify/search project information.

- Import/export of project profile.

- Project profile includes base information, query information, and test information.

2.1.2. Project/Job Status

- DBAuditor2 shall display the list of the test results when a project is open.

- Display current status of the data generation and current job.

2.1.3. Query Generation and Execution

- Execute batch of queries.

- Cancel during batch processing.

- Show progress during batch processing.

2.1.4. Test Report Generation

- Generate a report, display it to screen, and store it into a file.

- Display system usage information in various ways of displaying.

- Generate canceled report when a test is canceled.

2.1.5. Wizard

- Provide wizard with TPC-C/H test.

2.2. Constraints

- DBAuditor2 is developed in Java programming language; it’s constraint because DBAuditor2

is based on DBAuditor2.

- Using TCP/IP socket as a communication method between server and client is mandatory.

- DBAuditor2 shall be run on following operating systems: MS Windows, Linux 2.6.x, and HP

UX.

- DBAuditor2 shall support following DBMS: Oracle, MS SQL, DB2 and MySQL are

mandatory. Optionally, Informix and Sybase might be supported.



2.3. Quality Attribute Requirements

The DBAuditor2 shall fulfill the following quality attributes.

2.3.1. Performance

The main purpose of the DBAuditor2 is to benchmark various DBMSs’ performance. The

benchmark results would be meaningless if the reported CPU and Memory usage is immensely

affected by benchmark tool itself. Regarding to the benchmark result, credibility is very important.

While executing the benchmark, the DBAuditor2 shall minimize influences on the system where

benchmark tool runs on.

- Performance quality attribute scenario 1: A tester starts the DBAuditor2. After boot up, the

tester connects database. Schema creation and data generation is already done. The tester now

executes queries according to TPC-C. While executing queries, CPU usage of the system

where the DBAuditor2 runs on becomes 100%. Data transaction process should be separate

from the DBAuditor2. In this situation, the process of the DBAuditor2 must occupy less than

5% of CPU and Memory usage. [Environment of this response measure will be added after

discussion with the client.]


tester connects database. Schema creation is already done. The tester now generates test data

that will be inserted into DBMS. The generation time is measured; after the tester clicks

‘Generate test data’ button, generation time starts, and it ends just after generation is

completed. This generation time shall be within 10% of difference compared to the generation

time of DBGEN. DBGEN is a data generation tool which is provided by TPC, and TPC

strongly recommends using DBGEN when generating performance test data.[Issue]

2.3.2. Portability

The DBAuditor2 shall be capable to test various databases’ performance: DB2, Oracle, MySQL,

and MS-SQL.

- Portability quality attribute scenario 1: TTA can provide a service of Database performance

test for their clients. Because their clients run their Database on various operating systems,

TTA provides testing service according to the each client’s operating system. Currently, most

IT companies use Windows, Linux 2.6.x and HP UX. Thus, the DBAuditor2 shall be installed

on Windows, Linux 2.6.x and HP-UX with no modification of the system.

- Portability quality attribute scenario 2: TTA’s clients would request for testing performance of

various kinds of Databases. Currently, most IT companies use one of following DBMSs:

DB2, Oracle, MySQL, and MS-SQL. Thus, once the DBAuditor2 successfully is installed to

test a particular DBMS, the DBAuditor2 shall be operable on DB2, Oracle, MySQL, and MS-

SQL with no modification of the system.



2.3.3. Usability

The DBAuditor2 shall provide an easy way of executing tests. In TTA, testers perform benchmark

task, spending several hours or days for a test. Thus, the DBAuditor2 shall provide an easy way of

installation and test execution, intuitive menu, good learnability and comfortableness. DBAuditor2

shall satisfy end users in terms of easy-to-use.

2.3.4. Availability

The DBAuditor2 shall be up while test executions because correctness of benchmark result is very

important. When system fails, the system shall provide detailed information: the time failure occurred,

last actions performed before the failure, and possible solutions.

If a fault is occurred, the DBAuditor2 shall automatically cancel the testing, notify the failure of

testing and roll back that testing to “ready-to-go* ” status, according to TPC specifications (need guide)* ready-to-go: in this status, all parameters for testing environment are set so that a tester can start the

test by clicking start button.

2.3.5. Modifiability: change of query sets

The DBAuditor2 conducts DBMS performance testing according to TPC-C/H specification for

DB2, Oracle, MySQL, and MS-SQL. Those DBMSs can be newly released in the future. For the new

release of DBMSs, the test execution queries may be changed. The tester who is skilled in DBMSs can

modify the sets of testing queries with no modification of the system within 1 man-day.

2.3.6. Modifiability: change of specification

The client provides TPC-C/H benchmark services. TPC-C and TPC-H specification might be

updated in the future. A single developer who is skilled in TPC-C/H specification and understands the

updates of the specification shall modify the system so that it can support updated specification within

7 man-days. 7 man-days include development, test, integration, and installation.

2.3.7. Modifiability: addition of new benchmark service

The client provides TPC-C/H benchmark services. They might need to add another benchmark

services such as TPC-E. A single developer who understands the specification of newly added

benchmark service shall modify the system so that it can support the benchmark service within 90

man-days. 90 man-days include development, test, integration, and installation.



3. Architectural Decisions

3.1. Architectural Style

3.1.1. Client-Sever

The legacy system adopts client-server style. The system introduces RMI connection as a

communication channel. Also the system uses one additional communication channel for monitoring

process. The major reason for this separation of RMI communication was to remove an effect of

monitoring the system usage on the benchmark test, so as to satisfy accuracy quality attribute. That is,

if one RMI communication is shared by performance test and monitoring the system usage, the results

of the benchmark test may be affected by the system usage monitoring. On the contrary, the system

usage monitoring may be affected by the performance test, even though testers want to monitor the

system usage in a real time.

Client Application2

Client

Client Application1

Server Application1Server Application1

ResourceUsage

TestDBTestDB

Server Application2

Rule/Query/Schema

RepositoryConnT

JDBCConnT

RMIConnT

DB

File

<Legend>Application Connectors RepositorySystem

Server

Figure 2 Top level runtime view of the legacy DBAuditor

For these reasons, one RMI communication is divided into two RMI communications and both of

the accuracy of the performance test and real time monitoring of the system usage can be satisfied

with two separate RMI communications.



The architecture of DBAuditor2 system basically adopts client-server style. An end user uses the

client application on personal computer and the server application will run on a separate hardware, so

client-server architectural style is inevitably adopted.

Server

DataRule

Client

CQuerySet

SystemConfig

CProjectProfile

CLog SLog

DBGen

SystemMonitor ResourceUsage

SProjectProfile

GeneratedData

TargetDB

<Legend>

System

Process

Connectors

SocketCommT

JNIConnT

DataAccessT

ProcessControlT

JDBCConnT

Repository

Database

File

Client Side Server Side

SQuerySet

StreamCommT

Figure 3 Top level runtime view of DBAuditor2

According to 3, we can find out two most significant changes; one is the communication between

the client and the server, and the other is the number of processes in the client.

We introduce TCP/IP socket communication in order to promote performance in terms of latency.

Because the legacy system uses RMI communication to promote simple and transparent

communication between modules on client and the server, it has some drawbacks on latency of

communication. RMI is using TCP/IP socket communication in low-level. Therefore, RMI is an

abstraction of TCP/IP which generates more overhead. Thus, we decided to use TCP/IP socket

communication instead of RMI communication.



We have done an experiment in order to figure out the difference between RMI communication and

TCP/IP socket communication.

Table 1 Java RMI vs. TCP/IP socket

Data length Socket RMI Ratio10 7.34 sec 10.90 sec 1.48

100 2.56 sec 10.86 sec 4.25 1000 2.56 sec 11.10 sec 4.33

10000 9.85 sec 23.02 sec 2.34 100000 85.15 sec 144.59 sec 1.70

In this experiment, we implemented a server and a client. The server just sends a string of

predefined length to the client. Normally, socket communication shows shorter completion time. For

more information such as experiment environment, refer to Database Transaction Experiment

document [2].

We can think about two candidates regarding to the number of client processes; we can create a new

process or a new thread that accepts system monitoring data. The responsibility of the process/thread

is to receive system monitoring data from the server, and to send the information to a user interface.

The communication between processes cost more than the communication between threads because

threads can share memory more easily than processes. For this reasons, we merged two processes into

single process.

In 3, two TCP/IP socket communication channels are introduced. One channel is used to send and

receive information from the client to the server, and the other channel is used to transfer system

monitoring information. The first channel is a duplex channel; the server can send information to the

client, and the client can send commands to the server. And the other channel is a simplex channel; the

system monitoring process in the server always sends, and the monitor process in the client always

receives. By this separation, modifiability can be improved because we have separate connectors: one

for protocol, and the other for streaming.

The DBAuditor2 project was created by the Telecommunication Technology Association (TTA).

TTA is an IT standards organization, and its services include providing one-stop services for the

establishment of IT standards as well as provides testing and certification for IT products. DBMS

testing tool tests various kinds of DBMS based on several Transaction processing Performance

Councils (TPC) standards.

TTA Software Quality Evaluation Center (SQEC) had been using TeamQuest; it is a commercial

DBMS testing tool. However, the tool is too heavy to test simple TPC standards because it provides

too many features; some of them are useless for the organization. TTA wanted to have lighter and

simpler tool; still it should have core important DBMS testing functionalities. According to that needs,

DBAuditor is developed last year.



Even though DBAuditor meets some of the needs, still the tool is not easy to use for testers in TTA.

Thus, the DBAuditor2 project is created to enhance mainly usability. However, the organization faced

some problems related to performance in terms of CPU usage and latency while using DBAuditor.

Now, performance becomes one of main purpose of the DBAuditor2 project.

The DBAuditr2 based on the previous project’s result. Figure 4 shows the context diagram of the

DBAuditor2. A tester requests for performance test for a target DB. The DBAuditor2 system queries

target database and gets the test result. While executing benchmark testing, system should obtain

system usage monitoring data. The DBAuditor2 system sends requests to kernel OS in order to get the

system monitoring data. When the DBAuditor2 reports the results to the tester, it sends test results

from target DB as well as system monitoring data.

1. Request for benchmark

Tester Client

DBAuditor2

2. Test execution

Target System

3. Execute TPC-C/TPC-Hbenchmark test DBMS

WAS3. Execute TPC-APPBenchmark test

4. Test result

5. Test report

6. Report benchmark result

DBMS under test

WAS under test

Figure 4 Overall context diagram of DBAuditor2



4. Architectural Drivers

4.1. Functional Requirements

Here are key functional requirements that the DBAuditor2 shall support. For more detail of

functional requirements, refer to SRS documents.

4.1.1. Project management

- Create/delete/modify/search project information.

- Import/export of project profile.

- Project profile includes base information, query information, and test information.

4.1.2. Project/Job Status

- DBAuditor2 shall display the list of the test results when a project is open.

- Display current status of the data generation and current job.

4.1.3. Query Generation and Execution

- Execute batch of queries.

- Cancel during batch processing.

- Show progress during batch processing.

4.1.4. Test Report Generation

- Generate a report, display it to screen, and store it into a file.

- Display system usage information in various ways of displaying.

- Generate canceled report when a test is canceled.

4.1.5. Wizard

- Provide wizard with TPC-C/H test.

4.2. Constraints

- DBAuditor is developed in Java programming language; it’s constraint because DBAuditor2

is based on DBAuditor.

- Using TCP/IP socket as a communication method between server and client is mandatory.

- DBAuditor2 shall be run on following operating systems: MS Windows, Linux 2.6.x, and HP

UX.

- DBAuditor2 shall support following DBMS: Oracle, MS SQL, DB2 and MySQL are

mandatory. Optionally, Informix and Sybase might be supported.



4.3. Quality Attribute Requirements

The DBAuditor2 shall fulfill the following quality attributes.

4.3.1. Performance

The main purpose of the DBAuditor2 is to benchmark various DBMSs’ performance. The

benchmark results would be meaningless if the reported CPU and Memory usage is immensely

affected by benchmark tool itself. Regarding to the benchmark result, credibility is very important.

While executing the benchmark, the DBAuditor2 shall minimize influences on the system where

benchmark tool runs on.


tester connects database. Schema creation and data generation is already done. The tester now

executes queries according to TPC-C. While executing queries, CPU usage of the system

where the DBAuditor2 runs on becomes 100%. Data transaction process should be separate

from the DBAuditor2. In this situation, the process of the DBAuditor2 must occupy less than

5% of CPU and Memory usage. [Environment of this response measure will be added after

discussion with the client.]


tester connects database. Schema creation is already done. The tester now generates test data

that will be inserted into DBMS. The generation time is measured; after the tester clicks

‘Generate test data’ button, generation time starts, and it ends just after generation is

completed. This generation time shall be within 10% of difference compared to the generation

time of DBGEN which is provided by TPC .[Issue]

4.3.2. Usability

The DBAuditor2 shall provide an easy way of executing tests. In TTA, testers perform benchmark

task, spending several hours or days for a test. Thus, the DBAuditor2 shall provide an easy way of

installation and test execution, intuitive menu, good learnability and comfortableness. DBAuditor2

shall satisfy end users in terms of easy-to-use. [Issue: how to meaure the usability? With survey like

LikertScale? Or ]

Availability

The DBAuditor2 shall not be shut down during tests because correctness of benchmark result is

very important. If a fault is occurred, the DBAuditor2 shall automatically cancel the testing, notify the

failure of testing and roll back that testing to “ready-to-go*” status, according to TPC specifications

(need guide)*ready-to-go: in this status, all parameters for testing environment are set so that a tester can start the

test by clicking start button.



Modifiability: change of query sets

The DBAuditor conducts DBMS performance testing according to TPC-C/H specification for

various DBMSs. However, the specification can be updated, and as well the DBMS such as Oracle and

MySQL can be newly released in the future. For the updates of TPC-C/H specification and the release

of DBMSs, the queries which are used for performance test can be changed. The tester who is skilled

in various kinds of DBMS can modify the sets of testing queries without affection to or modification

of the DBAuditor itself within 1 man-day.

Modifiability: change of specification

With the DBAuditor2, the client provides two major benchmark services to a DBMS. Though they

currently performs TPC-C/H test with the DBAuditor2, according to TPC specification, they might

need to modify TPC-C/H test process by updates of TPC-C/H specification. This modification of

features shall be done by a single developer who is skilled in TPC-C/H specification and understands

the updates of the specification in 7 man-days.

Modifiability: addition of new benchmark service

With the DBAuditor2, the client provides two major benchmark services to a DBMS. Though they

currently performs TPC-C/H test with the DBAuditor2, according to TPC specification, they might

need to add other kinds benchmark services. These additions of benchmark services shall be done by a

single developer who understands the specification of newly added benchmark service in 90 man-

days.



Architectural Decisions

Architectural Style

Client-Sever

The architecture of DBAuditor2 system basically adopts client-server style. An end user uses the

client application on his own personal computer and the sever application will run on a separate sever,

so client-server architectural style is inevitably adopted. In this architecture, the client and the server

use one simple TCP/IP socket communication channel. Because the legacy system of DBAuditor uses

RMI communication to promote simple and transparent communication between modules on client

and the server, it has some drawbacks on latency of communication. RMI is using TCP/IP socket

communication in low-level. Therefore, RMI is an abstraction of TCP/IP which generates more

overhead. Thus, we decided to use TCP/IP socket communication for DBAuditor2 instead of RMI

communication.

Server

DataRule

Client

CQuerySet

SystemConfig

CProjectProfile

CLogSLog

DBGen

System Monitoring ResourceUsage

SProjectProfile

GeneratedData

TargetDB

<Legend>

System

Process

Connectors

SocketCommT

JNIConnT

DataAccessT

JDBCConnT

Repository

Database

File

SQuerySet

Client Server

Figure 5 Top level runtime view of DBAuditor2

In addition, the legacy DBAuditor uses the other communication channel for monitoring process.

The major reason for this separation of RMI communication was to remove an effect of monitoring the

system usage on the benchmark test, so as to satisfy accuracy quality attribute. That is, If one RMI

communication is shared by performance test and monitoring the system usage,, the results of the

performance test may be affected by the system usage monitoring. On the contrary, the system usage



monitoring may be affected by the performance test, even though testers want to monitor the system

usage in a real time

Client Application2

Client

Client Application1

Server Application1Server Application1

ResourceUsage

TestDBTestDB

Server Application2

Rule/Query/Schema

RepositoryConnT

JDBCConnT

RMIConnT

DB

File

<Legend>Application Connectors RepositorySystem

Server

Figure 6 Top level runtime view of the legacy DBAuditor

. For these reasons, one RMI communication is divided into two RMI communications and both of

the accuracy of the performance test and real time monitoring of the system usage can be satisfied

with two separated RMI communications. However, when adopting TCP/IC socket instead of RMI,

the accuracy might be promoted because the monitoring process affects little to the server’s computing

time.

Implicit invocation: publish-subscribe style[TBA]

In the both of the client and the server, we adopt implicit invocation architectural style, especially

publish-subscribe

There are several reasons we adopt this publish-subscribe style.

The most important factor that leads us to this decision is that there are complex interactions among

modules in legacy system; more complex interactions will be added to implement newly-added

requirements. Publish-subscribe style helps us to add/modify/remove a module easier.

Besides, there are some gaps between documentations and code. When considering that we need to

refactoring, that will be problematic in following implementation phase if the interaction among

modules is complex.

Thus, the publisher-subscriber architectural style might be applicable to this situation.



4.4. Tactics

4.4.1. Performance: Resource Demand

Our system will be implemented with Java because the target system has to be run on various

operating systems. Even though Java supports various running environments, usually Java inhibits the

performance of the target system because Java requires computational overhead. We adopted the

resource demand tactic to reduce the computational overhead.

- TCP/IP socket communication: The legacy system uses Java RMI for communication between

the server and the client. Java RMI is very convenient and simple communication method, but

it consumes much resource than TCP/IP socket communication. We have performed an

experiment to figure out which method is suitable for the target system. We will use TCP/IP

socket communication according to the result of the experiments.

- Java Native Interface: A program complied with Java must run on a Java virtual machine.

Therefore, usually Java-based programs consume much resource than other language-based

programs such as C or C++. To address this problem, we decided to use the Java Native

Interface (JNI) in hotspot codes. By using JNI, we expect that the target system will be faster

than the legacy system.

4.4.2. Modifiability: Localize modifications

The legacy system uses explicit call-returns. Because of that, it is hard to modify the legacy system.

If we modify a module, it affects on other related modules in most cases. To avoid this problem, we

will use an event-bus for implementation.

- Publish-subscribe style: An Publish-subscribe style promote the modifiability of the target

system because each module does not depend on other modules much. By maintaining

semantic coherences among modules, we can achieve the modifiability.

4.4.3. Modifiability: Defer binding time

The main purpose of the target system is to benchmark various database management systems and

operating systems. Thus, the target system shall support detail configurations to promote the

modifiability of the system.

- System configuration: As we mentioned above, the environment of the target system is varied.

We adopted the deferring binding time tactic to solve this problem. The target system will be

executed according to the system configurations; the binding will be done at runtime rather

than at compile time.

- User defined SQL statements: Even though most DBMSs support the standard specification of

SQL statements, some DBMSs make a problem with the standard SQL statements. Therefore,

we separate the SQL statements from the source code. The users can define their own SQL

statements according to their circumstances.



4.4.4. Usability: Separation of UI

In many cases, users want to change user interfaces frequently. If the user interface is tightly

coupled with the logic of the system, we cannot change the user interface easily. Therefore, we

adopted the separation of UI tactic. By adopting this tactic, we expect that the target system will

promote the usability.

- Status Manager: To separate the UI from the logic, we added a status manager which

manages the status of the system. Because the UI interact with only this manger, if we keep

the semantic coherences among the modules, we can change the UI without affecting on other

logic modules.

4.4.5. Usability: Support User Initiative

As we mentioned above, the target system runs on various environments and has to support various

DBMSs. To accomplish this goal, we adopted the support user initiative tactic.

- System configuration & User defined queries: The target system promotes the usability by

supporting user defined configurations. We expect that users can configure the system easily

and can benchmark whatever they want.

4.4.6. Portability

The portability is one of most important quality attributes because the target system runs on various

operating systems. To promote the portability of the system, we adopted the Java as a development

language because Java supports various OSs and various DBMSs through JDBC.

4.5. Other decisions

4.5.1. Project profile repository

One of the requirements of the system is managing project profile data. The project profile includes

general information of testing project and the result of each benchmarking test. Because the general

project information is to be managed by the client and the test results are to be managed by server,

there is an issue that which side should mainly manage the project profile data.



ServerClient CProjectProfile

SProjectProfile

<Legend>

System

Process

Connectors

DataAccessT

Repository

File

Client Server

Figure 7 The location of project profile

Most of controls in the testing process are performed on the client side. To start a benchmark test

project, a tester needs to initiate the test on the client side. In addition, to preserve the test result while

a test is being performed, the sever needs to save the data temporarily on it and the result of each

testing should be transferred from the server to the client after each test is finished. Thus, to reduce

communication traffics, it might be applicable that the client takes charge of managing the project

profile data.

The customer wants to share the project profile among testers, the latest versions of the project

profile also need to be stored on the server, which triggers a synchronization issue; the synchronization

might be up to the end user when it is necessary. This issue should be covered when the follow-up

design is performed through negotiation with the customer.

4.5.2. Comparison[TBD-figure for comparison]

The target system is based on the legacy system. It is important to figure out the strengths and

weaknesses of two systems, the target system and the legacy system, in terms of critical issues

mentioned above. By analyzing those advantages and disadvantages, then we can confirm the

architecture of the target system.



5. C & C Architectural View

The Component & Connector (C&C) view-type decomposes the system into components that have

some runtime presence such as processes, objects, storage and connectors or that represent pathways

of communication such as information flows, and access to shared storage. These components and

connectors are the elements represented in the view-types[3].

This view-type was selected because it helps the following roles[4]:

The software architect and project manager can argue and reason about architectural

properties and quality attribute requirements that the system must adhere to.

The software architect, programmer and tester can infer progression of data through the

system and how the structure of the system changes as it executes.

External stakeholders like customers, project evaluators can understand the system’s

principal executing components (including the major shared data sources) and their

interactions – therefore serving as a means for verification and validation of system

properties.

Maintainers of the project can get an overview of the system as a starting point of future

extensions and / or modifications.

The view-type has been represented using a combination of the call-return and publishsubscribe

styles because separating these two predominant styles into different views would have reduced the

understandability of the system as a whole. The interactions of the different components in this view-

type warrant the combination of the styles and they individually adhere to the rules of the

aforementioned styles.

5.1. Overall C&C View

The architecture involves four processes; one is in the client side and other three are contained in

the server side. The client side and the server side communicate on socket communication method.

Between the client and the server, there are two socket communication channels. One is to execute test

on the TargetDB and another one is to report system resource usage.



Server

DataRule

Client

CQuerySet

SystemConfig

CProjectProfile

CLog SLog

DBGen

SystemMonitor ResourceUsage

SProjectProfile

GeneratedData

TargetDB

<Legend>

System

Process

Connectors

SocketCommT

JNIConnT

DataAccessT

ProcessControlT

JDBCConnT

Repository

Database

File

Client Side Server Side

SQuerySet

StreamCommT

33]

Figure 8 C&C View of the system

Element Description

SystemConfigSystemConfig storage contains system configuration information that consists of server

address, default DB information and DB connection driver.

CProjectProfile

CProjectProfile storage in the client side is a file that contains project profiles. The

project profile consists of project name, project open date, target database name, and

test profile.

DataRuleDataRule storage contains a set of rules for generating data which is used to test the

TargetDB.

CQuerySet

CQuerySet storage contains a set of queries in the client side. In this storage, there are

multiple sets of queries for various kinds of data base; standard SQL query set, Oracle

query set, MySQL query set and user defined query set(s). Since client side’s query sets

are primary query sets between client and server side, users should be able to manage

these query sets directly and are responsible for synchronizing these query sets to

server side’s query sets, SQuerySet.

CLogData CLogData storage stores log data of the client side.

SLogData SLogData storage stores log data of the server side.

SQuerySet

SQuerySet storage contains a set of queries in the server side. In this storage, there are

multiple sets of queries for various kinds of data base; standard SQL query set, Oracle

query set, MySQL query set and user defined query set(s).



SProjectProfile

SProjectProfile storage in the server side is a file that contains project profiles. The

project profile consists of project name, project open date, database name, test profile,

and [TBA].

GeneratedDataGeneratedData storage contains data that is generated by DBGen and used for test

execution.

Server Server is responsible for generating test data, executing test and monitoring CPU usage.

ClientClient is responsible for user interaction, managing project profile, managing query sets

and configuring system.

TargetDB

TargetDB is the target database to be tested. TestExecuter and MassiveDataUploader

send queries through JDBC caller to use the TargetDB regardless of any kinds of

database.

ResourceUsage ResourceUsage storage contains resource usage information.

DBGenDBGen is an external process that is developed by TPC organization. DBGen generates

TPC-H data and stores it into GeneratedData storage.

SystemMonitor

SystemMonitor is a process that plays a role for monitoring resource usage of server

sides. To reduce affections on the Server’s performance and resource usage, this

process is operated independently to the Server

Table 2 Description of Overall C&C view



5.2. Server

The server side is consists of three processes: server application process, resource monitoring

process and DBGen process. The server application process is the main application to perform actual

testing. To do that, server application generates massive test data through DataGenerator for TPC-C

test and delegate generation of test data to DBGen process through JNI. This is because DBGen is

known as the most optimized in generating TPC-H test data and TPC recommends employing it to

promote generation performance.

The resource monitoring process is to monitor resource usage of the server during executing test.

ResourceMonitor component may request system resource usage information to OS when OS provides

the information. Or, it should contain a module that looks up the resource usage and generate related

information.

<Legend>

System

Process

Threads

Manager

Agent

CommHandler

Executer

Connectors

JNIConnT DataAccessT

ProcessControlT

JDBCConnT

StreamT

Repository

Database

File

EventBusT

Rec

ieve

r

Ann

ounc

er

SC

om

mun

icat

ion

Han

dle

r

SProjectManager

SLog

TestExecuter

MassiveDataUploader

DataGenerator

SStatusManager

DBGen

SStreamCommHandler

ResourceUsage

SResourceMonitor

SProjectProfile

GeneratedData

TargetDB

Server Side

SQuerySet

SLogRecorder

1...n

Figure 9 C&C view of the server

Element Description

SLogRecorder SLogRecorder is responsible for recording logs of EventBus in the server side.

SCommunicationHandler SCommunicationHandler plays a role of socket communication with the client side.



SCommunicatioHandler receives/sends packets from/to the client side, transforms

messages into a corresponding event with parameterized data, and announces it in

EventBus.

MassiveDataUploaderMassiveDataUuploader extracts generated data for test from GeneratedData and

uploads those data to the TargetDB according to the specification of TPC.

DataGeneratorDataGenerator invokes DBGen to generate data for TPC-H, and generates data for

TPC-C.

SStatusManager SStatusManager generates the server side’s status and report it to the client side.

TestExecuterTestExecuter is to execute sends a set of SQL statements to the TargetDB, and

announces the results of each execution.

SProjectManager

SProjectManager is a project manager in the server side. There is another project

manager in the client side. SProjectManager synchronizes ProjectProfile and

QuerySet storage with client side’s ProjectProfile storage and QuerySet storage.

JDBCCaller

JDBCCaller is a thread that connects the TestExecuter and the TargetDB via JDBC.

In case of TPC-C test, according to TPC specification, multiple VirtualTerminal

shall be run and connect to the TargetDB through respective JDBCCaller.

Table 3 Description of C&C view of the server

5.2.1. TestExecuter

The TestExecuter thread executes queries to create tables to the target database via JDBC, to

perform TPC-C test with multiple virtual terminal, and to perform TPC-H test. Since common vehicle

to pass messages or to request a service is based on the event bus, the JobDistributor handles event

subscribing and publishing, and classify events for SchemaBuilder, TPCCExecuter and

TPCHExecuter. While any one of these functions is working, rest two functions should not work.

Thus, JobDist invokes each executer components via the call-return connection. When each executer

component executes its function



VirturalTerminal2VirturalTerminal2

SchemaBuilder

TPCCExecuter VirturalTerminal

TPCHExecuter

JobD

ist

<Legend>Threads

Executer

Connectors

EventBusT

Reciever Announcer

CallReturnT

JobDistibutor

VirtualTerminal

ThreadInvokeTTestExecuter DataAccessT

JDBCConnT

Figure 10 Detail View of TestExecuter

Element Description

JobDistributor

JobDistributor is a thread that distributes job to the SchemaBuilder, the

TPCCExecuter and the TPCHExecuter. JobDistributer scribes all of events to

execute a query and publishes the result of execution of schema building, TPC-C

testing and TPC-H testing.

VirtualTerminal

VirtualTerminal is a thread which acts like a virtual terminal for the TargetDB.

According to TPC-C specification, VirtualTerminal mimics real database users’

actions, and multiple VirtualTerminal will be thrived to emulate real database

environment.

SchemaBuilder SchemaBuiler is responsible for creating tables into target database through JDBC.

TPCCExecuter

TPCCExecuter is responsible for executing TPC-C test against the target database

through JDBC. According to TPC-C specification, TPC-C test shall be done by

multiple virtual terminal so that the TPCCExecuter emulates real terminal user’s

behavior.

TPCHExecuterTPCHExecuter is responsible for executing TPC-H test against the target database

through JDBC

Table 4 Description of TestExecuter



5.3. Client

In Client side, there is a single process, Client process. The Client process contains User Interface,

CProjectManager, SystemConfigManager, TestCommander, DataRuleManager, QuerySetManager,

CStatusManager, CCommunicationHandler, CStreamCommHanlder, and CResoureMonitor. The main

connector among these components is an event bus. All messages and service requests are connected

via the event bus except resource monitoring function.

<Legend>

System

Process

Threads

Manager

Agent

UserInterface CommHandler

Connectors

DataAccessT

Repository

File

EventBusT

Reciever Announcer

DataRule

UserInterface

CProjectManager

SystemConfigManager

CStatusManager

TestCommander

DataRuleManager

QuerySetManager

CLogRecorder

CC

om

mun

icat

ionH

and

ler

CQuerySet

SystemConfig

CProjectProfile

CLog

Client Side

CStreamCommHandler

CResourceMonitor

StreamT

Figure 11 C&C View of the client

Element Description

UserInterface UserInterface handles user interactions, and conveys user requests to the EventBus.

SystemConfigureManagerSystemConfigureManager stores, modifies and loads system configuration

information into SystemConfig storage.

DataRuleManager

DataRuleManager creates, revises, updates and deletes (CRUD) data rule in

DataRule storage. DataRuleManager have three ports: announce port, receive port,

and user port. Through the receive port DataRuleManager receives events for

CRUD requests from EventBus. Through the announce port DataRuleManager

announces events to EventBus for CRUD finish-up. Through use port,

DataRuleManager stores and reads Data Rule in DataRule storage with the results

of CRUD tasks.



TestCommander

TestCommander interprets user’s command into packet units to request execute

test. TestCommander associates with three UI tasks: SchemaBuilding,

QueryExecution, DataGeneration. For example, in SchemaBuilding, once user

requests schema building, the user interface announces SchemaBuilding event on

EventBus; TestCommander receives schema building event; TestCommander

requests corresponding query set to QuerySetManager via EvenBus;

QuerySetManager conveys queries on EventBus to deliver queries to

TestCommander; TestCommander transforms queries into packets and announces

‘send schema building queries’ event to EventBus.

QuerySetManager

QuerySetManager creates, revises, updates and deletes query and query set in

QuerySet storage in the client side as well as exports/imports query and query set

to/from QuerySet Storage in the server side.

CLogRecorder LogRecorder records logs of client side; all user interactions and all events

CCoummunicationHandler

CCommunicationHandler sends packets to the server, and receives packets from

the server. CCommunicationHandler transforms a packet received from the server

into a corresponding event and data, and announce it with parameterized data.

CProjectManager

CProjectManager creates, revises, updates and deletes project profile in

ProjectProfile storage in client side. As well, ProjectManager exports/imports

query and query set to/from QuerySet Storage in the server side

CStatusManager

StatusManager gets the status of DBAuditor2 from the server side’s

SStatusManager through socket communication; status of test execution, status of

schema building and status of data generation.

ProjectProfileProjectProfile is an object that contains project profile information in the client

side.

QuerySet QuerySet is an object that contains query set information in the client side.

ResourceMonitor ResourceMonitor is an agent to look up system usage.

Table 5 Description of C&C view of the client



6. Module Architectural View

The Module view-type partitions the system into a unique non-overlapping set of hierarchically

decomposable implementation units (modules). The goal is to show how the modules are decomposed,

as well as the dependencies between modules[4].

This view-type was selected because it helps the following roles[4]:

The architect, who must define work assignments in such a way so as to minimize

dependencies between the modules and assign priorities (or sequences) to modules to

control existing dependencies,

The project manager, who must form teams, formulate project plans and schedule,

knowing the individual priorities (or sequences) of these modules,

Testers who use the modules as their unit of work to create test cases and perform the tests,

The configuration manager who is in charge of maintaining current and past versions of the

units in consistent and functional package-able assemblies, being able to produce a running

version of the system,

Developers, who are required to implement the modules,

Maintainers, who are tasked with modifying the software modules,

The view-type has been represented using the decomposition style. The decomposition style

represents the decomposition of the code into systems and subsystems representing a top-down view

of the system. This is important to the development team to understand their roles in terms of code

development and can be used as the basis of work assignments and completion measures [3][4].

The module view of DBAuditor2 system consists of 2 top-level modules: client and server. The

three main elements are identified at top level view. Each element is described below.



6.1. Representative Module View of the Server

ServerMain

ServerEventBusPackage

SeverControlPackage

initailize≪ ≫

extend≪ ≫

initailize≪ ≫

extend≪ ≫

UsageMonitorPackage

ServerSystemMonitor

invoke≪ ≫invoke≪ ≫

Figure 12 Top-level module view of Server

Element Description

ServerMainThis is the main class of the server, which invokes all control modules in the server and

initializes them to be ready.

SeverControlPackage

This package contains all logical control modules on the server. This package depends

on EventBustPackage because each control module extends the classes in the

EventBusPackage to implement the event-bus communication. One of control module,

TestExecutor, shall initiate ServersystemMonitor module.

EventBusPackage This package contains java classes which relate to implementing event bus.

UsageMonitorPackage This package includes modules monitoring resources of the server.

Table 6 Description of Top-level module view of server



6.2. Detailed Module View: Sever

The detailed module view of the server side is shown below. As we can see, TestExecutor,

ProejctProfileSynchronizer, ServerStatusManager, DataLoader, ServerCommunicationManager,

SeverLogManager, and DataGenerator are dependant on ServerEvnetBusPackage; they extend classes

in the ServerEventBusPacke to implement event-bus based communication.

ProjectProfi leSynchronizerT estExecutor DataLoader DataGeneratorServerStatusManager ServerCommunicationManager ServerLogManager

T estResultData≪ ≫

11

QueryDataData≪ ≫

1

1

1

1

1

DataGenerationRuleData≪ ≫

0..1

1

0..1


extend≪ ≫extend≪ ≫

ProjectProfi leData≪ ≫

1

1

1

*

1

1

1

*

DBGenExternal≪ ≫

invoke≪ ≫invoke≪ ≫

*

VirtualT erminal

*

JDBCInterface≪ ≫

1

1

SocketHandler

1

1

UsageMonitorPackage

ServerSystemMonitor

invoke≪ ≫

MonitorAppExternal≪ ≫

invoke≪ ≫

MonitorSocketHandler

1

ResourceUsageData≪ ≫

1

invoke≪ ≫

invoke≪ ≫

1

1

Figure 13 Detailed module view of Server

Element Description

TestExecutor

This module is responsible for executing the benchmark process on the server; the

process includes building schema, generating data and, and recording the execution

results of the target DBMS being benchmarked.

ProjectProfile

Synchronizer

The main purpose of ProjectProfileSynchronizer is to synchronize the server-side

project profile data with the client-side project profile data; the project profile data will

be mainly maintained in the client and will be stored also on the server for backup and

sharing.

ServerStatusManagerThis module maintains the status of the server. When the status changes the module

will notify it to the client.

DataLoaderIt loads massive data, which are prepared for benchmarking a DBMS on the server, to

the DBMS so that the system is ready to start the benchmark.

ServerCommunication

Manager

This module is responsible for all communications between the server-side modules

and the client side’s.

ServerLogManager This module logs the events generated by other modules on the server; the log data



will be stored in an external file on the server.

DataGenerator

This module generates data which are used to benchmark a DBMS; the data will be

stored in an external file. And it invokes DBGen to generate TPC-H benchmark test

data.

VirtualTerminal This module is responsible for simulating real users according to TPC-C specification.

ServerSocketHandler This module is responsible for TCP/IP communication with the client.

<<Data>>ProjectProfile

ProjectProfile is a data structure to maintain the project data; it shall include zero or

more TestResult data.

<<Data>>TestResult

This is a data structure which maintains the benchmark results of the target DBMS. It

includes test date, executed query, its execution time, test environment, and [TBD]

<<Data>>QueryData

QueryData is a data structure which contains query sets; it might be predefined query

set or user defined query set.

<<Data>>DataGenerationRule

This is a data structure which contains a set of rules for generating data. It might be

not instantiated if default data generation rules are used in an external data generation

application, for instance, DBGen.

<<Data>>ResourceUsage

This is a data structure that contains system usage data; it includes memory usage and

CPU usage rate; it shall be transferred to the client by ServerSystemMonitor which is

run on separated process. Because this data shall be transferred to the client directly by

the ServerSystemMonitor, it shall not be associated with TestResult data on server side

but once it was transferred it will be aggregated as a part of test result information on

the client side.

ServerSystemMonitor

This module reads server’s resource usage data periodically and sends them to the

remote client so as to inform the usage data to the end user; it shall be implemented as

separated process from the server process.

MonitorSocketHandlerThis module shall implement TCP/IP communication with the client for the system

monitoring module.

<<External>>MonitorApp

This is an external application invoked by ServerSystemMonitor through JNI; it shall

generate system resource usage information according to ResourceUsage data

structure: the resource might include CPU and Memory.

<<External>>DBGen

This is an external application invoked by DataGenerator through JNI; it will generate

data for TPC-H benchmark test.

<<Interface>>JDBC

This is an external module for communicating with a target DBMS.

Table 7 Description of detail module view of server



6.3. Representative Module View of Client

UIPackage ControlerPackage

ClientMain

initailize≪ ≫initailize≪ ≫ initailize≪ ≫initailize≪ ≫

extend≪ ≫

ClientEventBusPackage

extend≪ ≫extend≪ ≫ extend≪ ≫

Figure 14 Top-level module view of Client

Element Description

ClientMainThis package is the main class of the client, which invokes all control modules in the

client and initializes them to be ready.

ControlPackage

This package contains all logical control modules on the client. The package depends on

ClientEventBustPackage because each control module extends the classes in the

ClientEventBusPackage to implement the event-bus based communication.

ClientEvent-

BusPackageThis package contains java classes which are related to implementing event bus.

UIPackage This package includes all of user interface classes.

Table 8 Description of top-level module view of client



6.4. Detail Module View: Client

ClientCommunicationManagerClientStatusManager ClientLogManagerQuerySetManager DataRuleManagerTestCommanderProjectManager SystemConfigurationManager

SystemMonitorUI SchemaComposeUI ProjectManagerUI

1

1

WizardUI

11

ReportUIStatusInfomationUI

11

QueryMangerUI

ReportManager

QueryDataData≪ ≫

1

11

1ProjectProfile

Data≪ ≫

1

1 1

1

1

TestResultData≪ ≫

1

1 0..1

*

1

*

DataGenerationRuleData≪ ≫ 11

0..1

ResourceUsageData≪ ≫

1

1

1

1

1

ClientSocketHandler

1


extend≪ ≫extend≪ ≫

DataRuleManagerUI

111

ProgressDispayingWindow

1extend≪ ≫

SystemConfigurationUI

extend≪ ≫

1

1

1

1

SynchProjectProfile

1

1

OpenProject

1

1

1

1

1

1

ViewHistory

1

1

CreateProject

1

1

ConnectionUI

1

1

1

1

1 ClientMain1

1 1 1 1 1

1 1 1 1 1 1 1 1

1 1 1

1

11 1 1 1 1

1 1 1 1 1 1 1 1

1 1 1

1

1

TestExecutionUI

11 11

ClientResourceMonitor

1

1

1

1

1

1

1

ClientStreamHandler

1

Figure 15 Detailed module view of Client

Element Description

QuerySetMangerThis module is responsible for creating, updating query and query set in QueryData

in the client side.

StatusManagerThis module is responsible for maintaining the current status of DBAuditor2 and

dispalying it to the end user through information windows.

LogManagerThis module is responsible for recording the history of user triggered events and

server’s responses while the user uses the DBAuitor2.

CommunicationManagerThis module is responsible for communicating with the server. It will transform data

object to a stream of string for sending it to the server and vice-versa.

ProejctManager

This module is responsible for creating and updating project profile data in

ProjectProfile in the client side. As well, it exports/imports project profile to/from

ProjectProfile in the server side to synchronize the ProjectProfile; this

synchronization shall be triggered by the end user.

TestCommander

This module interprets user’s commands of generating schema, generation data, and

executing test query. And it sends the commands to the server one by one in

accordance with benchmarking process.

SystemConfiguration

Manager

This module is responsible for maintaining system’s configuration information

which is stored in an external text based file.

ReportManagerThis module is responsible for viewing the benchmark results from TestResult data

and it transforms the TestResult data into external text based format.



DataRuleManager

This module is responsible for editing data generation rules, and it stores in

DataGenreationRule; because data generation rules are pre-defined in TPC-C and

TPC-H benchmark, this module is only used for user defined test. When defining

rules, a tester shall include CVS, email address, address, zip code, phone number,

and so on.

<<Data>>ProjectProfile

ProjectProfile is a data structure to maintain the project data; it is the same as the

data structure of the server.

<<Data>>ResourceUsage

This is a data structure that contains system usage data; it includes memory usage

and CPU usage rate; it is the same as the data structure of the server.

<<Data>>QueryData

QueryData is a data structure which contains query sets.

<<Data>>TestResult

This is a data structure which maintains the benchmark results of the target DBMS.

It includes test date, executed query, its execution time, test environment, and

[TBD].

<<Data>>DataGenerationRule

This is a data structure which contains a set of rules for generating data.

SystemMonitorUIThis module is responsible for displaying the server’s resource usage information

and the status data will be maintain by SystemManager.

StatusInformationUI

This module is responsible for displaying the current status of the DBAudtior2 and

the progress status of benchmark; this is a part of default main window and its status

data will be maintain by StatusManager.

QueryManagerUIThis module is responsible for interfacing with an end user so as to make the end

user edit queries.

ProgressDisplaying

Window

This window shall display current progress of a certain benchmark task which

might take a long time to perform, for example, data generation; the status of

progress will be maintained by StatusManager.

SchemaComposeUI

This module is responsible for interfacing with an end user so as to make the end

user edit DB schema or generate schema on the target DBMS. In case of supporting

DBMSs, DBAuditor2 shall provide proper query set for four DBMSs: DB2, Oracle,

MySQL, MS-SQL. For the other DBMSs, a tester shall create a new query set or

edit form predefined query set for the DBMS by this module. QuerySetManager

shall control this module or process the events relevant to SchemaComposeUI.

DataGeneratorUI

This module is responsible for interfacing with an end user to edit data generation

rules and initiating to generate data for benchmark according to user’s initiating

command.

SystemConfigurationUI

This module is responsible for interfacing with an end user to edit system property

file which contains system’s configuration information; server’s IP, DBMS type,

DBMS user account, and the name of jdbc driver particular to the installed DBMS.

WizardUI

This module is responsible for interfacing with an end user; it provides wizard style

user interfaces which include panels and frames related to TPC-C, and TPC-H

benchmark.

ProjectManagerUI This module is responsible for interfacing with an end user so as to make the end



user create new project or update current project profile.

ReportUI This module is responsible for showing test result information to an end user.

ClientSocketHandler This module shall implement TCP/IP communication with the server.

Table 9 Description of detiail module view of client

6.5. Detail Module View: EventBusPackages

The Logic control modules in server shall extend the ServerCommandEventHandler to implement

event-bus based communication; the client also uses same approach. Following two figures illustrate

the related package.


ServerCommandEventHandler

ServerEvent

ServerEventBus

11

Java.Utiljava≪ ≫

Observable

extends≪ ≫

Observer

extends≪ ≫

extends≪ ≫

extends≪ ≫

Figure 16 ServerEventBusPackage


ClientEvent

ClientEventBus

1

ClientCommandEventHandler

1

Java.Utiljava≪ ≫

Observable

extends≪ ≫

Observer

extends≪ ≫

extends≪ ≫

extends≪ ≫

Figure 17 ClientEventBusPackage



7. Allocation Architectural View

7.1. Deployment view

ClientMachine

ClientDBAuditor2

ServerMachine

SystemMonitor

ServerDBAuditor2

QGen

DBGen

TCP/IP Commands

Figure 18 Deployment view



7.2. Implementation veiw

Figure 19 Package structure for implementation



8. Mapping between Architectural Views

C&C View Element Module View Element

SLogRecorder ServerLogManager

SCommunicationHandler ServerCommunicationManager

MassiveDataUploader MassiveDataUploader

DataGenerator DataGenerator

SStatusManager ServerStatusMoniotor

TestExecuter TestExecutor

SProjectManager ProjectProfileSynchronizer

JDBCCaller DataGenerator, ServerSystemMonitor

JobDistributor TestExecutor

VirtualTerminal VirtualTerminal

SchemaBuilder TestExecutor

TPCCExecuter TestExecutor

TPCHExecuter TestExecutor

UserInterface

SystemMonitorUI, StatusInformationUI,

ProgressDisplayingUI, TestExecutionUI,

DataRuleManagerUI, SchemaComposeUI,

QueryManagerUI, ReportUI,

ProjectManagerUI, SystemConfigurationUI

SystemConfigureManager SystemConfigurationManager

DataRuleManager DataRuleManager

TestCommander TestCommander

QuerySetManager QuerySetManager

CLogRecorder ClientLogManager

CCoummunicationHandlerClientCommunicationManager,

ClientSocketHandler

CProjectManager ProjectManager

StatusManager ClientStatusManager

ProjectProfile ProjectProfile

QuerySet QueryData

ResourceMonitor ClientResourceMonitor

Table 10 Mapping table between C&C view and Module view



9. Future Works

TBD



10. Appendix

10.1. Acronym

10.2. Functional requirements

Table 11 Project Management

ID Item Description

SR001Information modification

Users can modify all project profile except for the project ID.

SR002 Search project Users can search projects by elements of project profile.SR003 Deletion project Users can delete a project profile.

SR004 Check ID duplicationEvery project must have different project ID from other projects. Otherwise DBAuditor shall display an error message.

SR005 Time/DateWhen users create a project, date and time shall be derived from the client system, and those value shall be filled at date and time fields of the project profile.

SR006 Disable "Close" button Menu "닫기 (Project close)" shall enabled only when a project is opened.

SR007 Search project

When users search some projects, the client system shall provide the status of the progress or a message such as "프로젝트 목록을 조회중입니다. 잠시만 기다리세요. (Now searching projects. Please wait a second…"

SR008 Search projectWhen users search previously made project, a wide message window is poped up. The width of this window shall be narrowed.

SR009 Project historyDBAuditor shall provide the list of previous four projects which have already been created or opened.

SR010 Import Users can put the project profiles to the server.

SR011 Export Users can get the project profiles from the server.

Table 12 Project Profile

ID Item Description

SR012 Project profileIt consists project base information, query information, and test information

SR013 Base informationIt consists of project ID, project name, creation date and time, OS, DBMS, server type, and tester

SR014 Query It consists of TPC-C query, TPC-H query, and user defined querySR015 Test information It consists test type, test start date, test end date, and test resultsSR016 Test type TPC-C / TPC-H / User definition

SR017 Test environmentThis information is based on the project base information. Users can change this information.

SR018 Test results These are defined at TPC-C and TPC-H specifications.SR019 Project save Project profile information is stored as a HTML file or a XML file.

SR020 Project saveProject profile information is stored in a folder of which name is the project ID and the parent folder is defined in DBAuditor.

SR021 Project loadA project profile is loaded from a fold which is defined at DBAuditor.

SR022 Location Project profile is stored in the server.



SR023 Test resultsDBAuditor shall display the list of the test results when a project is opened.

SR024Show specific information

DBAuditor shall display all project profile base information.

SR025 Close projectDBAudtior shall notify with popup screen and clear it when users close a project.

Table 13 Project Status

ID Item Description

SR023 Test resultsDBAuditor shall display the list of the test results when a project is opened.

SR024Show specific information

DBAuditor shoud display all project profile base information.

SR025 Close projectDBAudtior shall notify with popup screen and clear it when users close a project.

Table 14 Query Generation and Execution

ID Item Description

SR026 QueriesDBAuditor shall provide TPC-H queries and TPC-C queries according to the DBMS type.

SR027 Modify queriesUsers can modify all queires which have been defined by developers or users.

SR028 Batch executionDBAuditor shall provide batch execution of queries according to the test type.

SR029 Pause/ResumeDuring query batch processing, pausing and resuming functions shall be provided.

SR030 Cancel During query batch processing, canceling function shall be provided.

SR031 ProgressDuring query batch processing, the current job status shall be displayed.

Table 15 Test Report

ID Item Description

SR032 Generate reportThe output method of the test results can be HTML file, XML file, CSV file, or the screen.

SR033 Format reportA test report shall include all elementes which can be gathered from a test.

SR034 Search reportUsers can search test reports by test type or project profile base information.

SR035 Delete report Users can delete test reporst from a project profile.SR036 Load report Saving report function is integrated into project save function.SR037 Save report Loading report function is integrated into project load function.

SR038 Resource graphDBAuditor shall display system usage information in various way of diplaying.

SR039 Canceled reportIf a test is canceled, DBAuditor shall generated canceled report with notification.

Table 16 Job Status

ID Item Description



SR040 Data generation DBAuditor shall display the current status of the data generation.SR041 Current job DBAuditor shall display what is current job clearly.

Table 17 Wizard

ID Item Description

SR042 Wizard Users can perform the TPC-C/H test with a benchmark wizard.

SR043 Pause/ResumeDuring query processing, pausing and resuming functions shall be provided.

SR044 Cancel During processing, canceling function shall be provided.SR045 Status The current status of benchmarking shall be displayed.

Table 18 Configuration

ID Item Description

SR046 Default IP setting The default IP shall be "127.0.0.1," not specific IP.

SR047 Save configuration The client shall provide a "현재 설정 저장 (Save current setting)" check box to save current IP value.

Table 19 Connection Control

ID Item Description

SR048 IP history The client shall provide IP histories which were inputed from users.

SR049 Project listAfter establishing connection with the server, the client shall display all projects located in the server to open immediately.

SR050 Connection close The client shall provide "닫기 (Close project)" menu to close the currently opened project.

SR051 The script of menuAt first time, the script of "재연결 (Reconnection)" menu in the client shall be changed as "연결 (Connection)."

SR052Remove auto connection function

The client shall not try to connect with the server.

Table 20 Environment

ID Item Description

SR053 OSThe server shall be run on various OS such as MS Windows, Linux2.6.x, and HP UX. And the client shall run on MS Windows.

SR072 System statusDBAuditor2 shall log its operations so as to anlyze problems caused a system fault.

SR077 TPC-C/HThe latest version of TPC-C/H specificaiton shall be implemented in the DBAuditor2, the version shall be explicitly noted in the artifacts of DBAuditor2.



10.3. Prioritized Quality Attribute Scenarios

Note that the following quality attribute scenarios are enumerated by its priority

Table 21 Quality Attribute Scenario 1Scenario Refinement for Scenario 1

Scenario(s):

The DBAuditor2's primary purpose is to benchmark various DBMSs. The benchmark result would be meaningless if the reported CPU and Memory usage is affected by benchmark tool itself. Regarding to the benchmark result, credibility is very important. While executing benchmark The DBAuditor2 shall minimize influences on DBMSs. The DBAuditor2 must occupy less than 5% of CPU during test execution.

Business Goals:TTA should provide accuracy benchmark result which is not affected by DBAuditor2 CPU usage.

Relevant QualityAttributes:

Performance(resource)

Sce

nar

io

Com

pon

ents

Stimulus: The system uses the least resources of DBMS server.Stimulus Source: Server(DBMS)

Environment: During test execution.Artifact (If Known): System

Response: The system shall be executed in least resources of DBMS server.ResponseMeasure:

less than 5% of CPU usage of the DBAuditor2 during test execution

Questions:

Issues:

How do we know CPU usage is under 5%? In what conditions (what kinds of machine and what kinds conditions)? Just test execution?Our Suggestion.

EnvironmentMachine: acceptance test machine.OS, DBMS

Benchmarking parameter- Test type (C/H/User Define), Test Options (Scale Factor, Warehouse,

Terminal#...)


Scenario(s):

A tester starts the DBAuditor2. After boot up, the tester connects database. Schema creation is already done. The tester now generates test data that will be inserted into DBMS. The generation time is measured; after the tester clicks ‘Generate test data’ button, generation time starts, and it ends just after generation is completed. This generation time shall be within 10% of difference compared to the generation time of DBGEN which is provided by TPC.

Business Goals: TTA should benchmark target DBMS within specified test period.Relevant Quality

Attributes:Performance(generation time)

Sce

nar

io C

omp

onen

ts

Stimulus:Starts to execute data generation, customer request, and competitiveness to TPC's DBGEN.

Stimulus Source: ClientEnvironment: During generating test data time

Artifact (If Known): Data Generation componentResponse: Data generation shall complete.ResponseMeasure:

Within 10% of difference compared to the execution time of DBGEN provided by The TPC.

Questions:

Issues:DBGEN supports only TPC-H. How we compare generation time of DBAuditor for TPC-C?




Scenario(s):

In TTA, testers perform benchmark task, spending several hours or days for a test. Thus, the DBAuditor2 shall provide an easy way of installation and test execution, intuitive menu, good learnability and comfortableness. DBAuditor2 shall satisfy end users in terms of easy-to-use.

Business Goals: Testers in TTA Relevant Quality

Attributes:Usability

Sce

nar

io C

omp

onen

ts Stimulus: Users want to use system efficiently, learn system feature, feel comfortable.Stimulus Source: End user.

Environment: During execution time or configure time.Artifact (If Known): The client system.

Response:DBAuditor2 support help system to end users so that they can learn about DBAuditor2 easily. DBAuditor2 support efficiency of using system to end users. Also, DBAuditor2 provides comfortable menu and system status.

ResponseMeasure:

In survey based on Likert scale, satisfaction score of DBAuditor2 usability shall be over 4 out of 5 from user feedback.

Questions:Issues:


Scenario(s):

Due to the correctness of testing, the DBAuditor2 shall not be shut down during test execution that may carry over 2 or more days. If a fault occurs in the course of long test execution, the DBAuditor2 shall cancel the testing, notify the failure of testing, and turn back the testing to ready-to-go status so that testers perform test again right away, according to TPC specifications (need guide)

Business Goals: TTA testers perform test execution efficiently as fast as possible.Relevant Quality

Attributes:Availability

Sce

nar

io C

omp

onen

ts

Stimulus: The server system should not be crashed during testsStimulus Source: The benchmark specification

Environment: Normal and degraded modeArtifact (If Known): Processes, communication channels, and storage.

Response:

If the server system works on tests, this system should not be affected by crash of the client system and disconnection of communication. The server system records system status and job status, and continue to operate in normal or degraded mode.

ResponseMeasure:

DBAuditor2 shall turn back fault status to ready-to-go status within 10minutes.

Questions:

Issues:For different scale factors, the interval time to go back to ready-to-go status is different. We need to look into interval time for more exact measure.




Scenario(s):

The DBAuditor conducts DBMS performance testing according to TPC-C/H specification for various DBMSs. However, the specification can be updated, and as well the DBMS such as Oracle and MySQL can be newly released in the future. For the updates of TPC-C/H specification and the release of DBMSs, the queries which are used for performance test can be changed. The tester who is skilled in various kinds of DBMS can modify the sets of testing queries without affection to or modification of the DBAuditor itself within 1 man-day.

Business Goals:TTA testers execute DBMS performance testing according to TPC-C/H specification for vaious DBMSs

Relevant QualityAttributes:

Modifiability

Sce

nar

io

Com

pon

ents

Stimulus:Stimulus Source:

Environment:Artifact (If Known):

Response:ResponseMeasure:

Questions:Issues:


Scenario(s):

With the DBAuditor2, the client provides two major benchmark services to a

DBMS. Though they currently performs TPC-C/H test with the DBAuditor2,

according to TPC specification, they might need to modify TPC-C/H test

process by updates of TPC-C/H specification. This modification of features

shall be done by a single developer who is skilled in TPC-C/H specification and

understands the updates of the specification in 7 man-days.

Business Goals:Relevant Quality

Attributes:Modifiability

Sce

nar

io

Com

pon

ents




Questions:Issues:


Scenario(s): With the DBAuditor2, the client provides two major benchmark services to a

DBMS. Though they currently performs TPC-C/H test with the DBAuditor2,

according to TPC specification, they might need to add other kinds benchmark

services. These additions of benchmark services shall be done by a single

developer who understands the specification of newly added benchmark service



in 90 man-days.

Business Goals:Relevant Quality

Attributes:Modifiability

Sce

nar

io

Com

pon

ents




Questions:Issues:



10.4. References

[1] http://dogbert.mse.cs.cmu.edu/mse2008Korea/AoB/repository/documents/AoB_SRS_v1.0_20080302.doc

[2] http://dogbert.mse.cs.cmu.edu/mse2008Korea/AoB/repository/misc/Communication_Experiment.doc

[3] Clements, Bachmann, Bass, Garlan, Ivers, Little, Nord, Stafford, Documenting Software Architectures –

Views and Beyond, Addison-Wesley, 2003

[4] Heng Chen, Myung-Joo Ko, Neel Mullick, Paulo Merson, “SEI ArchE System, Architecture & Design

documentation for the ‘Architecture Expert & Design Assistant’, 2004


http://dogbert.mse.cs.cmu.edu/mse2008Korea/AoB/repository/misc/Communication_Experiment.doc

http://dogbert.mse.cs.cmu.edu/mse2008Korea/AoB/repository/documents/AoB_SRS_v1.0_20080302.doc

Software Architecture Document v0.51.doc

Documents

Transcript of Software Architecture Document v0.51.doc