IMS database application manual - Max Planck...

IMS database application manual

The following manual includes standard operation procedures (SOP) for

installation and usage of the IMS database application. Chapter 1 – 8 refer to

Windows 7 operating systems and the Chapters 8 and 9 refer to a UNIX

environment.

Contents

1. Installation Java environment ........................................................................... 2

2. Installation PostgreSQL server .......................................................................... 3

3. Include PostgreSQL tools directory into your system path .............................. 10

4. Installation: first IMSDB application run .......................................................... 13

5. Upload a project ............................................................................................. 17

6. Patient view .................................................................................................... 22

7. Machine learning ............................................................................................ 24

8. Backup and restore ......................................................................................... 31

8.1. Creating a backup ..................................................................................... 31

8.2. Restoring the database ............................................................................. 33

8.3. Vacuuming ................................................................................................ 34

9. Installation Linux (Ubuntu) .............................................................................. 35

9.1. Installation Java environment ................................................................... 35

9.2. Installation PostgreSQL database server ................................................... 35

10. Ubuntu 12.10 (Linux): first IMSDB application run ....................................... 40

Chapter: Installation Java environment

2 Manual IMS database database application – Till Schneider – 19.11.2012 – filename: ImsDbUserManual

1. Installation Java environment In order to use the IMS database application you need to install the Java Runtime

environment (JRE) version 6 or later. For more experienced users, the JRE can be

downloaded manually from the page

http://www.oracle.com/technetwork/java/javase/downloads/index.html which

offers different versions for different operating systems.

A more convenient option is to test for the java version on the following page

http://www.java.com/de/download/installed.jsp. This page gives you information

about old versions which have to be removed, etc. If you do not have java

installed or the version is below 6 then click on “Downloads”

(http://www.java.com/de/download/) and get the installation file and execute it.

Then click on “install” (installieren). Uncheck the installation of browser addons.

.

After the installation the following window appears.

http://www.oracle.com/technetwork/java/javase/downloads/index.html

http://www.java.com/de/download/installed.jsp

http://www.java.com/de/download/

Chapter: Installation PostgreSQL server


2. Installation PostgreSQL server Go to http://www.postgresql.org/download/windows/

In section “One click installer” click on “Download” and choose the latest version

(version 9.0/9.1/9.2 have been tested).

If you use a 64 bit Windows 7 operating system, it is recommended to choose Win

x86-64.

If you use a 64 bit Windows 7 operating system, it is recommended to choose Win

x86-32.

Start the installer, click on next.

http://www.postgresql.org/download/windows/



Give a password for the superuser “postgres”. This login is required later if you

want to connect to your database.

Specify the following port 5432 and click next.

Click next.



Uncheck the box and click finish. The server is installed.



Start the following application

"C:\Program Files\PostgreSQL\[YourVersion]\bin\pgAdmin3.exe"

Connect to the database server



Create a new database



Call your database “imsdb”. Then your connection url is

“jdbc:postgresql://localhost:5432/imsdb”

The imsdb application will ask for this url later.



Your database should be visible in the tree:

Chapter: Include PostgreSQL tools directory into your system path


3. Include PostgreSQL tools directory into

your system path In order to include the backup and restore functionality, the PostgreSQL “bin”

directory must be included into your system path.

Open the system control panel

Click on system.



Click on environment variables.

Select the variable “Path” and click on “modify”.



Add the following directory (depending on your version)

; C:\Program Files\PostgreSQL\[your version]\bin

Do not forget the semicolon ; as separator.

Press OK.

Now you can start the imsdb application.

Chapter: Installation: first IMSDB application run


4. Installation: first IMSDB application

run The config file general.properties will be created after clicking on close (schließen).



After clicking on close (schließen), you have to choose a directory in which the

root directory “Dump” for log files, dump files and backup files is created.

In this example the directory “test1” is chosen. The “Dump” directory as well as

subdirectories of “Dump” are created.



The application then asks for a database connection url. A default url is given as

shown in the next snapshot. If you followed the previous installation manual for

the database server, this should be

Okay and you can leave it unchanged and just click on ok.

After giving the database connection url which is automatically inserted into the

config file general.properties, the application starts and you can click on the

button “Connect to IMS database”



You have to enter the login information of the database super user you created

when installing the database server. This user is typically named “postgres”.

The database application will detect that the necessary database tables are not

yet created. It asks if it should create them. Click on yes!

After all the database objects (e.g. tables) are created, the following progress bar

shows 100% progress.

Now you might want to create a backup of the clean database (version 0). See

Creating a backup on page 31.

Chapter: Upload a project


5. Upload a project Click on “Upload a project” and specify the directory (here:

“HemerProjectDatabase”) where the files of your project are located in.

Note that files in subdirectories of the project directory are ignored. Click on open.

The next dialog will ask you if you want to include a Patient table-file which

includes diseases, medication, or any other attributes related to patient ids. Those

ids are expected to show up in the related IMS measurement files in the field

“sample ID”. Click yes.



Choose the patient table 120902PatientDataTable.xls. Click open.

In order to have categories (ConceptClass names) for the uploaded

measurements, you can specify where these names are taken from. Here you can

choose the first option (or the 3rd option, same effect)

Click on the first option, click on choose (Auswählen).

Marginal information: The first option will assign alpha-numerical ids to

“hemer_control” and numerical ids to “hemer_patient” and

“hemer_patient_case” (case for several measurements of the same patient).



A dialog shows you which new measurements will be stored in the database and

which measurements are already stored. You could cancel the upload project

now, but we want to upload, so click on “Yes, please”.

The following progress monitors will show you the progress state.



You are asked if the system should overwrite already backupped IMS

measurement files. Choose yes to skip the overwriting.

Then the information of the patient table is stored into the generalized data

structure componend (concept, values, attribute and associations to the IMS

measurements).



The next dialog shows you that the uploading process is finished.

Now you might want to create a backup. See Creating a backup on page 31.

Chapter: Patient view


6. Patient view Click on the “Patient View” tab of the application.

Click on “Refresh” to show the patients. You can switch the categories (concept

class names) by using the select field next to the “Refresh” button!

Chapter: Patient view


Click on a particular patient in the tree to visualize her attributes and values. In

this example, we picked Patient 304886. This patient has two entries, meaning

two corresponding IMS measurements. Furthermore, the id of this patient

changed. Therefore there is an id1 attribute to collect the first id the patient was

registered with, the id2 attribute contains the new id of the patient. If the patient

got another new id, this id would be reflected in the id2 attribute in another entry

for the patient and the id1 attribute would stay unchanged for this new entry.

Chapter: Machine learning


7. Machine learning Click on the Machine Learning tab.

Click on “Refresh”.

On the left side, a list of attributes of the class “hemer_patient” are shown. Click

one attribute to compare instances having this attribute with instances which

reveal a false value for this attribute. Here we choose “insulin” and press the



“Query” button. It is also possible to select several attributes by holding shift to do

comparisons between e.g. control versus disease. Note that if you wanted to see

the control attribute also you have to switch to “all” in the combobox “Choose a

class for categorical attributes”.

Tell the application which classes of instances you want to include in the

comparison. Hold the shift key to select several concept classes! Then press OK.

A dialog pops up which shows you about removed instances because they do not

belong to your chosen classes. Click OK.



The next dialog tells you about which instances are removed because they do not

have at least one “true” value for the given attributes. IMPORTANT: This removal

is only performed if two or more attributes were chosen. If then for both

attributes are false values, it makes no sense to include them into the result

dataset for the machine learning.

In the case of only one attribute is given, samples with false and true values for

this attribute should be included, so nothing is removed here. Click OK.

The next dialog is used to exclude peaks (peak regions). Hold the shift key to select

several peaks.

Press OK.



The following table includes all parameters of the peaks. RT and IM denote the

coordinate of the maximum signal intensity, target-RT and target-IM denote the

center point of the target region used to retrieve the peaks. Target-radRT and

target-radIM denote the radius of the target region. PeakNo denotes an artificial

index of the peaks which are ordered according to the increasing RT.



The decision tree which was trained on the overall dataset pops up. You can click

on a node to enter another visualization window. Right click gives you the choice

of changing the font size etc.

We click on the root node.



After clicking on the root node of the tree a data visualization window opens.

This window includes all instances of the dataset. If you click on a lower node, only

the instances which are included in THIS particular node are shown! You can

change the axes according to the particular features or you choose the nominal

class as axis, etc.



The corresponding WEKA dataset is shown in the area on the right side of the

application. The evaluation result of the ten fold cross validation (which is not

reasonable for really small datasets) is shown below.

Chapter: Backup and restore


8. Backup and restore

8.1. Creating a backup After you uploaded a project, it is recommended to create a backup of the

database.

Go to the dump window by clicking on “Menu” and “Backup”.

Click on “Dump!” to create a backup.



The process is finished when you see the following message.

Take a look into your root directory (“Dump”) and find the created dump archive

file.



8.2. Restoring the database It is ensured that all users are disconnected from the database before the process

starts.

If you want to restore a dump archive, click on “Restore” and choose the desired

dump archive.

Then the restore process will start and end with the following dialog.



8.3. Vacuuming The main reason for having the vacuum button is to remove unused large objects

referring to IMS measurement files which became orphan because of doing a

restore. If you do not store the IMS measurement files into the database (default

case, general.properties: saveImsRawData=0), then you do not need this method.

It is ensured that all users are disconnected from the database before the process

starts because it requires table locks.

Basically if you use this command the postgres application “vacuumlo” is called to

mark the raw data and vacuum full to rewrite all table contents. Also According to

the postgres manual, vacuum full is only recommended in a case where a big

amount of tuples are deleted, such as having orphaned large objects. The

“Analyze” flag is set for the vacuum full command. “Analyze” examines all tables

and updates statistics used by the query optimizer. If you just want to update such

statistics you do not need the vacuum button here. Open pgAdmin3.exe and

vacuum analyze by right click on your database and follow the vacuum dialog.

According to the PostgreSQL documentation

”VACUUM FULL rewrites the entire contents of the table into a new disk file

with no extra space, allowing unused space to be returned to the operating

system. This form is much slower and requires an exclusive lock on each table

while it is being processed.” (http://www.postgresql.org/docs/9.1/static/sql-

vacuum.html, last accessed 19.11.2012).

http://www.postgresql.org/docs/9.1/static/sql-vacuum.html

http://www.postgresql.org/docs/9.1/static/sql-vacuum.html

Chapter: Installation Linux (Ubuntu)


9. Installation Linux (Ubuntu) The following installation steps are performed on Ubuntu 12.10.

9.1. Installation Java environment In order to run the IMS database application, a Java Runtime Environment is

required.

Install OpenJDK version > 6. We take version 7.:

$ sudo apt-get install openjdk-7-jre

Check your version. There should be at least 1.6

9.2. Installation PostgreSQL database server 1. Install the latest PostgreSQL server which is in this case postgresql-9.1:

$ sudo apt-get install postgresql

Note that if you want to make sure installing version 9.1:

$ sudo apt-get install postgresql-9.1



2. The following contrib package is required in order to use the vacuumlo

command-line tool:

$ sudo apt-get install postgresql-contrib

If you want to make sure version 9.1:

$ sudo apt-get install postgresql-contrib-9.1

3. Install the graphical user interface tool Pgadmin3 for managing the postgres

server.

$ sudo apt-get install pgadmin3



4. Enter the postgres terminal to set a password for the super user postgres

which was created automatically for administration.

$ sudo –u postgres psql postgres

You will enter the postgres terminal, there you set the pw. It will be

decrypted and send as an alter role command, no plain text or log etc.:

#\password postgres

Enter new password:

Enter it again:

5. Check the port and if your server is running:

$ sudo service postgresql status



or

$ /etc/init.d/postgresql status

You can start/stop the server:

$ sudo service postgresql start

$ sudo service postgresql stop

$ sudo service postgresql restart

This also works if you call the following

$ sudo /etc/init.d/postgresql start

$ sudo /etc/init.d/postgresql stop

$ sudo /etc/init.d/postgresql restart

6. Use the convenience command line tool to create a database with name

“imsdb”. (This tool does nothing else than calling CREATE DATABASE in the

postgres terminal.). Further details can be found on

http://www.postgresql.org/docs/9.1/static/manage-ag-createdb.html

Alternatively you can use this graphical tool pgAdmin3 to create the

database “imsdb” as described in the Windows 7 installation guide or you

follow the next steps and create the database by using the command line.

http://www.postgresql.org/docs/9.1/static/manage-ag-createdb.html



$ sudo –u postgres createdb imsdb

7. Optionally you can check the details of your database by entering the

postgres terminal.

$ sudo –u postgres psql postgres

# \l

Chapter: Ubuntu 12.10 (Linux): first IMSDB application run


10. Ubuntu 12.10 (Linux): first IMSDB

application run Compared to the first run in Windows 7 and in the UNIX environment, there are

no application specific differences. Therefore the author refers to Installation: first

IMSDB application run on page 13 for details.

Start the application:

$ java –jar 121118ImsDbApp.jar

The following screenshots illustrate the first steps of starting the application,

uploading a project. Please see Upload a project on page 17 for details.



Hint: If you followed some other installation of the postgres server and

the vacuum command in the dump window of the IMS database application

(menu -> backup -> vacuum) is not working because vacuumlo tool is missing

("Error: pg_wrapper: invalid command name") you need to install

the contrib package of postgres I: sudo apt-get install postgresql-contrib-9.1 which

was stated above to install.

IMS database application manual - Max Planck...

Documents

Transcript of IMS database application manual - Max Planck...