Instal Guide

ECLIPSE* reservoir simulation software

Installation Guide

Version 2013.2

Proprietary noticeCopyright 1982 - 2013 Schlumberger. All rights reserved. Reproduction or alteration without prior written permission is prohibited, exceptas allowed under applicable law.

Use of this product is governed by the License Agreement. Schlumberger makes no warranties, express, implied, or statutory, with respectto the product described herein and disclaims without limitations any warranties of merchantability or fitness for a particular purpose.

Trademarks & service marks"Schlumberger," the Schlumberger logotype, and other words or symbols used to identify the products and services described herein areeither trademarks, trade names, or service marks of Schlumberger and its licensors, or are the property of their respective owners. Thesemarks may not be copied, imitated, or used, in whole or in part, without the express prior written permission of their owners. In addition,covers, page headers, custom graphics, icons, and other design elements may be service marks, trademarks, and/or trade dress ofSchlumberger and may not be copied, imitated, or used, in whole or in part, without the express prior written permission of Schlumberger.

ECLIPSE Suite Installation Guide Table of contents

3

Table of contentsList of figures ...... ...................................................................................................................................................................4List of tables ....... ...................................................................................................................................................................5

Chapter 1 - Simulation software system requirements ................................................................. 7General requirements .............................................................................................................................................................7

Chapter 2 - Simulation software installation .................................................................................. 9Disk organization ...................................................................................................................................................................9Installing PC software from disk ...........................................................................................................................................10Directory structure ................................................................................................................................................................12Installing Linux software from disk........................................................................................................................................13Directory structure ................................................................................................................................................................23

Chapter 3 - Licensing ..................................................................................................................... 25Installing Schlumberger licensing .........................................................................................................................................25

Chapter 4 - Running the software ................................................................................................. 27Running ECLIPSE in batch mode on a PC...........................................................................................................................27Parallel ECLIPSE on Linux...................................................................................................................................................28

Chapter 5 - Microsoft HPC installation ......................................................................................... 33Installation .......... .................................................................................................................................................................33Checking the InfiniBand driver firmware...............................................................................................................................34Configuring cluster information on the head node................................................................................................................35Configuring nodes ................................................................................................................................................................37Configuring the HPC web services interface on the head node ...........................................................................................38Using the license aware activation filter................................................................................................................................39Configuring client PCs ..........................................................................................................................................................52

Appendix A - Legacy macros and scripts .................................................................................... 53About the macros and scripts ...............................................................................................................................................53Macros: command line flags.................................................................................................................................................56Examples............ .................................................................................................................................................................57

Appendix B - Index ......................................................................................................................... 59

4 ECLIPSE Suite Installation GuideList of figures

List of figuresFigure 2.1 ........... ssh setup.................................................................................................................................................17Figure 5.1 ........... The filter configuration window ...............................................................................................................40Figure 5.2 ........... Filter configuration ..................................................................................................................................41Figure 5.3 ........... License configuration panel ....................................................................................................................41Figure 5.4 ........... Filter settings...........................................................................................................................................42Figure 5.5 ........... License reservation configuration ...........................................................................................................43Figure 5.6 ........... The User Rules tab .................................................................................................................................45Figure 5.7 ........... Custom rules populated with properties..................................................................................................45Figure 5.8 ........... Creating a New Rule screen ...................................................................................................................46Figure 5.9 ........... FLEXlm license checking by feature.......................................................................................................47Figure 5.10 ......... FLEXlm license checking by user ...........................................................................................................48Figure 5.11 ......... License reservations checking................................................................................................................49Figure 5.12 ......... HPC filter log screen ...............................................................................................................................50Figure 5.13 ......... License aware scheduler default options................................................................................................51

ECLIPSE Suite Installation Guide List of tables

5

List of tablesTable 1.1 Supported operating systems and compilers.............................................................................................7Table 1.2 Third party tools and applications ..............................................................................................................8Table 2.1 Platform MPI interconnect search options ...............................................................................................20Table 2.2 Intel MPI interconnect settings.................................................................................................................21

6 ECLIPSE Suite Installation GuideList of tables

ECLIPSE Suite Installation Guide Simulation software system requirementsGeneral requirements

7

Chapter 1Simulation software system

requirements

General requirementsFor information on the platform and product availability for this release, see the release overview in the "ECLIPSE Suite Release Notes".

Software products and supported platformsThese requirements are specific for the 2013.2 release of ECLIPSE suite. The suite is released on disk for the platforms and operating systems listed in Table 1.1 and with the third party products listed in Table 1.2.

Table 1.1 Supported operating systems and compilers

Machine Operating System Required Software Notes

PC Window 7 Professional 64 SP1

Minimum screen resolution for PC systems is 1024x768.

Windows HPC Windows Server 2008 R2 HPC x64 SP1

HPC SP3 Pack Simulators only.

Windows Server 2012 HPC x64

HPC Pack See "Microsoft HPC installation" on page 33 for full details.Simulators only.

Linux x86_64 Red Hat 5 update 5, or Red Hat 6 update 3

This is the 64-bit version for the AMD Opteron and Intel EM64T chips.Simulators only.

Linux PPC 64 Red Hat 5, update 5 POE POE is required to run ECLIPSE 100 and ECLIPSE 300 in parallel. Note, POE is not supplied and must be purchased from IBM.

Only ECLIPSE 100 and ECLIPSE 300 are available on this platform.

8 Simulation software system requirements ECLIPSE Suite Installation GuideGeneral requirements

Note Third party security and anti-virus software such as Norton and McAfee can cause problems.

Notes The 2013.2 ECLIPSE suite is not supported on 32-bit operating systems.

For more details on requirements for parallel ECLIPSE 100 and ECLIPSE 300, see "Parallel ECLIPSE on Linux" on page 28.

The simulators are supported on Windows Server 2008 R2 SP1 and Windows Server 2012. Pre- and post-processing software is not supported.

Table 1.2 Third party tools and applications

Software Version Notes CommentsLSF LSF version 7.0, update 6 HPC,

LSF 8.0.2.Needs to be purchased separately

We strongly recommend that you install the HPC version.

Platform Computing can add an LSF SIS integration kit to augment resource checking in the simulator macros by including license checking. For further details contact Platform Computing

www.platform.com

FLEXlm 11.8 Supplied on disk, installed automatically when Tools are selected.

Used for License Manager configuration.

Carnac 1.2b145 Supplied on disk, installed automatically.

Used by FloGrid.

Intel MPI 4, update 3 Supplied on disk.

On Linux, it is installed automatically so long as tools are selected. See "Configuring Intel MPI and Platform MPI for Linux" on page 16.

On Windows, this must be installed separately.

Intel MPI is required for using parallel ECLIPSE, ECLIPSE 300 and R2SL on PC Windows systems. See "Installing Intel MPI" on page 11 and http://software.intel.com/en-us/intel-mpi-library/

Platform MPI 8.2 Supplied on disk for Linux installations.

On Linux, it is installed automatically so long as tools are selected. See "Configuring Intel MPI and Platform MPI for Linux" on page 16.

Latest documentation is for version 8.1.

Web/HTML browser Internet Explorer 7 or greater, Firefox 3.5 or greater, or equivalent.

Used by on-line help.

ECLIPSE Suite Installation Guide Simulation software installationDisk organization

9

Chapter 2Simulation software installation

Disk organizationThe 2013.2 software is distributed on a disk which contains:

The 2013.2 installation with all documentation, data and tutorials for Windows

Intel MPI Runtime installation

Documentation in PDF (Portable Document Format)

Dongle drivers

The simulators for all x86 64-bit architectures compiled for Linux

Utility resources which contains:

Benchmarks

PC resources

Full documentation (manuals and on-line help), data and tutorials for Linux

LINUX resources (LSF Integration Kit etc.).

For installation on a PC, see "Installing PC software from disk" on page 10.

For installation on Linux systems, see "Installing Linux software from disk" on page 13.

Note A version of ECLIPSE is available for machines using the IBM PPC 64 architecture. This version is distributed on an additional disk. If you require this version, please request it from your Account Manager.To install ECLIPSE from this disk, follow the same procedures as described for standard installations. The IBM build of ECLIPSE requires IBM POE. This must be purchased from IBM and is not supplied with ECLIPSE.

10 Simulation Software installation ECLIPSE Suite Installation GuideInstalling PC software from disk

Installing PC software from disk

Installation process1 Insert the disk into the disk drive.

2 The Schlumberger installation panel should appear; if not, then using the File Browser to access the disk, run the script D:\setup.exe.

3 Click on Install Products.

4 Then click on ECLIPSE 2013.2.

This will install the 2013.2 programs.

5 Once installation is complete, reboot the machine.

Notes You must install in a directory path that has no spaces in the path name, NOT for example

in C:\Program Files\ecl. We assume C:\ecl as the root in the text that follows. Occasionally, for large disks, the installation program will calculate the space available

incorrectly; as long as sufficient space is available, this should not cause any problems and the installation should go through correctly.

If insufficient disk space is available to complete the installation a blank message box will appear. Also, the installation procedure assumes that the Windows directory is C:\WINDOWS.

It is essential you disable your virus checker temporarily during the installation process.

The disk contains directories \3rdparty\pc\resource and /3rdparty/linux/resource. The PC directory contains various third party applications.

To view the on-line manuals you will need to install Acrobat Reader.

Hint Adobe Acrobat Reader for your platform can be downloaded from the Adobe website, at: http://www.adobe.com/products/acrobat/readstep.html.Selection of Include option for searching PDF files, is essential. See the Adobe Acrobat Reader On-Line Help for more information on the search function.Please do not use Adobe Reader X. Instead, use versions up to 9.5, or version XI.

User manualsFor 2013.2 the user manuals in PDF format are installed by default.

From the Simulation Launcher, click on the ECLIPSE manual option in the left-hand panel and then select a product version from the drop-down menu. This displays the documentation bookshelf which is a PDF document.

You may access manuals from previous versions (2007.1, for example) from the Simulation Launcher, if they are installed.

ECLIPSE Suite Installation Guide Simulation Software installationInstalling PC software from disk

11

Configuring PC software

Standalone installation on windowsHaving installed the software from the disk, check the environment in

Start | Settings | Control Panel | System | Advanced | Environment Variables

The System path should include the ECL\HOME and ECL\MACROS directories, in that order.

Installing Intel MPIYou will need to install Intel MPI if you plan to run parallel ECLIPSE, ECLIPSE 300 or Avocet IAM.

Note If you are using Windows HPC, you do not normally have to install Intel MPI.

To install Intel MPI you need to:

1 Insert the installation disk into the disk drive. Then:

Select the ...\3rdparty\PC\resource\IntelMpi directory on the installation disk.

Double click on the file called w-mpi_rt-p-x.x.x.xxx.exe to start the installation.

Alternatively:

Click on Install Products.

Select Intel MPI.

2 Follow the normal installation wizard procedures to install Intel MPI.

Select Next or Continue to move between wizard screens and accept default values where prompted.

The PC simulation launcherThe installation process creates a program group containing the Simulation Launcher. A shortcut to this launcher is also placed on the desktop of the PC.

12 Simulation Software installation ECLIPSE Suite Installation GuideDirectory structure

Directory structureAt the top level this looks like

ecl\home ecl\macros ecl\2013.2

The home directory contains: the macro $eclrc.bat additional resource files.

The macros directory: macros for running the programs and utilities, for example eclrun macros called by the program, and utility macros (see "Running the software" on page 27)

specific files used for configuration purposes; in particular eclrun.config and CONFIG.ECL

The tools directory contains a pc subdirectory which contains: a number of subdirectories corresponding to various utilities such as convert and

expand The 2013.2 directory contains:

a bin directory containing executables. a number of program directories corresponding to the various programs, such as eclipse,

flogrid, etc.A number of directories containing utilities and building blocks, for example:

manuals - contains the manuals bookshelf rele_nt - contains the release notes the resource directory that contains resources shared by several programs.Each program directory (within the 2013.2 directory) contains some of the following:

a data directory containing sample datasets

one or more manual directories, such as eclipse\ecl_rm, flogrid\flog_ug, etc. a tutorials directory containing data for one or more tutorials, for example

flogrid\tutorials a help directory containing on-line help.

ECLIPSE Suite Installation Guide Simulation Software installationInstalling Linux software from disk

13

Installing Linux software from disk

Preparing to install 2013.2 simulation software2013.2 is an incremental release and can be installed independently of previous major or incremental releases. It can be installed over existing installations.

Installation notes Only use static mounts for any directories holding home directories, ECLIPSE data or

ECLIPSE executables. Problems have been noted with auto-mounted file systems.

When installing software, bear these requirements in mind:

It is imperative that the previous versions of reservoir simulation software are installed before installing the current version, that is, install versions in release order, starting with the earliest version. Do not install the current version followed by an older version.

If you are installing ECLIPSE and Intersect, always install a version of ECLIPSE before an equivalent version of Intersect, for example, install ECLIPSE 2013.1 and then Intersect 2013.1.

If for any reason you do install an earlier version of the software, be sure not to install the macros. To do this, deselect the macros in the installation options.

It is assumed in this chapter that the software is installed on a default path /ecl.

Macros and tools directoriesUsers with existing ECLIPSE installations should always ensure that the macros directory is backed up before proceeding with the software installation.

1 Copy the macros directory to macros.old. This is because the macros directory contains vital information about the existing software installation (in CONFIG.ECL).

2 Move the tools directory to tools.old. The tools.old directory may later be removed when the new installation is working satisfactorily.

For all applications, the manuals are available in PDF (Portable Document Format) for browsing and printing.

ECLIPSE and Intel MPI

Note Intel MPI is automatically installed when Tools are selected in the Installation. However further steps may be required for configuration. See the Intel MPI section. "Configuring Intel MPI and Platform MPI for Linux" on page 16.

14 Simulation Software installation ECLIPSE Suite Installation GuideInstalling Linux software from disk

Installation process1 To install, insert the relevant disk into the disk drive.

2 Mount the installation disk as a file system.

a On Linux systems:

In the default configuration of a Linux system the disk is automatically mounted as /mnt/cdrom or /media/cd.

Note Letting Linux auto-mount the disk may set default options that mean the install scripts will not run. To correct this, we recommend that you unmount and the mount the installation disk manually. To unmount the installation disk use the command: umount -k /dev/cdrom. To mount the installation disk use the command mount /dev/cdrom /tmp/a. Please note that /tmp/a should be an existing directory.

Note The installation disk can be ejected using the eject command.

3 When the installation disk is successfully mounted run the install C-shell script which is supplied on the disk:

/media/cdrom/ECLIPSE/UNIX/install/cdinst.csh

4 Choose a version to install (default: Linux x86_64).

5 Check the correct version for you machine has been automatically detected.

6 Choose the option and press enter.

7 If you wish to install ECLIPSE for more than one type of machine, or would like to install the documentation and data, rerun the cdinst.csh script.

SchlumbergerThe following versions were found for installation0) [ ] Exit without installing1) [ ] documentation_and_data 2013.2 week xxxx2) [x] linux_x86_64 2013.2 week xxxx

Available programs, data, tutorials and help for linux_x86_64 2013.2 week xxxx1) ConvertGrid2) E3003) Eclipse4) Expand5) Extract... ... ...xx) ToolsSelect A - Install all programs, data, tutorials and help

N - Install none of theseS - Select a subset

Enter A to install all applications.Enter proposed location for installation (default /ecl) :


15

8 Select the install directory.

Note Always install ECLIPSE to the same location as any previous versions. There is an internal directory structure to prevent any old versions being overwritten.

Note ECLIPSE for different architectures should also be installed into the same location

The software then installs:.

9 Always answer yes to the Install the Macros question, unless installing an earlier version, in which case, do not install the macros.

The current macros directory, if any, will be backed up with a suffix consisting of the date and time of install, for example macros.backup.13:24:52.230209.

10 Finally migrate any custom settings from the old configuration files into the configuration files in the new macros directory. Settings may be stored in any of the following files:

CONFIG.ECL eclrun.config @eclrcSee the "ECLRUN User Guide" for details.

Note Do not just copy the old files into the new macros directory. From 2009.1 onwards, eclrun.config uses an XML format which differs from that required in previous releases, see the ECLRUN User Guide for full details.

11 After installation the users environment needs to be set up to reflect the ECLIPSE installation. A sample .cshrc file is shown below. Edit the paths if required and copy to each users home directory.

Installing ConvertGrid (size xxxxxxx)Installing E300 (size xxxxxxxx)Installing Eclipse (size xxxxxxxx)Installing Expand (size xxxxxx)Installing Extract (size xxxxxx)... ... ... Installing Tools (size xxxxxxx)Installing Resource

(size xxxxxxxx)(size xxxxx)

Resource installed (total size xxxxxxxx)

Do you want to install the macros [default n]?y


Example .cshrc file

Configuring Intel MPI and Platform MPI for LinuxIntel MPI and Platform MPI are installed with the ECLIPSE installation. No separate installation is required.

Both of these MPIs use ssh to setup the communication between nodes, so it is necessary to make sure that password less ssh is setup between any computers that may be used to run a parallel ECLIPSE or ECLIPSE 300 job.

The objective of the steps below is to make sure that the public and private keys exist for the user and that the authorized_keys file exists and has the correct keys inside.

Hint Additionally it is usually a good idea to set the ssh configuration to automatically accept connections to new machines. If this step is not completed then you would have to login to every machine in the cluster before you could guarantee that a MPI job would work correctly.

What you need to do depends on the current setup.

Warning If other applications rely on ssh to work, please talk to your system administrator to check the following procedure will not cause any problems with other applications.

#!/bin/tcsh#Edit the line below to reflect the location of the license server.#LM_LICENSE_FILE was used by Eclipse pre 2007.2setenv SLBSLS_LICENSE_FILE [email protected]

#Edit the line below to set the install location.setenv ECLPATH /ecl

#Set Endian so Linux files line up with other OS'sif ( `uname` == "Linux" ) then setenv F_UFMTENDIAN bigendif

#Source eclrunsetup if it exists, so eclrun worksif ( -r $ECLPATH/macros/@eclrunsetup.csh ) then source $ECLPATH/macros/@eclrunsetup.cshendif

#Source LSF environment scriptif ( -r /lsf/conf/cshrc.lsf ) then source /lsf/conf/cshrc.lsfendif


17

1 First check if everything is already working, by trying to connect to other machines in the cluster. For example, from comp001 ssh to comp002, no password should be required.

2 If this works then skip to the "ssh configuration" on page 17.

3 If not then what to do next depends on the current setup. The flowchart in Figure 2.1 describes how to set up ssh.

Figure 2.1 ssh setup

ssh configuration1 First check that ssh is setup to accept connections to new machines, as shown above.

2 Then either create the file called config in the users .ssh directory, or add the line if it already exists.

3 The permissions for the config file should be 400. Change them if necessary.

comp001:/home/user>ssh comp002Last login: Thu Apr 9 10:37:03 2009 from comp001.geoquestcomp002:/home/user>

comp002:/home/user>cd .sshcomp002:/home/user/.ssh>cat configStrictHostKeyChecking=no


4 The permissions for the authorized_keys file should be 600. Change them if necessary.

5 The permissions of $HOME and $HOME/.ssh must not allow other users to write to those directories.

Configuring LSFIf you are using LSF, you need to edit the lsf startup file /etc/init.d/lsf to add an entry for ulimit so that both Intel MPI and Platform MPI can run when under LSF control. To do this, open the file and add the bold entry for ulimit as shown here:

Note See the "ECLRUN User Guide" for more configuration settings for LSF.

#!/bin/sh# $Id: startup.svr4,v 1.10 2008/04/08 06:13:09 xltang Exp $## Start and stop LSF daemons, System V / OSF version# Make sure we're running a shell that understands functions## The following is for the Linux chkconfig utlility# chkconfig: 35 99 01# description: Load Sharing Facility## The following is for the Linux insserv utility### BEGIN INIT INFO# Provides: lsf# Required-Start: $remote_fs# Required-Stop: $remote_fs# Default-Start: 3 5# Default-Stop: 0 1 2 6# Description: Start LSF daemons### END INIT INFO

#line added so ECLIPSE can run over Infiniband when under LSF control.ulimit -l 1024000000

if test "$SH5" != "/bin/sh5" -a -r /bin/sh5then SH5=/bin/sh5 export SH5 exec /bin/sh5 "$0" "$@"fi

check_env () {if [ x$LSF_ENVDIR = x ]; then # Using default path of lsf.conf... LSF_CONF=/lsftop/lsf/conf/lsf.conf


19

Note LSF by default saves temporary files in a hidden directory called .lsbatch which is inside the users home directory. This can cause problems if the home directories don't have much free space, or quotas are enabled. This can be avoided by adding the following setting in lsf.conf:

LSB_STDOUT_DIRECT=Y

You must restart LSF for this change to take effect.

If you are using LSF HPC and the Intel MPI, you may need to edit the MPI location in the wrapper script, e.g. wrapping the Intel MPI in the intelmpi_wrapper script. The following example assumes that ECLIPSE has been installed to /ecl and lsf has been installed to /lsf. It also assumes that you are using Intel MPI. If you have installed elsewhere, please use the appropriate path.

1 In the LSF installation directory (/lsf in our example), edit the file

/lsf/8.0/linux2.6-glibc2.3-x86_64/bin/intelmpi_wrapper 2 Search for the line MPI_TOPDIR="........" 3 Replace with the correct location of the Intel MPI. If the default settings have been used

this line should look like this

MPI_TOPDIR="/ecl/tools/linux_x86_64/intel/mpi/4.0/" 4 Find all occurrences of "$MPI_TOPDIR/bin" and replace them with

"$MPI_TOPDIR/bin64" 5 If you wish to use SSH to start the MPI daemons:

a Search for the line

MPDBOOT_CMD="$MPI_TOPDIR/bin64/mpdboot" b Change it to

MPDBOOT_CMD="$MPI_TOPDIR/bin64/mpdboot -r /usr/bin/ssh" 6 With Intel MPI 4.0.3, some changes were made to the way jobs were launched by the

introduction of the HYDRA loader from OpenMPI. This impacts the way jobs are launched under LSF as it will automatically select blaunch rather than the older ssh as the method of communication between mpi daemons.

We have seen this give issues and to turn it off and go back to the old method of using ssh you need to set the environment variable I_MPI_HYDRA_BOOTSTRAP to ssh. This can be done by either:

under csh/tcsh using setenv I_MPI_HYDRA_BOOTSTRAP ssh in your .cshrc file, or

under bash using export I_MPI_HYDRA_BOOTSTRAP=ssh in the .profile file

PBS settingsLicense scheduling is available with ECLIPSE and PBSPro. To install and configure PBSPro, see the instructions on the installation disk in the folder 3rdparty/PBSPro.To switch on PBSPro, use the same settings as those for LSF and Windows HPC.

In eclrun.config find the setting:False


Change the setting to:

TrueThe LSFLicenses setting is used by PBSPro.

Note Only ECLRUN is supported. The old scripts starting with the character @ are not able to schedule licenses under PBS.

Intel MPI settingsThere are some tuning parameters which you can add to the mpiexec.conf file which can improve performance. You can find this file in the directory

Ecl/tools/linux_x86_64/intel/mpi/4.0.3/etc or in the local work directory. Add the following tuning parameters to the file: -genv I_MPI_PIN_PROCESSOR_LIST allcores:map=scatter-genv I_MPI_PIN_CELL core

Platform MPI settingsThe Platform MPI should automatically determine and use the best interface. If you wish to set the interface then it is possible by setting an environment variable. The search order for the interconnect is determined by the environment variable MPI_IC_ORDER (which is a colon-separated list of interconnect names). The most common options are shown in Table 2.1.

Example

To set the MPI to use the DAPL device, add the following line to the users .cshrc file.

setenv MPI_IC_ORDER UDAPLor

setenv MPI_IC_ORDER udapl:tcpUsing upper case for UDAPL sets the instruction to use that device or exit. Using lower case sets the instruction to try the device and, if it doesnt work, try another. Shared memory is always implemented if possible. The standard ECLIPSE scripts will add a few extra lines into the top of the output files and these will display the interconnects used.

Table 2.1 Platform MPI interconnect search options

Device Type Description SettingInfiniBand (IB) InfiniBand can be SDR, DDR or QDR. IBV

Infinipath Infinipath made by QLogic PSM

uDAPL (10g iWarp Ethernet and InfiniBand)

uDAPL interface is used by 10G iWarp network cards. Additionally it can also be used by most types of InfiniBand & Infinipath.

UDAPL

Standard Ethernet Standard 100Mbs, 1Gbs and 10Gbs network cards. TCP

Myrinet GM based Myrinet cards. GM


21

The following extract from the FILENAME.OUT file uses TCP to communicate between nodes.

Alternatively it can be overridden by using command line arguments to the eclrun script. The following will force the Platform MPI to use the IBV setting regardless of the environment variable setting.

eclrun -c plmpi -mpi-args=-IBV eclipse DATASETNAME

Intel MPI settingsThe Intel MPI detects and uses the correct interface. As for the Platform MPI, setting an environment variable will force an interconnect to be chosen. See Table 2.1.

Set the I_MPI_FABRICS_LIST environment variable as follows: setenv I_MPI_FABRICS_LIST dapl,tcpThis sets Intel MPI to try the chosen devices in order.

I_MPI_FALLBACK and I_MPI_DEVICE are no longer required and should be removed if set.

To print out the connection or interface type used set the following environment variable:

setenv I_MPI_DEBUG 2For more information, see http://software.intel.com/en-us/intel-mpi-library/

Note The latest Intel MPI HYDRA launch mechanism is a 32-bit application that starts the ECLIPSE processes. This means 32-bit libraries must be installed for this to work. On Redhat 6 the required rpm is called glibc-2.12-1.47.el6.i686.rpm. Depending on your installation choices, this may have some prerequisites that must also be installed.

IBM POE (MPI on IBM POWER machines)Please refer to IBM for more information about POE.

Installing InfiniBand driversIt is essential that you install the OFED drivers if you want to use InfiniBand. See www.openfabrics.org for more information.

host | 0 1======|=========== 0 : SHM TCP 1 : TCP SHM

Table 2.2 Intel MPI interconnect settings

Device Type Description SettinguDAPL uDAPL interface is used by 10G iWarp, InfiniBand and Infinipath cards. dapl

Standard Ethernet Standard 100Mbs, 1Gbs and 10Gbs network cards. tcp

Infinipath Infinipath cards from QLogic. tmi


Note The drivers can also be installed from the Redhat installation disk. Depending on the type of hardware in your system, you may need to download and install a later driver from www.openfabrics.org.

General InfiniBand notesTo use InfiniBand (IB) cards, you must assign an IP address to the card. This allows you to "ping" the other computers via the IB interface. If the ping succeeds then you know all the hardware is physically connected properly.

In addition see the specific Voltaire and Silverstorm sections that follow.

InfiniBand driversFollow the install instructions from the InfiniBand supplier.

However, the following additional steps may be necessary. A good indication is when IP over IB is working properly, but ECLIPSE is not.

For Red Hat 5 or later, edit the file /etc/security/limits.conf and add the following two lines:

* soft memlock 100000000* hard memlock 100000000

The value denotes the number of kilobytes that may be locked by a process. The file above contains further documentation.

Note The steps described above will allow any user in the system to lock as much as the whole memory set in the configuration files.

Infinipath driversTo obtain drivers and instructions, visit this site: www.qlogic.com.

User manuals and on-line helpUser manuals are provided in PDF format. On-line help is provided in HTML 3.2.

For 2013.2 the user manuals are installed by default as part of the installation on PC, and from the installation disk for Linux. (See "Disk organization" on page 9 for more information.) The on-line help is provided by default as part of the installation with the associated program, on each platform. Since the manuals form part of the on-line help in many instances, they are also provided in HTML format.

To print a manual or part of a manual, open the PDF in a PDF reader and print the pages that you require.

ECLIPSE Suite Installation Guide Simulation Software installationDirectory structure

23

Directory structureAt the top level this looks like

ecl/macros ecl/tools ecl/2013.2

The macros directory contains: macros for running the programs and utilities, for example @eclipse, @e300 (see

"Running the software" on page 27)

macros called by the program, and utility macros, for example @check_chip which returns the chip type

specific files used for configuration purposes; in particular @eclrc and CONFIG.ECL files used for monitoring license usage, for example flex.log.The tools directory contains subdirectories for each machine type; each subdirectory contains:

a number of subdirectories corresponding to various utilities such as convert and expand

The 2013.2 directory contains:

a bin directory containing subdirectories for each machine type; each subdirectory contains executables.

a lib directory containing subdirectories for each machine type; each subdirectory contains shared objects.

a number of directories corresponding to the various programs, such as eclipse, e300, etc., containing data, tutorials and documentation.

a number of directories containing utilities and building blocks, for example

manuals contains the manuals bookshelf rele_nt contains the release notes

The resource directory which contains resources shared by several programs.

24 Simulation Software installation ECLIPSE Suite Installation GuideDirectory structure

ECLIPSE Suite Installation Guide LicensingInstalling Schlumberger licensing

25

Chapter 3Licensing

Installing Schlumberger licensingTo run ECLIPSE Suite you need a valid license configured. You can either connect to an already existing license server, or you can configure the license on your local machine either as a Standalone solution or as a license server. This section describes how you install FLEXnet and configure your license.

1 First launch Schlumberger Licensing installer from the ECLIPSE installation disk.

2 Click on Install Products then Install SLB Licensing.

The Schlumberger FLEXlm installer starts. This installs all necessary utilities to configure licensing with FLEXnet, including standard FLEXnet utilities and the Schlumberger licensing tool.

3 Click Next to proceed.

4 Click Change to choose another destination than the default.

If you have multiple license files that you need to run using the Schlumberger Licensing tool you should be aware of that there is a limitation in the length of the combined path to the license files. You should use a shorter installation path for example C:\SLBLicensing

5 Click Next and then Install to proceed

For further information on the licensing utility please see the Installing Schlumberger Licensing user guide.

26 Licensing ECLIPSE Suite Installation GuideInstalling Schlumberger licensing

ECLIPSE Suite Installation Guide Running the softwareRunning ECLIPSE in batch mode on a PC

27

Chapter 4Running the software

Running ECLIPSE in batch mode on a PCThe ECLIPSE simulators are batch programs, and for many users it is desirable to use a batch queuing system to submit jobs. The following describes how batch queues may be invoked using the macros.

There are no provisions within the ECLIPSE launcher or the macros to set ECLIPSE 100 or ECLIPSE 300 to run in batch mode on a PC.

You can run a batch file on a PC. To do this, create a file called RUN.BAT containing commands like those in the example:

To start the runs, double-click on the file RUN.BAT in Windows Explorer.

cd \data\dir1 eclrun -v 2011.1 eclipse DATASET1cd \data\dir2 eclrun -v 2011.2 eclipse DATASET2cd \data\dir3 eclrun -v 2012.1 e300 DATASET3cd \data\dir4 eclrun -v 2013.2 e300 DATASET4

28 Running the software ECLIPSE Suite Installation GuideParallel ECLIPSE on Linux

Parallel ECLIPSE on Linux

Linux systems - macros and scriptsIt is recommended that ECLRUN is used to run most of the simulators and other programs. Please refer to the "ECLRUN User Guide" for more details.

Installing and configuring parallel ECLIPSEMPI versions of parallel ECLIPSE 100 and ECLIPSE 300 are supplied for Linux, and Windows.

Software requirements for LinuxYou must use a supported version of Linux. For information, see "Software products and supported platforms" on page 7.

Intel MPI software is supplied with the ECLIPSE installation.

Installation procedureSee "Installing Linux software from disk" on page 13.

Notes on setting up the OS for parallel ECLIPSE

Increasing the size of MPI memory buffersOccasionally ECLIPSE or ECLIPSE 300 can exceed the size of the default memory set in the MPI messaging, particularly with large problems. In order to increase this it is possible to use the PARAOPTS keyword in ECLIPSE, which should be set to:

The NSTACK parameter should also be set to a reasonable number (DEFAULT: 10).

Fast restarts with parallel ECLIPSE 100ECLIPSE 100 supports fast restarts for parallel runs with the following precautions:

1 The restart run must use the same number of processors and the same domain split as the base run

2 The eclrun macro requires that the number of processors be specified explicitly on the command line using the --np flag, for example:eclrun --np 2 RESTARTFILENAME

ECLIPSE 300 does not support fast restarts in parallel.

PARAOPTS2* 100 /

ECLIPSE Suite Installation Guide Running the softwareParallel ECLIPSE on Linux

29

Running parallel ECLIPSE with queueing systemsIt is generally advised to run both ECLIPSE and parallel ECLIPSE using a commercial queuing system. Only where one user is using a machine is it possible to adequately use the machine in a profitable way. The main advantage of using parallel ECLIPSE, which is speed, is lost if a machine is overloaded, as all parallel tasks will slow down. This usually results in run times longer than for serial ECLIPSE. Queuing systems will allow greater throughput of jobs by using the available resources more efficiently.

Support for a commercial queuing system is available in the eclrun macros. The queuing systems is:

LSF from Platform Computing - http://www.platform.com

The support in the macro provides for a limited number of the total available options for this platform, allowing queuing for jobs that specify only the number of processors for each job submitted. Additional support is provided in this queuing system for supporting memory requirements, later start dates, specifying groups of machines to execute on, etc. However, the macros provide a common route to queuing on these systems.

Running ECLIPSE software with LSFECLIPSE 100, ECLIPSE 300, parallel ECLIPSE, parallel ECLIPSE 300, ECLIPSE Office and FrontSim may all be run with LSF, using the standard simulator macros.

LSF SIS integration kit

Note Before installing the LSF integration kit, please read the documentation supplied with it.

This kit:

requires on-site configuration of LSF - refer to the integration kit documentation.

provides a dynamic method of license checking against the Resource requirement. FLEXlm is queried for licenses using an elim (External Load Information Manager), provided by Platform.

The integration kit and instructions can be found on the installation disk under the 3rdparty\LSF\resources directory.

Note If you are updating a version of LSF already installed, do not edit the scripts other than where documented.

If the resources can be met, the job runs; otherwise, LSF monitors the resources until they are met.


SetupThe following environmental variables can be set to control LSF functionality within the macros:

Ensure that you add the following to the user's .cshrc file: if (-d /lsf/conf) then

source /lsf/conf/cshrc.lsf endif

This assumes that LSF has been installed in directory /lsf; otherwise, amend as appropriate. See "Example .cshrc file" on page 16.

Note If you are using ECLRUN, then, to turn on License Aware Scheduling, set LSFLICENSES to TRUE in the eclrun.config file. See the "ECLRUN User Guide" for more information.

Using LSF with the simulator macros

Note See the "ECLRUN User Guide" for details of how to set up this command.

The use of LSF is triggered by adding the flag -lsf. This is ignored if ECL_LSF is true.

Examples for serial jobs:

eclrun -s localhost -q QNAME eclipse SPE9 eclrun -s localhost -q QNAME e300 CASE1 eclrun -s localhost -q QNAME frontsim ECL_SAMPLE01

for parallel jobs

eclrun -s localhost -q QNAME eclipse 50X50X6 eclrun -s localhost -q QNAME e300 20X20X6

The macros create a temporary job file ecl.tmp. that is used by LSF's bsub.The appropriate "resources" are generated by LSF.

How to choose the parallel executableOn Linux, the parallel executables for Intel MPI may be used. Intel MPI is available from 2009.1. See the examples below:

ECL_LSFHPCThis switches on the LSF HPC extensions. If you have the HPC version of LSF we recommend that you set this variable. For information see "Configuring LSF" on page 18.

ECL_MR_SCHEDULINGset this variable to correctly queue multiple realization jobs. Follow the instructions in the LSF-SIS integration kit.

ECLIPSE Suite Installation Guide Running the softwareParallel ECLIPSE on Linux

31

eclrun -s localhost -q QNAME -c ilmpi eclipse 50X50X6 eclrun -s localhost -q QNAME -c ilmpi e300 20X20X6

Note ilmpi means that Intel MPI executable will be used.plmpi means that the Platform MPI executable will be used.ibmmpi means that the IBM POE MPI will be used (POWER)

Checking a job's progress with bjobs The command bjobs can be used to show the status of all the user's jobs. bjob shows the status of a specific job, for example: PEND, RUN,

SUSPENDED, FAILED, DONE bjobs communicates with the LSF daemons, so if it is overused it can slow the system

down.

It is better to use

bhist -l as it reports from the event queue.

Checking a jobs progress

Checking a job's progress with bpeek You can also use bpeek to see output about a job while it is running. The command shows stdout and stderr.

NotesA host file can be used in place of a queuing system. If you do not specify a name for the host file, a host file containing the name of the current host is auto-generated. This file can then be updated using the relevant formats:

The standard hostfile format is of the form Node01Node02Node03

linux-cluster{hkw}[204]: bjobs 3408JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME3408 saf RUN normal linux-clust comp00.geoq *K2.175640 May 22 08:59

comp05.geoquestlinux-cluster{hkw}[205]: bjobs 3414JOBID USER STAT QUEUE FROM_HOST EXEC_HOST JOB_NAME SUBMIT_TIME3414 grb RUN parallel linux-clust comp06.geoq T May 22 13:51

comp09.geoquestcomp02.geoquestcomp01.geoquest

linux-cluster{hkw}[206]:


Node04 This will run a process on each node in turn.

In the case of nodes with twin CPUs, to specify that a process is to be run on each CPU, use a hostfile of the formNode01Node01Node02Node02

ECLIPSE Suite Installation Guide Microsoft HPC installation Installation

33

Chapter 5Microsoft HPC installation

InstallationThis guide is designed to take you through how to install and configure Microsoft Windows Server 2012 HPC on the cluster side and on the laptop/PC side. This allows users to run simulation jobs from a laptop/PC and submit to a Windows cluster. This would normally be done through ECLIPSE Office or Petrel using the eclrun command. It can also be done using the eclrun command directly from the command line and indeed that would be the preferred option when testing a connection and/or debugging the system. Refer to the "ECLRUN User Guide" for the full definition of all the switches.

Note The Microsoft C routine libraries must be installed on all nodes in the cluster. Install both 32-bit and 64-bit runtimes from the vcredist_vs9 and vcredist_vs10 directories which can be found on the DVD in the \3rdparty\PC directory.

Install the Windows cluster following the instructions from Microsoft. After installation, run all of the diagnostics and ensure that everything is working properly before making the following changes specifically for ECLIPSE:

"Checking the InfiniBand driver firmware" on page 34

"Configuring cluster information on the head node" on page 35

"Configuring nodes" on page 37

"Configuring the HPC web services interface on the head node" on page 38

"Using the license aware activation filter" on page 39

"Configuring client PCs" on page 52

Installation notes

Installing Intel MPI on the cluster is not recommended.

There is a caveat when selecting passwords for the cluster user because ECLRUN cannot pass passwords with certain restricted characters in them. See the "ECLRUN User Guide" for more information.

34 Microsoft HPC installation ECLIPSE Suite Installation GuideChecking the InfiniBand driver firmware

Checking the InfiniBand driver firmwareWe have found that the current InfiniBand (IB) drivers released from Mellanox require the firmware on the cards to be at the latest level in order to work with HPC. In most cases you can download the firmware from the Mellanox website www.mellanox.com but you may need to determine what type of card you have. To do this:

Download the Mellanox Firmware Tools (MFT) from http://www.mellanox.com/content/pages.php?pg=management_tools&menu_section=34

Follow the instructions to get your card type.

Then download the correct firmware for your card.

It is important that you do download the correct firmware or you can risk damaging your card. The firmware needs to be installed before the IB drivers. The current generation of IB drivers (2.2.1) can be obtained from www.openfabrics.org for the generic drivers. There may be specialist drivers for certain modern IB cards that you will need to obtain directly from the Host Channel Adapter (HCA) manufacturers. You need to check on this with your card manufacturer.

Once the drivers are installed check that you can see the IB HCAs in the device manager. If you cannot see them, the HPC installation will not see them.

ECLIPSE Suite Installation Guide Microsoft HPC installation Configuring cluster information on the head node

35

Configuring cluster information on the head node In order to use the ECLIPSE software with the cluster a certain number of environment

variables need to be set. These need to be set as administrator on the cluster head node.

For Windows Server 2008 R2, you do this by running:

For Windows Server 2012 some of these variables have been deprecated and you run:

The location defined by the ECLPATH variable must be shared and accessible by all users and machines that will submit to the cluster and/or run cluster jobs.

If you intend to submit Multiple Realization (MR) jobs to the cluster, you need to set the MR scheduling environment variable: cluscfg setenvs ECL_MR_SCHEDULING=trueThis has to be done in conjunction with adding the following line to the eclrun.config file on each client machine:

True The DefaultHoldDuration parameter specifies the time between license checks. By

default, if no licenses are available, the scheduler will wait 900 seconds before checking again. Setting this to 90 is recommended:cluscfg setparams DefaultHoldDuration=90

To check the environment variables settings, use

cluscfg listenvsThis sets the variable for all users and all compute nodes. If these are not set, ECLRUN will return an error. These settings require HPC Service Pack 3 to work.

Note Note the use of UNC paths. This is vital to the current implementation of HPC as we do not install ECLIPSE on the compute nodes but only on a file system that must be visible to all the compute nodes and the head node. For ECLIPSE 2013.2 all the required DLLs are installed not only in the lib directory as previously but also in the bin directory. This means you do not have to install any ECLIPSE software on the compute nodes.

cluscfg setenvs SLBSLS_LICENSE_FILE=7321@licenseservercluscfg setenvs ECLPATH=\\139.10.10.1\eclcluscfg setenvs MPIEXEC_AFFINITY=1cluscfg setparams AffinityType=NoJobscluscfg setenvs MPICH_ND_EAGER_LIMIT=2000000000cluscfg setenvs MPICH_SHM_EAGER_LIMIT=2000000000

cluscfg setenvs SLBSLS_LICENSE_FILE=7321@licenseservercluscfg setenvs ECLPATH=\\139.10.10.1\eclcluscfg setenvs MPIEXEC_AFFINITY=1cluscfg setparams AffinityType=NoJobscluscfg setenvs MSMPI_CONNECTIVITY_TABLE=1cluscfg setenvs MSMPI_ND_EAGER_LIMIT=2000000000cluscfg setenvs MSMPI_SHM_EAGER_LIMIT=2000000000

36 Microsoft HPC installation ECLIPSE Suite Installation GuideConfiguring cluster information on the head node

If you want to alter any of the system parameters you can find those that are available using the command:

mpiexec help, mpiexec help2 or mpiexec help3Do not reset too many of these options.

Set the license server by IP address rather than name, although both should work.

We also require that wherever the data is stored it must also be visible to the compute nodes for both read and write. The ECLRUN program will check this before it submits a job.

For HPC the eclrun command takes the form:eclrun s localhost q -u eclipse It can also be useful to set the debug option when testing the connection:

eclrun s localhost q -u --debug=both eclipse

Note The server for submitting to HPC is always localhost. This option can only be used on systems where the HPC client pack has been installed (at the very least).

ECLIPSE Suite Installation Guide Microsoft HPC installation Configuring nodes

37

Configuring nodes

Compute nodes and head node file system definitionIf you are using normal storage, that is the SMB2 file system, then you need to run PowerShell as an admin user and run this command (a single command line):

Set-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Services\LanmanWorkstation\Parameters" RequireSecureNegotiate -Value 0 -ForceYou must run this command on all compute nodes and the head node. If you don't you will not be able to access storage that uses SMB2 file systems, which is the most used file system currently. This is because Windows Server 2012 (and Windows 8) use the new SMB3 file system.

VCC library installation on compute nodesYou need to make sure that all the required VCC libraries are installed on all compute nodes. These can be found on the ECLIPSE DVD in the following locations: 3rdparty\PC\resource\vcredist_vs93rdparty\PC\resource\vcredist_vs10

Windows power configurationOn Windows HPC it is necessary to set the power configuration to high performance. Not setting this correctly can have a 15% performance impact on simulation runs.

To perform this configuration, you must have admin rights on the head node and all of the cluster nodes.

To see what configuration you have type powercfg -l in a DOS prompt on the head node. It should return:Existing Power Schemes (* Active)-----------------------------------Power Scheme GUID: 381b4222-f694-41f0-9685-ff5bb260df2e (Balanced)Power Scheme GUID: 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c (High performance) *Power Scheme GUID: a1841308-3541-4fab-bc81-f71556f20b4a (Power saver)If you do not have high performance set, you will not see the entry marked with an asterisk (*). To add this setting, use the clusrun command in node management to run the command on all cluster nodes:clusrun /all powercfg -s 8c5e7fda-e8bf-4a96-9a85-a6e23a8c635c

38 Microsoft HPC installation ECLIPSE Suite Installation GuideConfiguring the HPC web services interface on the head node

Configuring the HPC web services interface on the head node

The Web services or REST interface allows users to monitor their jobs through a remote interface as well as monitoring cluster load. Monitoring the cluster load through the basic Job Manager interface. It is also possible to set up a job submission web page for ECLIPSE that is similar to that provided by EnginFrame. The web interface may provide useful functionality, subject to some limitations. To install the web interface:

1 Install Web services components from

Program files\Microsoft HPC Pack 2012\Bin\HpcWebComponents2 Run a Powershell Window as administrator

Note that Powershell scripts and arguments are case sensitive

3 To set unrestricted policy executions so that you can run scripts:

a To check the current execution policy settings, run the command

Get-ExecutionPolicy -Listb If the execution policy is restricted, run the following command to set it to unrestricted:

Set-ExecutionPolicy -ExecutionPolicy unrestricted4 Change to the \bin directory:

Cd program files\Microsoft HPC Pack 2012\bin5 Enable the service portal.

If you want to use the default port number of 443, type:.\Set-HpcWebComponents.ps1 -Service Portal -enableIf you want to use a different port, type:.\Set-HpcWebComponents.ps1 -Service Portal -enable -Port

6 When prompted for the certificate, enter 0 (zero) for a self-signed certificate.7 Then enable the REST interface making sure that you use the same port number as step 5:

.\Set-HpcWebComponents.ps1 -Service REST -enableor .\Set-HpcWebComponents.ps1 -Service REST -enable -Port

8 To restart the scheduler, type:

Net stop hpcschedulerNet start hpcscheduler

9 To test access to the web interface, go to another machine and type

https:///HpcPortalIf you cannot access the web service, repeat the setup procedure.

10 After setting up web access you should restrict execution policy again by running the command:

Set-ExecutionPolicy -ExecutionPolicy restricted

ECLIPSE Suite Installation Guide Microsoft HPC installation Using the license aware activation filter

39

Using the license aware activation filter

IntroductionThe License-Aware Activation Filter is a program that prevents jobs from being run without the correct amount of required licenses. You can also use it to create rules to control how many jobs users can run and at what times.

This program installs an Application Filter in the Windows HPC 2008/2012 Cluster Manager Job Scheduler.

InstallationThe program files are in a zipped file called LicenseAware.zip in the 3rd party applications directory (3rdparty\PC\resources\). You simply drag the zip file from this directory to a directory of your choosing on the cluster head node. Unzip the file to install the individual program components, which are:

User Windows Interface (HPCLicFilterUI.exe) The Filter program that will being installed on Windows HPC 2008/2012

(HPCLicFilter.exe) Special configuration file (HPCLicFilterUI.exe.config)Make sure that you put all of these components into the same directory.

Filter configuration

The filter configuration windowThe filter configuration window provides an interface to the filter configuration settings. You must be an administrator to configure the filter.

To use the window, double click on the HPCLicFilterUI.exe program. This displays the MSHPC:License Aware Activation Filter window.

40 Microsoft HPC installation ECLIPSE Suite Installation GuideUsing the license aware activation filter

Figure 5.1 The filter configuration window

The main areas on the window are:

The top part of the window is for FLEXlm configuration

The lmstat argument string is hardcoded, but is displayed by the window for information.

The area below the Windows HPC Activation Filter heading is for the filter.

The main window controls are

File | Save Config: This saves your current configuration for future use. Only one configuration will be allowed.

File | Exit: This exits from the tool.

Help | About: Provides information about the version of the program.

Apply: This button must be used each time you modify any configuration in order to validate the new configuration. This button also will save your configuration.

Exit: This button has the same effect as File | Exit. Note that this does not save the configuration.

There is also a status bar at the bottom of the window which provides information about the programs activities.

Configuring FLEXlm1 Open the Filter Configuration tab in the MSHPC:License Aware Activation Filter

window.


41

Figure 5.2 Filter configuration

2 Click on the Browse button and use it to find the lmutil.exe program.The default location is in the ecl/macros directory.

3 Click on Apply.

This applies the default command lmstat a c using the executable that you located.

Configuring a license server1 Open the Filter Configuration tab in the MSHPC:License Aware Activation Filter

window. Figure 5.3 License configuration panel

2 Enter the service port and the machine on which the license is working.

When a valid license or service port is given, the rest of the controls will become available:

Confirm/Test: This allows you to test your configuration.

If the connection to the license server was successful, the Check Log button is green, indicating a positive response

If the connection with the license server was unsuccessful, the window displays a warning message and the Check Log button is red. You need to check the logs and correct the issue.

If the license server details are not valid, the Enable Filter on Windows HPC option on the window is disabled.

Check Log: Clicking on this opens a new window with the output text from running the command lmutil lmstat.

Configuring the activation filterIn this section, you can enable or disable the activation filter, determine the level of information provided, where to output the log (debug) file and what to do if there are not enough licenses. The settings in this window automatically configure the Windows HPC 2008/2012 activation filter.


Figure 5.4 Filter settings

Enable filter on Windows HPC

Setting this checkbox on attempts to activate the filter executable (HPCLicFilter.exe) in the Windows HPC Filter configuration. If that is successful, the bottom of the window shows the successful activation, but no other messages are displayed. If you see error messages, this indicates that the activation was not successful. Please see "Dealing with errors" on page 50 for information on how you might correct the issue(s).

Info Level - This defines the level of information to be provided by the filter messages:

INFO: Shows only information messages about the behavior of the filter.

DEBUG: Shows useful messages to trace the program procedures and get information about jobs and parameters.

The default is INFO

This parameter can be changed anytime.

Debug file name

You need to supply the name of a filter log in order to get all messages from the activation filter. You can use the browse button to define an existing file, or you can define a new one in the prompt window.

To set this parameter you must uncheck the Enable filter on Windows HPC option. After changing the log file information, you can set the option back on.

Setting the default action when licenses are not availableYou can choose what happens when licenses there are not available using the If the job have not enough licenses options.You can change this parameter at anytime, but you need to press APPLY to make a change.

Run the job do not check license

The scheduler will attempt to run the job even if no license is available. It has the same effect as disabling the filter.

Stop the queue until a license is available

Do not run the job. Keep the resources and do not run anything else until the current job starts. This setting ideal for configuring a first come first served queue.

Note: This status will block the queue, and will not allow any other job to run until the current one can run.

Hold the job and Keep Resources allocated

Do not run the job. Keep the resources allocated for this job, but other jobs may be started on other resources.


43

Hold the job and do not reserve any resources

Do not run the job and do not reserve any resources. The job is put on hold for a period of time and the scheduler will not attempt to schedule the job again until the hold time has passed. (By default, the hold time is two minutes, but you can change this at the HPC Manager configuration console.)

Fail the job

The filter will cancel the current job. In this way the queue will not be blocked, but the job will be canceled.

License reservationsYou can use the Enable license reservations and Hold reservation only for settings to help to avoid racing conditions with license allocation.

Figure 5.5 License reservation configuration

Without these settings, the filtering sequence is:

1. The ECLIPSE job is submitted.

2. HPC calls to the installed custom filter

3. The filter checks for all licenses needed for this job.

The filter can only check for licenses but cannot reserve any.

4. The filter decides what to do according to the number of available licenses.

It can release or run the job (give a green light to current job to run)

5. HPC gets the return code from the filter

It submits, holds or cancels the job depending on the filter return code.

6. HPC then goes to the next job and repeats the cycle.

It is not clear how long a job will take for initialization. In this phase, a job (ECLIPSE submission job) is not getting any licenses and it is during initialization that racing conditions can arise. For example, a new job may get a green light for licenses while the first job is still initializing. If this new job has a faster initialization phase than the first one, it takes over any licenses that the previous job did not take because it is still initializing. Alternatively the second job could fail if the first job is initialized first and takes the licenses, in which case there will not be any available for the second job.


Using the reservation buttons, you can instruct the filter to reserve a license for the job that is being submitted and for how long to hold the reservation. This way the license is waiting until the job can take it. This reservation is taken into account by the activation filter (theres no real reservation done using FLEXlm). The filter keeps these reservations until the allotted time has passed and then deletes the reservation automatically.

Enable MR Checking

You can use this filter to check for MR (Multiple Realization) licenses and make appropriate adjustments. If this is set on, the filter determines whether or not the simulation is a MR simulation and assigns the appropriate licenses.

For MR checking "Configuring cluster information on the head node" on page 35 specifies the MR environment variable that needs to be set on the head node. In addition, the eclrun.config file must contain the entry:True

Configuring user and time restrictionsIn order to provide a higher level of control, you, as a system administrator can configure the filter to limit the number of jobs specific users can run and the number of cores those jobs can use.

You can also configure the filter such that the user job/core restrictions are only active at certain times of day and certain days of the week. For example the filter could be configured using restrictive rules to ensure each user can only run 1 job using a maximum of 8 cores during peak times (Monday-Friday, 0900-1700). During off peak times all the rules can be lifted and the cluster serves all jobs as they are encounter (subject to sufficient licenses being available).

Rules can be specified as a global (default) rule for all users, individually, or for groups of users to speed up entry of the rules.

Using the User Rules tabTo use rules, check the Use Rules checkbox to enable the rest of the form.


45

Figure 5.6 The User Rules tab

Use Active Days to define which days the rules defined will be enforced. By default All is checked, meaning that the rules are valid every day. You can also specify specific days for finer control.

Use Active Times to define the times that the defined rules are active. Checking the 24H checkbox enables the rules all day for the days defined in the Active Days section. The times must be specified in the hhmm format and be valid times. Times can span into the following day. For example to have the rules active overnight you could specify 2000 - 0800.

Use Default Rules to define a rule that applies to all users that do not have a specific rule configured. This can be enabled or disabled for finer control.

The Custom Rules box shows any defined rules. For example:Figure 5.7 Custom rules populated with properties

Creating a new ruleTo create a new rule, click the New Rule button. This displays the Create a New Rule window.


Figure 5.8 Creating a New Rule screen

From this screen you can define custom rules for individual users, or to speed up input, groups of users. When adding a group of users you must specify a name for the group and specify the users for which the rule will apply in the box using a comma separated list. For example:

corp\user1,corp\user2,corp\user3

Click the Add user or Add group button and the new entry will appear in the Custom Rules box.

Editing and deleting rulesTo edit a rule, select the rule from the list and click edit. This opens the new rule form populated with the editing rule. Make the required changes and click the add button to save the changes. When editing an entry you must click the add button else the data will be lost for the current rule.

To delete a rule simply select the required rule and click delete.

Editing the user.config file directlyYou can edit rules directly in the user.config file in the installation directory for the filters using the following syntax:

ACTIVETIME|hhmm:hhmmACTIVEDAYS|DDD[:DDD:DDD:DDD...]DEFAULT|JOBS:n|CORES:nUSER|JOBS:n|CORES:nGROUP:name|USERS:DOMAIN\username[,DOMAIN\username,....]|JOBS:n|CORES:n


47

Checking FLEXlm license details

Checking details by featureUse this option to see all of the features in your current license.

1 Open the MSHPC:License Aware Activation Filter window.

2 Click on the Flexlm License Details tab.

3 Click on the By Feature tab.Figure 5.9 FLEXlm license checking by feature

Clicking on Refresh polls the license server and brings back all the current features available. For example:

Eclipse (56/1000/944)

Taking this example, each part means:

the used license amount (56)

the total license amount(1000)

the available license amount (944)

If you select any feature in this list, you get a list of all current users using this feature, including

the user alias

the workstation requesting the feature


the version of the feature that is being used

which server is serving the feature

the time at which the service was requested

Checking details by userUse this option to see all of the current users using the licenses.



3 Click on the By User tab.Figure 5.10 FLEXlm license checking by user

Clicking on Refresh button polls the license server and brings back a fresh list of users.

After that, you can select any user, and you can then see which features this user is using and the number of licenses used for each feature.

Checking FLEXlm license reservationsUse this option to see the license reservations made by the License Aware Scheduler. The information is updated automatically once you have set it up using the Attach to log setting described in "Checking filter logs" on page 49.

Use this option to see all of the current license reservations.



49


3 Click on the License Reservations tab.Figure 5.11 License reservations checking

These reservations displayed are read-only, so you cannot modify them. They are managed automatically by the License Aware Scheduling Filter.

Checking filter logsTo look at the License Aware Scheduling Filter messages:


2 Click on the Filter Logs tab.


Figure 5.12 HPC filter log screen

The messages are displayed in the HPC Filter Log pane. You can use the window buttons to determine what to do with the messages:

Clear: This option clears the log window, but not the log file.

Attach to log: This allow you to see real time messages from the Filter.

Using this setting, every time the filter is activated (by a submitted job), the information about license reservations is automatically updated. See "Checking FLEXlm license details" on page 47. If you do not check on this setting, you have to use the refresh button to update the license reservation information.

Deattach: This button detaches from real time log gathering.

The tool clears the scroll bar history when it reaches 1000 lines in length to reduce the amount of memory used to bring the information on line. This means that the tool brings in the last 1000 lines from the log file.

Dealing with errors

Setting the default failure statusTo set the default failure status:


2 Click on Help and select Default return failure status.


51

This displays the License Aware Scheduler default options window: Figure 5.13 License aware scheduler default options

3 Select the required action and click on Ok to set it.

Error conditionsThe program errors are self explanatory, and the messages include a possible solution for each case. Some recommendations on operating the filter are to:

Install all the Filter software (HPCLicFilter components) into the same directory. Run the MSHPC:License Aware Activation Filter application at the cluster head node as

Administrator, in order to configure it correctly.

Monitor the size of the log file. You could leave the log information level as INFO to avoid generating a long log file. See "Configuring the activation filter" on page 41.

52 Microsoft HPC installation ECLIPSE Suite Installation GuideConfiguring client PCs

Configuring client PCsAfter installing the HPC client pack, you have to add an environment variable on client PCs that are running Petrel and the Simulation Launcher. On each client PC, add:CCP_SCHEDULER=If you don't set the CCP_SCHEDULER environment variable, you get an error of the form:

If you dont set the scheduler, ECLRUN and remote job submission will not work.

D:\>cluscfg listenvsNo connection could be made because the target machine actively refused it 127.0.0.1:5800

ECLIPSE Suite Installation Guide Legacy macros and scriptsAbout the macros and scripts

53

Appendix ALegacy macros and scripts

About the macros and scripts

Note It is recommended that ECLRUN is used to run most of the simulators and other programs. Please refer to the "ECLRUN User Guide" for more details. The legacy macros are still released but may have limited capabilities compared to ECLRUN.

For Linux systems, the software can be run using scripts supplied in the macros directory in place of ECLRUN. The following scripts are available to run the License Manager:

The default location for the following files for use with the License Manager is the ecl/macros directory

The following macros are available to run the principal simulator software programs:

@flexstart Start up the License Manager@lmdown Shut down the License Manager@lmhostid Get FLEXlm hostid for the machine@lmreread Reinitialize License Manager@lmstat Check license status

license.dat License fileflex.log Activity log for License Manager

@e300 Compositional simulator@eclipse Black oil simulator@frontsim Streamline simulator@graf Post processor@grid Gridding and mapping@mpieclipse Run ECLIPSE 100 simulator in parallel mode@mpie300 Run ECLIPSE 300 simulator in parallel mode

54 Legacy macros and scripts ECLIPSE Suite Installation GuideAbout the macros and scripts

The following macros are available to run the utility programs (those marked * are no longer released):

Batch operations:

File format conversions:

File manipulation:

File name changes:

Viewing:

Note Not all programs are supplied with the 2013.2 release; some older applications have been retired. The scripts have been left so they can be used to run older versions.

The following environmental variables can be set to control LSF functionality within the macros:

@fill Corner point geometry generation

@convert Converts formatted / unformatted ECLIPSE output@convertgrid Converts between .GRID and .EGRID formats@dos2unix Converts ASCII from MS-DOS to UNIX format@rescueconvert Converts from older versions to RESCUE Version 19 format@unix2dos Convert ASCII from UNIX to MS-DOS format

@expand Merge INCLUDE files into master file@extract Extract subset of data from ECLIPSE output@glkread Checks the contents of a Stratamodel binary .glk file

@change_prefix Changes prefix of file name@change_suffix Changes suffix of file name@lower Lowercase file name@rename Change ECLIPSE filename case and extensions@upper Uppercase file name

@manuals Launches Manuals bookshelf@pdf Launches Adobe Acrobat

ECL_LSF set true if LSF is installedECL_LSF_BYDEFAULT

If this and ECL_LSF are set to TRUE then jobs are submitted using LSF without the need for the -lsf flag. For compatibility with ECLIPSE Office the -nolsf flag has been added as a new option to stop nested bsub commands.

ECL_LSFHPCThis switches on the LSF HPC extensions. If you have the HPC version of LSF we recommend that you set this variable. For information see "Configuring LSF" on page 18.

ECL_LSF_PARALLELQdefines the default queues for parallel jobs

ECLIPSE Suite Installation Guide Legacy macros and scriptsAbout the macros and scripts

55

ECL_LSF_SERIALQdefines the default queues for serial jobs

ECL_LSF_LICCHECKset if LSF is to control FLEXlm licensing and the LSF-SIS integration kit has been installed (contact Platform Computing www.platform.com for details).

ECL_MR_SCHEDULINGset this variable to correctly queue multiple realization jobs. Follow the instructions in the LSF-SIS integration kit.

56 Legacy macros and scripts ECLIPSE Suite Installation GuideMacros: command line flags

Macros: command line flagsThe Linux macros are written using c-shell scripts. It is essential that the macros directory is placed on the users path; this is not necessary, however, to ensure their successful execution. The macros will automatically detect all versions of software installed, and will invoke questions as necessary to launch the program required.

The macros require little or no configuration to launch interactive products. However, it is possible to configure the macros using command line flags; this is especially useful for running the ECLIPSE simulators. The main command line flags for running the ECLIPSE 100 and ECLIPSE 300 simulators are listed below:

For parallel ECLIPSE, namely @mpieclipse, @mpie300 the following special options are available:

See "Using LSF with the simulator macros" on page 30.

If -procs is not set, the macro will attempt to read the PARALLEL keyword in the dataset to get the number of processors.

-data /home/user/workarea to set default working directory, or other directory.-file to define a dataset file name. -help to obtain a list of available flags with brief description.-second to define a secondary run title.

should be enclosed in quotes.-fore or -foreground or -fg defines a foreground run (the default)-back or -batch defines a background (queued) run-priority defines priority of a background run-ver or -vers or -version defines version of software to run, for example:

-ver 2005a _1 -file TESTCASE-proc or -procs or -processors defines number of processors for a Parallel run.-local uses the local configuration file ECL.CFG.

-hostfile hosts.txt Configuration file for Parallel run (same as -machinefile)

-machinefile hosts.txt Configuration file for Parallel run (same as -hostfile)

-lsf For runs using LSF.

-ilmpi Option for processing parallel jobs using Intel MPI-plmpi Option for processing parallel jobs using Platform MPI-ibmmpi Option for processing parallel jobs using IBM POE MPI

-nolocal Do not run on front end when a parallel job is run.

ECLIPSE Suite Installation Guide Legacy macros and scriptsExamples

57

Examples@eclipse -version 2013.2 -local -lsf TESTCASE @mpie300 -lsf -lsfqueue normal -version 2013.2 -local TESTCASE

58 Legacy macros and scripts ECLIPSE Suite Installation GuideExamples

ECLIPSE Suite Installation Guide Index 59

Appendix BIndex

BBatch mode

PC . . . . . . . . . . . . . . 27

CConfiguring PC software . . 11

cshrc . . . . . . . . . . . . . . . . 16

IInstalling

Parallel ECLIPSE . . . 28PC software . . . . . . . . 10

Intel MPI . . . . . . . . . . 11, 16

LLinux . . . . . . . . . . . . . . . . 13

Linux systemsmacros and scripts . . . 28

LSF SIS integration kit . . . . 29

MMacros

command line flags, Linux56Linux systems . . . . . . 28

Microsoft HPC . . . . . . . . . 33

PParallel ECLIPSE

Installing . . . . . . . . . 28Linux . . . . . . . . . . . . 28Setting up . . . . . . . . . 28

PC launcher . . . . . . . . . . . .11

PC software . . . . . . . 10 to 11

SScripts

Linux systems . . . . . . 28

Software products . . . . . . . . 7

Software requirementsLinux . . . . . . . . . . . . 28

Standalone installation . . . .11Windows XP . . . . . . . .11

Supported platforms . . . . . . 7

60 Index ECLIPSE Suite Installation Guide

ECLIPSE Suite Installation GuideTable of contentsList of figuresList of tables

Simulation software system requirementsGeneral requirementsSoftware products and supported platformsNotes

Simulation software installationDisk organizationInstalling PC software from diskInstallation processConfiguring PC softwareInstalling Intel MPIThe PC simulation launcher

Directory structureInstalling Linux software from diskPreparing to install 2013.2 simulation softwareECLIPSE and Intel MPIInstallation processUser manuals and on-line help

Directory structure

LicensingInstalling Schlumberger licensing

Running the softwareRunning ECLIPSE in batch mode on a PCParallel ECLIPSE on LinuxLinux systems - macros and scriptsInstalling and configuring parallel ECLIPSENotes on setting up the OS for parallel ECLIPSERunning ECLIPSE software with LSF

Microsoft HPC installationInstallationChecking the InfiniBand driver firmwareConfiguring cluster information on the head nodeConfiguring nodesConfiguring the HPC web services interface on the head nodeUsing the license aware activation filterIntroductionInstallationFilter configurationConfiguring user and time restrictionsChecking FLEXlm license detailsChecking filter logsDealing with errors

Configuring client PCs

Legacy macros and scriptsAbout the macros and scriptsMacros: command line flagsExamples

Index

Instal Guide

Documents

Transcript of Instal Guide