Installation and Setup for IBM InfoSphere Streams V4.0

© 2015 IBM Corporation

Install and Setup

IBM InfoSphere Streams Version 4.0

Laurie A Williams

Install developer

<For questions about this presentation contact [email protected]>

mailto:[email protected]

2 © 2015 IBM Corporation

Important Disclaimer

THE INFORMATION CONTAINED IN THIS PRESENTATION IS PROVIDED FOR INFORMATIONALPURPOSES ONLY.

WHILE EFFORTS WERE MADE TO VERIFY THE COMPLETENESS AND ACCURACY OF THEINFORMATION CONTAINED IN THIS PRESENTATION, IT IS PROVIDED “AS IS”, WITHOUT WARRANTYOF ANY KIND, EXPRESS OR IMPLIED.

IN ADDITION, THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY,WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.

IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OROTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.

NOTHING CONTAINED IN THIS PRESENTATION IS INTENDED TO, OR SHALL HAVE THE EFFECT OF:

• CREATING ANY WARRANTY OR REPRESENTATION FROM IBM (OR ITS AFFILIATES OR ITS ORTHEIR SUPPLIERS AND/OR LICENSORS); OR

• ALTERING THE TERMS AND CONDITIONS OF THE APPLICABLE LICENSE AGREEMENTGOVERNING THE USE OF IBM SOFTWARE.

IBM’s statements regarding its plans, directions, and intent are subject to change orwithdrawal without notice at IBM’s sole discretion. Information regarding potentialfuture products is intended to outline our general product direction and it should notbe relied on in making a purchasing decision. The information mentioned regardingpotential future products is not a commitment, promise, or legal obligation to deliverany material, code or functionality. Information about potential future products maynot be incorporated into any contract. The development, release, and timing of anyfuture features or functionality described for our products remains at our solediscretion.

THIS INFORMATION IS BASED ON IBM’S CURRENT PRODUCT PLANS AND STRATEGY, WHICH ARE SUBJECT TO CHANGE BY IBM WITHOUT NOTICE.

IBM SHALL NOT BE RESPONSIBLE FOR ANY DAMAGES ARISING OUT OF THE USE OF, OR OTHERWISE RELATED TO, THIS PRESENTATION OR ANY OTHER DOCUMENTATION.


Agenda

Installation

Domain Manager

Setting up a multi-host environment

Demo


Installation

Two installation packages

– Main (full product)

• Download from PPA

– Domain host

• Created from main installation

• Subset of the product

Selection of the edition to install

– User must select the edition to install

– Use streamtool updatelicense to update the license from Developer or Non-Production to Product

Default installation directory

– Install as user root: /opt/ibm/InfoSphere_Streams

– Install as non-root user: ~/InfoSphere_Streams

Version directory appended to the root installation directory specified during the installation

– Each new version, fixpack, interim fix is installed to different version directory.

– Can use same root directory for different versions, but must have same installation owner and group.

– User can install new code or fix without having to stop all applications, instances and domains.

– Installation root directory should be different than Version 3.2.1.x or prior.

Updated software dependencies

Supported Java development kits for x86_64 and Power

– Dropped Java 1.6

– Product runs on Java 1.7

Platforms supported (separate installation package for RH6, RH7, SLES)

– Dropped RHEL 5

– Dropped RHEL 6.2 for PPC

– Added RHEL 7

– Added POWER8 for PPC

SELinux

– Removed the InfoSphere Streams SELinux policy module.

– No installation restrictions when SELinux is enabled.

Ulimit

– Increase in minimum value for: Max user processes >= 5000

– Same minimum value for: Open files >= 1024

– Captured with Streams logs

– Reported by dependency checker and checkhost


Uninstallation

• Uninstallation program for each version

• Before uninstalling:• Stop all applications, instances and domain using this version

• Uninstallation code checks for active code only on the current host

• Cannot detect if code is active if installed in a shared file system

• If the system service is defined using this installation need to do an streamtool unregisterdomainhost

• Uninstallation log placed in home directory

• Only console and silent mode


Domain Manager

• Replacement for First Steps application

• Use combination of Domain Manager and Streams Console to do setup

• User interface to do set up, manage multiple domains for multiple Zookeeper installations

• Create basic and enterprise domains

• Remove

• Start

• Stop

• Edit domain properties

• Open Streams Console

• Generate public/private keys for a domain

• Install streams studio on local system

• Only runs native to host launched from

• How to open Domain Manager:

• Main installation post installation task

• source streamsprofile.sh; streamtool launch --domainmgr

• Script in <install-dir>/<version>/DomainManager.sh (does source and launch command)

• Caches application settings which can be changed in the application.


Multi-host environment setup (HA)(Preferred setup running domain controller as a system service.)

• Installation options:• Install the product on each host.

• Install the main installation package on one or more hosts and the domain host installation package

on all additional hosts.

• Install the product in a shared file system directory that is accessible from each host.

• Set up the domain controller service as a Linux system service on all hosts. This service

provides automatic recovery for the streams services from host failures. The product

provides interfaces to configure and manage the Linux system service.

• Requires user root to install domain host installation package and set up the system service.

• For managing and storing configuration information, InfoSphere Streams requires an Apache

ZooKeeper server.

• For authentication use a Lightweight Directory Access Protocol (LDAP) server or Pluggable

Authentication Module (PAM) with the LDAP backend for user authentication (or other

backend that can be setup as highly available).

• Configure domains and instances to have standby domain and instance services.


Pre-Version 4.0 Compatibility Setup(For users who already have Shared file system and SSH setup and do not want to do move to running

domain controller as system service.)

• Installation options:

• Install the product in a shared file system directory that is accessible from each host.

• Install the product on each host• Install the product in the same location on each host.

• The installation owner and group must be the same on each host.

• The same version of the product must be installed on each host.

• If the installation is shared by multiple users, you must start the domains as the installation

owner.

• Set up SSH for each user of the product.

• No immediate recovery of Streams services if host crashes.

• For managing and storing configuration information, InfoSphere Streams requires an Apache

ZooKeeper server.

• For user authentication, use a Lightweight Directory Access Protocol (LDAP) server or

Pluggable Authentication Module (PAM).


Demo


streamtool commands to manage system service

registerdomainhost – registers a host with a domain, installs the domain controller

as a system service, starts the system service

stopdomainhost – stops the domain control system service on this host

startdomainhost – starts the domain control system service on this host

unregisterdomainhost – removes the resource from the domain, removes the

domain controller system service

getdomainhoststatus – returns the status of the domain controller system service

For information on streamtool commands, run streamtool man <command> or

see:http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.

ea.doc/streamtool/doc/streamtool.html.

http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.ea.doc/streamtool/doc/streamtool.html


References

streamsdev articles– https://developer.ibm.com/streamsdev/docs/multi-host-environment-installing-host-setting-domain/

– https://developer.ibm.com/streamsdev/docs/streams-4-0-enterprise-install-and-setup-videos/

Knowledge Center documentation– For considerations on setting up a multi-host environment see:

http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install.doc/doc/ibminfospherestreams-

install-options-configuration.html.

– For more information on setting up the enterprise domain and setting up the domain controller to run as a Linux system

service see: http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.cfg.doc/doc/setting-up-

enterprise-domain.html.

– For information about system and software dependencies see:


install-prerequisites.html.

– For information about installing the main installation package, see:


install.html.

– For a description on the tags see, http://www-

01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.welcome.doc/doc/services.html.

– For additional information on setting up the tags see http://www-

01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.admin.doc/doc/movingtags.html

https://developer.ibm.com/streamsdev/docs/multi-host-environment-installing-host-setting-domain/

https://developer.ibm.com/streamsdev/docs/streams-4-0-enterprise-install-and-setup-videos/

http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install.doc/doc/ibminfospherestreams-install-options-configuration.html

http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.cfg.doc/doc/setting-up-enterprise-domain.html

http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install.doc/doc/ibminfospherestreams-install-prerequisites.html

http://www.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.install.doc/doc/ibminfospherestreams-install.html

http://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.welcome.doc/doc/services.html

http://www-01.ibm.com/support/knowledgecenter/SSCRJU_4.0.0/com.ibm.streams.admin.doc/doc/movingtags.html


Questions?


Backups


Summary of setup for HA environment(Instructions for installing all hosts then creating the domain.)

• Set up external Zookeeper; and set STREAMS_ZKCONNECT to the connection string

• Set up authentication mechanism (LDAP or PAM over LDAP)

• Using the main installation package install the product.

• Run ‘streamtool registerdomainhost’ command on the main management host

• Create the domain host package using “streamtool mkhostpkg”• Creates a tar file with the domain host installation binary, dependency checker script,

streamsdomainhostsetup.sh script, response file, etc…

• For each additional host that you want in the domain• Copy the domain host package to the host

• Extract the contents of the domain host package

• Run the dependency checker script

• Run the streamsdomainhostsetup.sh script

• Installs the boot strap code

• Registers the host with the domain, create the system service, register the system service with

the system and start the system service (i.e. streamtool registerdomainhost)

• From the main management host. Use the Domain Manager and Stream Console; or

streamtool to do this configuration.• Create an enterprise domain

• Edit the domain properties and set the highAvailabilityCount property to at least 2.

• Tag resources

• Create instances

• Configure permissions

• Monitor your domain and instances


Install: Select Edition


Install: Installation directory


Install: Post installation


Create host package

Create the domain host packagelaurieaw@d0428b05 ~$ source InfoSphere_Streams/4.0.0.0/bin/streamsprofile.sh

InfoSphere Streams environment variables have been set.

laurieaw@d0428b05 ~$ streamtool mkhostpkg -d mydomain --zkconnect d0701b02:2181,d0701b04:2181,d0701b05:2181

CDISA0070I Creating the following installation package: /homes/hny8/laurieaw/StreamsDomainHost-4.0.0.0-x86_64-el6.tar.

CDISA0069I The following installation package was successfully created: /homes/hny8/laurieaw/StreamsDomainHost-4.0.0.0-x86_64-el6.tar.

Copy the package to each additional host

From each additional host – Extract the package

[streamsadmin@f0401v25 ~] tar xf StreamsDomainHost-4.0.0.0-x86_64-el6.tar

– Run the setup script to install and register the host with the domainroot@f0401v25 StreamsDomainHost]# ./streamsdomainhostsetup.sh -v

CDISI0016I Installing the product in the following directory: /opt/ibm/InfoSphere_Streams...

CDISI0017I Registering the local host with the following domain: StreamsDomain...

CDISA0046I The ibm-streams-StreamsDomain system service was created on f0401v25.pok.hpc-ng.ibm.com in domain

StreamsDomain.

CDISA0049I Starting the ibm-streams-StreamsDomain system service on f0401v25.pok.hpc-ng.ibm.com in domain StreamsDomain.

CDISA0050I The ibm-streams-StreamsDomain system service started successfully on f0401v25.pok.hpc-ng.ibm.com in domain

StreamsDomain.

CDISA0053I The ibm-streams-StreamsDomain system service is started on f0401v25.pok.hpc-ng.ibm.com in domain StreamsDomain.

CDISI0018I The host setup completed successfully.


Domain Manager initial panel


Create LDAP domain


Change properties


Edit domain properties


Open Streams Console


Manage Resources and Tags


Tag resources

Installation and Setup for IBM InfoSphere Streams V4.0

Data & Analytics

Transcript of Installation and Setup for IBM InfoSphere Streams V4.0