informix Embeddability and Autonomics

© 2012 IBM Corporation

Informix - Autonomics & Embeddability

John F. Miller III ([email protected])Embedability Architect

© 2012 IBM Corporation2

Agenda

■ Informix – An Enterprise-class Embedded Database

■ Types of Embedded Database Solutions

■ Features in Informix – Autonomics & Embeddability

■ Resources

■ Summary

Informix – Autonomics & Embeddability


White Paper - “Smart Systems Require an Embeddable Database: Six Criteria to Consider”

■ Installation

■ Integration

■ Performance

■ Resource Management

■ Administration

■ Availability



Informix Version 11 – An Enterprise-class Embeddable Database

■ Administration (Set-it and Forget-it)– Self-managing Autonomics

■ Resource Management– Storage Provisioning, Data Compression, Memory

■ Installation and Deployment– Low Footprint, Rapid deployment, Component-based Install

■ Availability (Flexible and Reliable)– Enterprise Replication (ER), Flexible GRID– High-Availability Data Replication (HDR), Continuous Availability (RSS, SDS)

■ Performance– Blazing fast OLTP and DSS performance

■ Security– Encryption, Standard and Secure Communication Protocols

■ Integration– Database Extensions– Linux, Windows, UNIX and Mac OS– Virtualization



Types of Embedded Database Solutions

■ Deep Embedding– There is no DBA!– The end-user is completely unaware of the

existence of the embedded database– Think Informix inside...

• Cisco's hardware• *Point of Sale• Cognos Express

■ Integrated Embedding– The database is integrated within the application– Occasionally managed by a local/remote DBA– Think Informix inside...

• Custom retail in-store applications• Supply-chain/CRM applications



Dynamically Scalable Architecture

■ Proprietary thread-based architecture– Informix is in control of which thread to execute when– Dynamic configuration and resource allocation– One instance configuration file

■ Process– Load balancing across virtual processors (OS processes)– Extremely fast thread switches

■ Memory– Memory allocation adjusts to workload– Intelligent and improved priority manager for the buffer pool– Shared Memory contention is controlled using spin-locks



Dynamically Scalable Architecture

■ Parallel I/O– Queries - scans, sorts, joins, aggregations etc.– Data loads, Index builds– Backup and Restore– Recovery– Light scans are used to bypass the buffer pool for improved

mixed-workload performance

■ Storage– Automatic addition of data chunks and data spaces– Increased file I/O performance by bypassing OS file system

buffer cache



Dynamic Process Management

■ Automatically add CPU vps during server startup

■ Automatically add AIO vps when the database server detects sub-optimal performance

■ The application can also instruct the database server to add or drop...– CPU vps during peak and off-peak workloads– ENCRYPT vps that support encrypt and decrypt operations for column-level encryption– User-defined vps that support execution of C and Java UDRs– Network vps such as TLI, SOC, STR and SHM to support more user connections– MSC vps to help with network authentication



Dynamic Memory Management

■ Automatically allocate virtual shared memory segments which are needed to handle more clients or execute a large DSS query

■ Low memory manager can run customer application rules to provided priority to critical applications during low resources times

■ The application can also instruct the database server to free shared memory segments

■ The database server uses the least-recently used (LRU) queues to replace the cached data in the buffer pool

■ Allocate memory to Non-PDQ sessions

■ Increased concurrency with configurable private memory cache for each CPU vp

■ Automatically terminate idle user sessions



Dynamic Session Management

■ Limit the maximum number of sessions that can connect to Informix

■ Enhanced connection management for HA clusters– Reroute clients based on best fit– Redirects clients away from failed nodes

■ Enhanced client session information in system tables (sysmaster:syssesappinfo)

■ Control external directives for a session– Allows DBA to modify query plan without changing the application or SQL

■ Session configuration routines– sysdbopen()– sysdbclose()



Dynamic Log Management and Checkpoints

■ Physical Log– Resize/Move the physical log without bringing the database offline– Recovery Time Objective policy– Non-blocking checkpoints– Automatic LRU tuning

■ Logical Log– Dynamically allocate a log file after the current log file when the next log file contains an

open transaction– Prevent logical log files from filling and hanging the system during long transaction

rollbacks

■ Dynamic Lock Allocation– Allow an application to continue processing even if the initial number of configured locks

have been consumed



Performance

■ Influence the query optimizer's decision for improving query performance during sessions that use both OLTP and DSS queries

– SET ENVIRONMENT OPTCOMPIND

■ Monitor query performance with SQL Query Drill-Down

■ Improve query performance by adding an index on a table without acquiring an exclusive lock on the table for the complete duration

■ Forest-of-Trees Indices, Multi-Index Scans

■ Automatic calculation of statistics for a new index

■ Auto re-prepare of SQL statements and SPL routines



Performance

■ Automatic Update Statistics using Database Scheduler

■ Improved UPDATE STATISTICS performance with table fragment level statistics vs. only table statistics

■ Automatically recollect fragment level statistics after an online attach/detach fragment operation

■ Rescan and recollect statistics only if there's sufficient DML activity in the fragment

■ Statistics on temporary tables



Automatic Storage Provisioning

■ To proactively or reactively add storage to eliminate out of space errors– Monitor spaces and automatically grow a data space when its free space falls

below a threshold– Stall an SQL which is about to fail because of insufficient space until new

space is allocated to the depleted data space

■ The ability to let the database server know about disk space that can be used to solve storage issues in the future (Storage pool)

– Raw Devices– Cooked Files– Directories

■ Benefits– "Out-of-space" errors are virtually eliminated– Automatic expansion of dbspaces, temporary dbspaces, sbspaces,

temporary sbspaces, and blobspaces from the defined storage pool



Deep Compression and Storage Management

■ Online Table Data Compression– Reduces the amount of storage taken by a single row– All operations can be executed while modification take place– Compress, Repack and Shrink

■ Automatic Compression determines which tables to compress based on thresholds

■ Defragmenting table extents when the database is online

■ Index Compaction– Adaptive Linear Index Cleaning– Ensure that the index pages are kept full

■ Prevent accidental disk initialization



Event-driven Administration

■ The database server can execute a program that operates either whenever certain noteworthy event alarms occur or every time any event alarm occurs

■ Some noteworthy event alarms– Failure of a database, table, index, chunk or dbspace taken

offline– Internal subsystem failure– Start-up failure and detection of long transaction, etc.

■ You can receive notification of an event alarm through email

■ Comprehensive event alarm documentation on IBM InfoCenter



System Database for Administration - sysadmin

■ A new system database created and managed by the database server

■ Provides the infrastructure for...– SQL Admin APIs– Query Drill-Down (SQL History Tracing)– Database Scheduler

■ Informix DBA's Swiss Army Knife– sysmaster + sysadmin



Remote Administration with SQL Admin APIs

■ Previous versions of Informix provided only command-line utilities for database administration

– Lack of programmatic interface for database administration made it increasingly difficult to automate repetitive and routine administrative tasks

■ SQL Admin APIs were introduced for the first time in Informix Version 11

– Provides a significant advantage as compared to legacy command-line administrative utilities

– With Open Admin Tool it provides a centralized GUI interface to administer multiple Informix instances using a web browser




■ Two sets of User-Defined Routines (UDRs) to administer Informix– sysadmin:task()– sysadmin:admin()

■ Provides identical functionality as legacy command line utilities

■ Wide scope of SQL Admin APIs currently available in Informix Version 11.7– Space Management (Adding/Dropping a dbspace, chunk, etc.)– Configuration Management (Altering physical log, etc.)– System Validation (Checking data consistency, etc.)– Routine Maintenance Actions (Adding/Dropping logical logs, etc.)– Enterprise Replication– Flexible GRID, Continuous Availability

■ Example:– execute function sysadmin:task ('defragment', 'sales_demo:sales');




■ IBM Informix InfoCenter for SQL Administrative API functions



Database Scheduler

■ Schedule common administrative tasks for automatic execution– Once during server startup– Pre-defined schedule

■ Database Scheduler Tasks & Sensors– Tasks are used for performing administrative actions– Sensors are primarily used for monitoring resources and use that data for analyzing

database server efficiency

■ Tasks & Sensors are defined in sysadmin:ph_task table

■ Additional task can be added and scheduled


© 2012 IBM Corporation

Some Database Scheduler Tasks Geared Toward Embeddability

22

Task Name Task Description

Idle_user_timeout Terminates users who have been idle to long

Mon_low_storage Monitors and proactively add space when necessary

Add_storage Reactively add space

Auto_tune_cpu_vps Tunes the informix vp based on the hardware configured

Auto_crsd Automatically optimize table storage (compress, repack, shrink and defragment)

Refresh_table_stats Adjust statistics after Attaching or Detaching a fragment

Online_log_rotate Rotate and Purge the online.log

Auto Update Stats Automatically update statistics when they are stale

Check_backup Ensure a backup has been taken

Check_for_ipa Warn users about table with In-Place Alters

Bad_index_alert Post an alert if a bad index is found


Database SchedulerInformix – Autonomics & Embeddability


Deploying pre-configured pre-populated Informix instances

■ Typical Steps– Install Informix on the template machine– Setup and configure the template instance– Create a snapshot– Rapidly deploy the snapshot

■ Informix Embeddability Toolkit– Collection of tools, scripts and information for easy automation of reduced footprint

Informix deployments– Tools included

• Informix Deployment Assistant (ifxdeployassist)• Informix Deployment Utility (ifxdeploy)• Configuration file (ifxdeploy.conf)



Deployment Assistant

■ Simplified method of packaging snapshots of Informix instances

■ Ability to reduce the footprint of the snapshots

■ Displays file associations and grouping

■ Built-in support for compression

■ Command line (ifxdeployassist)



Deployment Utility

■ Extremely lightweight command line tool (ifxdeploy) to decompress and deploy the pre-configured pre-populated snapshot on multiple hosts

■ Can be used for adding nodes to Flexible GRID

■ Configuration file (ifxdeploy.conf)



Just a Few of the Many Resources AvailableInformix – Autonomics & Embeddability

Embedding IBM Informixhttp://www.redbooks.ibm.com/abstracts/sg247666.html

Smart Systems Require an Embeddable Database: Six Criteria to Consider

http://public.dhe.ibm.com/common/ssi/ecm/en/csw03007usen/CSW03007USEN.PDF

What's new in IBM Informix 11.7ftp://public.dhe.ibm.com/common/ssi/ecm/en/imd14343usen/IMD14343USEN.PDF

Storage Provisioninghttp://www.ibm.com/developerworks/data/library/techarticle/dm-1103storageprovisioning/index.html

http://www.redbooks.ibm.com/abstracts/sg247666.html

http://www.redbooks.ibm.com/abstracts/sg247666.html



ftp://public.dhe.ibm.com/common/ssi/ecm/en/imd14343usen/IMD14343USEN.PDF

ftp://public.dhe.ibm.com/common/ssi/ecm/en/imd14343usen/IMD14343USEN.PDF

http://www.ibm.com/developerworks/data/library/techarticle/dm-1103storageprovisioning/index.html




Summary

■ Informix – An Enterprise-class Embedded Database

■ Types of Embedded Database Solutions– Deep Embedding– Integrated Embedding

■ Autonomics & Embeddability in Informix– Dynamic Scalable Architecture– Self-* features for process, memory, session, storage etc.– SQL Administration APIs– Database Scheduler– Deploying pre-configured pre-populated Informix instances

■ Resources


informix Embeddability and Autonomics

Software

Transcript of informix Embeddability and Autonomics