Teuchos: Utilities for

Developers & Users

November 4th, 1:15-4:30pm

Roscoe BartlettMike Heroux

Kris KampshoffKevin LongPaul Sexton

Heidi Thornquist

Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy under contract DE-AC04-94AL85000.

What is Teuchos (TEF-hos)? Lightweight, portable utility package What’s new?

First release in Trilinos 4.0 Templated BLAS/LAPACK interface Templated serial dense matrix / vector Timing classes Templated parameter list Numeric traits mechanisms ( ordinal / scalar ) Memory management:

• Reference-counted pointers

• Pre-allocated workspace

Exception handling Command-line parsing “Extended” utilities:

• XML parsing

• MPI communicators / traits

Users

Developers

Templated Parameter List Design based on NOX parameter list Adopted by Amesos, ML, Ifpack, AztecOO, …

Parameters can be numeric datatypes, pointers to vectors or functions, and other parameter lists.

// Empty parameter listTeuchos::ParameterList My_List;

// Setting parametersMy_List.set("Max Iters", 1550);My_List.set("Tolerance", 1e-10);My_List.set("Solver", "GMRES");

// Create sublistTeuchos::ParameterList& Prec_List =

My_List.sublist("Preconditioner");

// Setting parameters in sublistPrec_List.set("Type", "ILU");Prec_List.set("Drop Tolerance", 1e-3);

Parameters can be input and retrieved with templated “set” and “get” methods, as well as helper functions. Parameter lists can be queried for attributes.

// Has a solver been chosen?bool solver_defined = My_List.isParameter(“Solver”);

// Has a preconditioner been chosen?bool prec_defined = My_List.isSublist(“Preconditioner”);

// Has a double-precision drop tolerance been chosen? (Helper function)bool drop_tol = Teuchos::isParameterType<double>(Prec_List,

“Drop Tolerance”);

ReturnType foo_solver( Teuchos::ParameterList& solverPL,… ){ // Getting parameters int max_iters = solverPL.template<int> get("Max Iters"); std::string solver = getParameter(solverPL, “Solver”) double tol = solverPL.get("Tolerance", 1e-8);

// Get sublist ( same as making sublist ) Teuchos::ParameterList precPL =

solverPL.sublist("Preconditioner");… return Ok;}

Parameters can be input and retrieved with templated “set” and “get” methods, as well as helper functions.

Numeric Traits Mechanisms

Assumed: addition, subtraction, multiplication, and division OrdinalTraits

zero & one int & long int

ScalarTraits zero, one, magnitude type, absolute value, conjugate,

square root, random number generator, … std::numeric_limits float, double, complex<float>, and complex<double>

Required: none, a compile-time check is performed. Arbitrary precision arithmetic

Generic programming technique that uses templated interfaces to define the standard behavior of

datatypes.

Arbitrary Precision LibrariesARPREC & GNU MP

ARPREC (Bailey, et. al.) uses arrays of 64-bit floating-point numbers to represent high-

precision floating-point numbers. provides support for datatypes with differing precision levels. --enable-teuchos-arprec

GNU MP uses fullwords, arbitrary precision mantissa, and limited precision

exponent to represent high-precision floating-point numbers. platform specific kernels written in assembly language perform

common inner-loop functions. provides support for datatypes with differing precision levels. --enable-teuchos-gmp

What about communication of arbitrary precision variables?

Primitive Type Traits

A traits class for decomposing an object into an array of primitive objects ARPREC: primitive type = double GNU MP: primitive type = int, byte(?)

Can be used in communicating more complex datatypes Essential for Tpetra, TSFCore, …

Platform dependent issues here ...

Memory ManagementTeuchos::RefCountPtr

Reference-counted pointers combat seg faults, memory leaks, headaches ...

RefCountPtr Intro Paper:$(TRILINOS_HOME)/doc/RefCountPtr/RefCountPtrBeginnersGuideSAND.tex

http://software.sandia.gov/Trilinos/RefCountPtrBeginnersGuideSAND.pdf

RefCountPtr<A> a_null_ptr, // Create a reference-counted NULL pointer. a_ptr2 = rcp(new A), // Initialize reference-counted pointers. a_ptr3 = rcp(new A); // "" A *ra_ptr2 = new A, // Initialize non-reference counted pointers. *ra_ptr3 = new A; // "" a_ptr2 = rcp(ra_ptr3); // Assign from a raw pointer (only do this once!) a_ptr3 = a_ptr1; // Assign one smart pointer to another. a_ptr2->f(); // Access a member of A using -> ra_ptr2->f(); // "" *a_ptr2 = *a_ptr3; // Dereference the objects and assign. *ra_ptr2 = *ra_ptr3; // ""

// Get the raw C++ pointer. A* true_ptr = a_ptr1.get();

Memory ManagementTeuchos::Workspace

Pre-allocated workspace (Teuchos::WorkspaceStore) Avoid calls to new/delete with arrays of un-initialized or default

initialized objects as automatic variables on the stack (global). Calls new/delete by default. Very useful in codes/functions/methods that are commonly used

and riddled with creation/destruction of arrays:int* index = new int[ num_vecs ];

#include “Teuchos_Workspace.hpp”

int main( int argc, char* argc[] ) { size = 100; Teuchos::set_default_workspace_store( Teuchos::rcp(new Teuchos::WorkspaceStoreInitializeable(10*size)));

Teuchos::WorkspaceStore *wss_(Teuchos::get_default_workspace_store().get()); Teuchos::Workspace<double> b(wss_,size,false); Teuchos::Workspace<int> index(wss_,size,false);

return 0;}

Memory ManagementTeuchos::Workspace

Memory block size = 100Number of call loops = 500000

Timing raw new and delete for temporaries ... time = 0.589 sec

Timing std::vector for temporaries ... time = 0.657 sec

Timing std::valarray for temporaries ... time = 0.627 sec

Timing Teuchos::Workspace for temporaries ... time = 0.268 sec

*** Statistics for automatic array workspace: Number of megabytes of preallocated workspace = 0.001 Number of megabytes needed = 0.0008 Number of allocations using preallocated workspace = 500000 Number of dynamic allocations beyond preallocated workspace = 0

Relative time (lower is better): raw new/delete = 2.19776 std::vector = 2.45149 std::valarray = 2.33955 Teuchos::Workspace = 1

Exception Handling

TEST_FOR_EXCEPTION( n > 100, std::out_of_range , "Error, n = " << n << is bad" );

/home/bob/project/src/my_source_file.cpp:225: n > 100: Error, n = 125 is bad

TEST_FOR_EXCEPTION macro: Macro for throwing an exception with breakpointing:

#define TEST_FOR_EXCEPTION( throw_exception_test, Exception, msg )

#define TEST_FOR_EXCEPT( throw_exception_test )

DEVELOPER’S NOTE: If there is an error with this macro ...

• Did you use “-DHAVE_CONFIG_H” in your makefile?• Consider using “trilinos_make_macros.mak”

Teuchos::Command Line Processor

Basic command line parser for input from (argc,argv[]) Facilitates organization of configurable executables/examples Creates help string for command line options Command line options can be standard data types / enumerated

lists Let’s look at an example ...

Teuchos::CommandLineProcessorExample

#include “Teuchos_CommandLineProcessor.hpp”int main(int argc, char* argv[] ){ // Creating an empty command line processor looks like: Teuchos::CommandLineProcessor My_CLP;

// Set an integer command line option. int NumIters = 1550; My_CLP.setOption("iterations", &NumIters, "Number of iterations"); // Set a boolean command line option. bool Precondition; My_CLP.setOption("precondition","no-precondition", &Precondition,"Preconditioning flag"); ... // Allow an unrecognized option to be ignored (warning is printed) My_CLP.recogniseAllOptions(true); // Throw and exception if an option found is not recognized or My_CLP.throwExceptions(false); Teuchos::CommandLineProcessor::EParseCommandLineReturn

parseReturn= My_CLP.parse( argc, argv ); if (parseReturn != PARSE_SUCCESSFUL)

return 1; return 0;}

$ ./CLP_example.exe --help

Usage: ./CLP_example [options] options: --help Prints this help message --pause-for-debugging Pauses for user input to allow attaching a debugger --iterations int Number of iterations (default: --iterations=1550) --tolerance double Tolerance (default: --tolerance=1e-10) --solver string Linear solver (default: --solver="GMRES") --precondition bool Preconditioning flag --no-precondition (default: --no-precondition) --speed enum Speed of our solver Valid options: "slow", "medium", "fast" (default: --speed="medium")

Teuchos Developers Overview

Parameter list is easy to use and quickly being adopted by most Trilinos packages.

Templated tools libraries are essential for the development of templated solver libraries. Numeric traits mechanism allows the use of arbitrary datatypes in

templated code. Templated BLAS/LAPACK wrappers and SDM allow easy

integration of arbitrary precision for high-precision kernels.

Teuchos tools provide portable solutions for developers. See Teuchos website for more info:http://software.sandia.gov/Trilinos/packages/teuchos