SciDAC SSS Quarterly Report
description
Transcript of SciDAC SSS Quarterly Report
SciDAC SSS Quarterly Report
Sandia Labs
August 27, 2004
William McLendon
Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company,for the United States Department of Energy’s National Nuclear Security Administration
under contract DE-AC04-94AL85000.
APItest - Overview
• Current Release : v0.2.0
• Test Driver tool for testing software– Ordered tests
– API Testing
• Available for download:– ftp://ftp.sandia.gov/outgoing/apitest/
APItest - Features
• Scripted Test Driver– XML based tests
• Object Oriented– Easy to define new test
types.• Conditional test ordering
– Batch Scripts• Can call other batches
• Built In Test Types– Shell– Script– SSS
• Can set environment variables for tests.
• SUID Test Capability– CAUTION!
• Timeouts• User Interface Types
– Web Browser– Command Line
• Test Results Export– Save Test results to an
XML file.• Open Source
– LGPL• Fully scripted in Python• Portable
APItest - Requirements
• Packages required to run APItest:
– Python 2.3 or greater
– ElementTree (www.effbot.org)
– Twisted (www.twistedmatrix.com)
– ssslib (ftp://ftp.mcs.anl.gov/pub/sss/)• optional, for SSSlib tests.
APItest - Execution
• There are two ways to execute APItest
• Command Line Only– $> apitest -f test_file.[apt|apb]
• Web browser GUI (Interactive)– $> apitest httpd
• -i [--iroot] : root directory to look for tests• -p [--port] : port number (default = 2112)• -h [--host] : host (default = localhost)
• --help will display command line usage.
APItest - GUI (Top Page)
• Top level index page for APItest
• APItest recurses from the [testroot] to find tests.
• File Extensions:– TEST : .apt– BATCH : .apb
• Batch tests shown in bold
• Organized by directory
APItest - GUI (Test Run Execution)
• Quick status view of test execution.
• This screen loads when the EXECUTE button is selected.
• Color coded for easy viewing.
• Currently running test is indicated.
• Refreshes every 15 secs.
APItest - GUI (Test Run Summary)
• Summary of test results
• Color coding
• Batch Features
– Cumulative summaries• Total # of tests• # that passed (green)• # that failed (red)
• Can click the hyperlinks to see more details about a test or batch.
APItest - GUI (Batch Summary)
• Clicking a hyperlink into a batch test gives us a summary view.
• In this example, one test failed a dependency
– FAILDEP in the status column indicates this.
• Clicking on the scriptTest2.apt link will show us more data about this test.
APItest - GUI (Test Details - FAILDEP)
• Detailed view of scriptTest2.apt from previous slide.
• Failed a dependency on cmdTest2.apt– Expected : FAIL– Actual : PASS
APItest - GUI (List of Test Runs)
• Clicking the View All Runs button gives this screen.
• Test Runs are sorted by date and time.
• Click links to view details of a particular run.
APItest - GUI (Test Details - PASSED)
• Example of a test that was executed and passed.
• Summary Data
• Dependency Data
– Only shown if test was run as part of a batch.
• Output
– Side-by-side comparison
– Dependent on type of test.
SSS - Service Directory Startup Test
PREP TEST SDOFF
SSS START TEST SDON
RM EMNG SDSTAT
SSS STOP TEST SDOFF
APItest - Ongoing Work
• Develop more tests for SSS components!• Validation via file matching.• Configuration file.• Test Developer GUI.• Results browser.• Session management and security (Cookies?)• Additional native test types.
– HTTP, TCP/IP, XMLRPC• DB Connectivity.• User Guide.• Bug fixing.
APItest - Test Types
• CMD– Execute a command via the command line.
• SCRIPT– Execute a script.
• SSS– Execute a ssslib communication.
APItest - Example CMD Test
<testDef>
<info> Example test. Runs ls -lr from /tmp, expects 0 exit status and any input.</info>
<test type="cmd” wdir="/tmp">
<command interpreter="/bin/ksh">ls</command>
<input name="argument" format="literal">-l</input> <input name="argument" format="literal">-r</input> <input name="stdin" format="literal" />
<output name="stdout" format="regexp">.*</output> <output name="stderr" format="literal" /> <output name="status" format="literal">0</output>
</test></testDef>
APItest - Example Script Test
<testDef>
<info>Timeout test</info>
<test name="timestamp1" type="script" timeout="4">
<command interpreter="/bin/bash" wdir="/tmp">for x in 1 2 3 4 5 6 7 8 9 10 do echo "timeout_test :${x}: `date`” sleep 1done </command>
<output name="stdout" format="regexp">.*timeout.*</output> <output name="stderr" format="literal" /> <output name="status" format="literal">0</output>
</test></testDef>
APItest - Example ENVVAR Test
<testDef> <info> This script executes but adds some stuff to the environment. NOTE: setting envvar OVERRIDES any inherited environment variables. </info> <test type="script">
<command interpreter="/bin/csh">#!/bin/csh env</command>
<input name="envvar" format="literal" key="test_envvar_1">foobar</input> <input name="envvar" format="literal" key="test_envvar_2">foo</input>
<output name="stdout" format="regexp"> .*test_envvar_1=foobar\n.*test_envvar_2=foo\n.*</output> <output name="stderr" format="literal" /> <output name="status" format="literal">0</output>
</test></testDef>
APItest - Example SUID Test
<testDef>
<test type="script">
<command interpreter="python" uname="wcmclen" gid="500">#!/usr/bin/env python import os print os.getuid() print os.getgid() </command>
<input name="argument">/tmp/testuser/</input> <input name="argument">-l</input>
<output name="stdout" format="regexp">^27030$\n^500$</output> <output name="stderr" format="literal" /> <output name="status" format="literal">0</output> </test></testDef>
APItest - Example SSS Test
<testDef>
<info>Sample sss query of the service directory</info>
<test type="sss" destination="service-directory">
<input name="sendbuf"><get-location><location component='service-directory' host='*' port='*' protocol='*' schema_version='*' tier='*'/></get-location></input>
<output name="recvbuf" format="regexp">.*</output>
</test>
</testDef>