Channel ArchiverOverview

Jan. 2003

Channel Archiver Channel Access client Stores samples in disk files Design target: handle 10000 values/sec

Documentation, Snapshots:LANL EPICS Page.From http://www.aps.anl.gov/epics,see Other Sites, LANL, Tools, Archiver.

Operation Config. file lists channels to archive

# Example: <channel> <period [s]>channel_X 1channel_Y 10# Monitor: might change every 0.2 secondchannel_Z 0.2 Monitor

Scanned Periodically store most recent value

Monitor Store all incoming values – up to buffer limit

Original time stamps are stored!

1s Period Data @ 9.96, 10.98, 11.76 s ?

Components

CA

DataDirectory

Storage Retrieval

LibIO (C++)

ArchiveEngine

LibIO (C++)

WinBrowser(Win32)

LibIO (C++)

xarr(Unix)

LibIO (C++)

StripTool(Unix)

LibIO (C++)CGIExport

WebServer

Native:

LibIO (C++)SWIG

TCL tk,BLT

LibIO (C++)SWIG

Pythontk,BLT,PMW

LibIO (C++)SWIG

PerlCGI,DBI,DBDScripting:

Online: (also possible via scripting)

LibIO (C++)

Archive-Export

LibIO (C++)

Archive-Manager

ArchiveEngine Syntax:

ArchiveEngine [options] config [dir]

Options:-d <desciption>-p <port> : change HTTPD from 4812-l <log> : write log file

Engine’s HTTPD URL of engine’s HTTPD: http://<machine>:<port> Status & Config., not data! “Client Pull”: Updates on reload Changes (added groups/channels) written to cfg

subdirectory, original config. files unchanged

“Directory File” File name, not a directory name Specified when running archive engine

Default: “freq_directory”. Better: “dir”. Used by retrieval tools Details of “directory” and “data” files

are given in documentation Note: Keep directory & data files

together, don’t modify them.

ArchiveExport Syntax:

ArchiveExport [options] [dir] {channel names}

Options:-start “mm/dd/yyyy hh:mm:ss”-end “mm/dd/yyyy hh:mm:ss”-fill-interpolate <seconds>-gnuplot-Matlab…

Export Options Original Time Stamps:

ill-suited for Spreadsheets “Fill” missing values

by repetition Linear Interpolation

for given period

(initial #N/A until all channels have valid value)

Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 #N/A -0.0860063/22/00 17:02:37.401 0.054358 #N/A3/22/00 17:02:37.511 #N/A -0.111776

...

Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 0.071824 -0.086013/22/00 17:02:37.401 0.054358 -0.086013/22/00 17:02:37.511 0.054358 -0.111783/22/00 17:02:39.411 0.139948 -0.11178...

Time A B3/22/00 17:02:28.700 0.071824 #N/A3/22/00 17:02:28.701 #N/A -0.086013/22/00 17:02:30.000 0.069216 -0.089813/22/00 17:02:32.000 0.065201 -0.095663/22/00 17:02:34.000 0.061186 -0.101513/22/00 17:02:36.000 0.057171 -0.10736…

“Fill”, “Interpol.”, ...

6.5

7

7.5

8

8.5

9

0 0.0002 0.0004 0.0006 0.0008 0.001 0.0012 0.0014

Time [s]

Val

ue [a

.u.]

Original Samples"filled"Linear Interpol.

MATLAB Export

Toolsgenerate MATLAB command file ASCII, portable No MATLAB

binaries required Full value info,

time & status Big & slow

WinBrowser Win. Only

CGI Export

Data Access from any Web browser

Download formats: Spreadsheet, Matlab

Data

Web ServerCGIExport

WWW

CGI Export Online Plots

Scripting Allows writing

perl/tcl/python programs for Automated generation

of daily beam statistics for Web page

Specialized scripts to answer questions like:“How often was XX below 10.0 and for how long?”

…

ArchiveEngine Details More options, see manual:

!write_period <seconds> “Groups” for

getting organized conditional archiving!group <other file><channel> <period> Disable

Retrieval f. Multiple Archives Run several ArchiveEngines, creating

several “small enough” data sets Look at them all at once: master_version=1

# List sub-archives, most recent first/archives/2001/july/dir/archives/2001/june/dir/archives/2001/may/dir# …# Then check Fred's "xyz" archive /home/fred/xyzarchive/dir

Sub-archives should not “overlap”

Suggested Setup Long Term Archive

Sample ~10min, kept forever Middle Term

Sample every 60s, restarted after 15 days, switching between two directories

Short TermUp to ‘Monitor’, toggling two directories every shift

High-grained data for last shift, good data for last 2 weeks, some log of everything kept forever

CAManager (Thomas Birke, BESSY) ArchiveEngine control GUI

Start/Stop

Status of (multiple) ArchiveEngines

Periodic re-start in different directory,

updating a MultiArchive File

Data Management Do not separate Directory & Data

files! Periodically restart Engine in new

directory to keep data sets manageable (CAManager helps)

ArchiveManager program can copy selected channels & time range, performing basic repairs and compression while doing this

Summary “ChannelArchiver” is a Toolset for archiving

any ChannelAccess data Some generic retrieval options, scripting

and Matlab allow further analysis Future Plans:

Port to R3.14 and RH8 CORBA interface for data retrieval

(common w/ JLab archiver, replace tcl/perl/python binding, allow Java access to data)