Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.
-
Upload
lillian-willis -
Category
Documents
-
view
214 -
download
0
Transcript of Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.
![Page 1: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/1.jpg)
Sep 4, 2008NVOSS 2008 1
VO From the Command-Line
Mike FitzpatrickNOAO
![Page 2: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/2.jpg)
Sep 4, 2008NVOSS 2008 2
Introduction
• The Portals discussed yesterday show typical VO-access modes using the web.
• However, this isn’t easily scriptable (same is true for GUI applications).– Saving results is done manually– Data not easily saved for analysis in other
environments
• What we need are general VO client applications we can script
![Page 3: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/3.jpg)
Sep 4, 2008NVOSS 2008 3
VOClient Overview
• Hides details of VO implementation from applications
• Provides: – Hi-level, easy-to-use interface between
applications and the distributed VO framework,– Client-side VO interfaces returning higher-level
structures (like tables) rather than a service API,– Equivalent functionality in all supported
languages
• Caches entire result until queried by app
![Page 4: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/4.jpg)
Sep 4, 2008NVOSS 2008 4
Architecture
• Modeled after database server• VOClient Daemon
– Multi-threaded– Java access to VO svcs– Layered on DAL, Registry and services
• API– C-based, custom and SWIG bindings– Communicates w/ voclientd using proprietary RPC
protocol
• Connection Oriented– Allows voclientd to maintain state on behalf of client
PythonIRAF,
IDL, etc. Java
VOCLIENTD
dalclientregistry,
etc.
Other…
![Page 5: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/5.jpg)
Sep 4, 2008NVOSS 2008 5
API Example
voc_initVOClient (config_opts)
voc_closeVOClient (shutdown_flag)
query = voc_regQuery (term, orValues) voc_regAddSearchTerm (query, term, orValue) voc_regRemoveSearchTerm (query, term)
count = voc_regGetSTCount (query) str = voc_regGetQueryString (query)
res = voc_regExecute (query) str = voc_regExecuteRaw (query)
![Page 6: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/6.jpg)
Sep 4, 2008NVOSS 2008 6
From API to Apps
VOSESAME
Name resolver using Sesame service (Simbad/NED)Configurable output, multiple inputs
VOREGISTRYVO Registry search and resource resolution
VODATAUses capabilities of both of the aboveVO catalog and image access (spectra in progress)
![Page 7: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/7.jpg)
Sep 4, 2008NVOSS 2008 7
VO-CLI in NVOSS
• Software in NVOSS_HOME$vo-cli directory•To build:
% cd $NVOSS_HOME/vo-cli% make install
This installs binaries in the package bin directory, to use these tools you must install to some directory in your path like the $NVOSS_HOME/bin
% cp bin/* $NVOSS_HOME/bin % cp voclientd $NVOSS_HOME/bin % cp -r voclient.jars $NVOSS_HOME/bin
![Page 8: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/8.jpg)
Sep 4, 2008NVOSS 2008 8
VOSesame: Name Resolution
Usage:
vosesame [<flags>] [<objfile> | [<target> ...] ] -a print all information about the object -d print position in decimal degrees (default) -e print position errors -h print help summary -n print object name -t print object type -s print position as sexagesimal coordinates -v print verbose warning output -i invert search to print unresolved objects -f force Sesame svc invocation and ignore cached values -o specify output file (default: stdout) -A output an ASCII table of results (default) -C output a CSV table of results -T output a TSV table of results -H output a table heading
![Page 9: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/9.jpg)
Sep 4, 2008NVOSS 2008 9
VOSesame: Examples
1) Print the coordinates of NGC4456 decimal degrees
% vosesame ngc4456 186.960000 -30.120000
2) Print the sexagesimal coordinates of multiple objects include the type:
% vosesame -st m31 00:42:44.31 +41:16:09.4 LINER
3) Print the decimal coordinates of those same objects listed in the file 'myobjs.txt', output as CSV, include a header, and print the id, coords, type:
% vosesame -CHndt myobjs.txt #Name,DRA,DDEC,Type, m31,10.684625,41.269278,LINER m51,202.468208,47.194667,Seyfert_2 m99,184.706333,14.416778,HII_G : : : :
![Page 10: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/10.jpg)
Sep 4, 2008NVOSS 2008 10
VOSesame: Examples
• Calculator Mode:% vosesame -nd
m98 m98 183.45154 14.89944
[EOF]
• Print the sexagesimal and decimal values for multiple user coords:
% vosesame -s -c 12:30:0.0 -45:00:0.0 -c 187.5 2.05
12:30:00.0 -45:00:00.0 12:30:00.0 02:03:00.0
![Page 11: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/11.jpg)
Sep 4, 2008NVOSS 2008 11
VORegistry: Registry Query/Resolve
• Command-line query of the VO Registry• Two Major Modes:
– Search (typical keyword search)– Resolve (ShortName/Identifier to something
else)
• List full resource record• List table metadata• Count matching records• Arbitrary ADQL searches• Constrain searches
– Bandpass, Service Type or ContentLevel
![Page 12: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/12.jpg)
Sep 4, 2008NVOSS 2008 12
VORegistry: Examples
1) Find all services with radio data of Abell clusters.
% voregistry -b radio abell
2) Find all resources that mention Keck, how many are image services?
% voregistry -count keck% voregistry -t image -count
keck
![Page 13: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/13.jpg)
Sep 4, 2008NVOSS 2008 13
VOData: Data Access
• Multi-threaded queries (hundreds/min)• Simple names for resources (e.g. ‘2mass-
xsc’ and not “ivo://irsa.ipac/2MASS-XSC”)• Various output formats
(XML/ASCII/CSV/KML…)• Not complete in terms of what VOClient API
provides• Callable from variety of scripting
languages• Interacts with NVO Portal• Targeted for power users
![Page 14: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/14.jpg)
Sep 4, 2008NVOSS 2008 14
VOData: Usage
vodata [<flags>] [ <resource> [[ <objname> [ <sr> ]]] ]
vodata [<flags>] [ <resource> [[ <ra> <dec> [ <sr> ]]] ]
vodata [<flags>] [ <url> ]
![Page 15: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/15.jpg)
Sep 4, 2008NVOSS 2008 15
VO-CLI
• For all tasks, a ‘-h’ flag will print help• The voclientd will be started
automatically• Web interface allows you to experiment
with options. Note the command being executed is printed.
• So, what can we actually do with this??
![Page 16: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/16.jpg)
Sep 4, 2008NVOSS 2008 16
Scavenger Hunt
1) How many image services currently provide data for the GOODS survey?
![Page 17: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/17.jpg)
Sep 4, 2008NVOSS 2008 17
Scavenger Hunt
1) How many image services currently provide data for the GOODS survey?
Solution:
% voregistry -t image goods% voregistry -count -t image goods
![Page 18: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/18.jpg)
Sep 4, 2008NVOSS 2008 18
Scavenger Hunt
2) Where can you get an HST 4350A image of the center of the HDF?
![Page 19: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/19.jpg)
Sep 4, 2008NVOSS 2008 19
Scavenger Hunt
2) Where can you get an HST 4350A image of the center of the HDF?
Solution:% vodata -O hdf -eu HST.GOODS.Cutout \
"Hubble Deep Field" 0.01
• the file 'hdf.urls' will contain the acrefs, last image is the F435
• possible to use the "-get" flag to simply download them all, or elsepass the hdf.urls in to download
![Page 20: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/20.jpg)
Sep 4, 2008NVOSS 2008 20
Scavenger Hunt
3) How many Bautz-Morgan type II Abell clusters are within 20 degrees of the south pole?
![Page 21: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/21.jpg)
Sep 4, 2008NVOSS 2008 21
Scavenger Hunt
3) How many Bautz-Morgan type II Abell clusters are within 20 degrees of the south pole?
Solution:
% vodata -V -O - ivo://nasa.heasarc/abell 0.0 -90.0 20 | \
stilts tpipe ifmt=votable in=- \ cmd='select "bmtype == \"II\""'
omode="count"
![Page 22: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/22.jpg)
Sep 4, 2008NVOSS 2008 22
Scavenger Hunt
4) How many of them are in the ROSAT FSC (within 10')
![Page 23: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/23.jpg)
Sep 4, 2008NVOSS 2008 23
Scavenger Hunt
4) How many of them are in the ROSAT FSC (within 10')Solution:Regenerate the data query, but save results: % vodata -V -O - ivo://nasa.heasarc/abell 0.0 -90.0 20 | \
stilts tpipe ifmt=votable in=- \ cmd='select "bmtype == \"II\""' \ ofmt=votable out=bmII.xml
Submit to Inventory, looking for ROSAT catalogs:% vodata -I -rm 10 any bmII.xml | grep ROSAT
total of only 4 clusters match. Query the RASS FSC for the data:% vodata -O - -A -rm 10 RASS/FSC bmII.xml
We see that one of the clusters (A3628) matches to 5 distinct X-ray sources, but in this tool we didn't crossmatch to the Abell number. (This could be done easily using STILTS, however)
![Page 24: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/24.jpg)
Sep 4, 2008NVOSS 2008 24
Scavenger Hunt
5) How many chromospherically active binaries are EUV sources
… seen by the ROSAT WFC? …seen by EUVE?
![Page 25: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/25.jpg)
Sep 4, 2008NVOSS 2008 25
Scavenger Hunt
5) How many chromospherically active binaries are EUV sources? … seen by the ROSAT WFC? …seen by EUVE?
Solution:Do a registry query to find the catalog and download it
as an all-sky table: % voregistry chromospherically active binary
% vodata -V -O sources ChrAcBin 0.0 0.0 180.0
Send it to Inventory and grep for results:% vodata -I -rs 10 any source.xml | egrep -i "rosat|euv"
Second EUVE catalog shows 30 matches, WFC 2RE shows 9 matches
![Page 26: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/26.jpg)
Sep 4, 2008NVOSS 2008 26
Scavenger Hunt
6) How many stars with known exoplanets have been observed by ST?
Solution:
- Try to find a catalog in the Registry:% voregistry -R exoplanet
- Access the entire table and send it to Inventory: % vodata -O planets -V J/ApJ/646/505 0.0 0.0
180.0 % vodata -I -rm 1 any planets_table2.xml | grep
HST
![Page 27: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/27.jpg)
Sep 4, 2008NVOSS 2008 27
Scavenger Hunt
7) Write a script that will show you the rough number of HST, Chandra and Spitzer observations at a given location.
![Page 28: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/28.jpg)
Sep 4, 2008NVOSS 2008 28
Scavenger Hunt
7) Write a script that will show you the rough number of HST, Chandra and Spitzer observations at a given location.
Solution 1:
vodata -n hst $1 $2 .02 2>/dev/null | grep hst vodata -n spitzer $1 $2 .2 2>/dev/null | grep spitzer vodata -n chandra $1 $2 .2 2>/dev/null | grep chandra
When given 3c273 as the object this gives: hst 238 C Hubble Space Telescope spitzer 23 C Spitzer Space Telescope Observation Log chandra 35 I Chandra X-Ray Observatory Data Archive
![Page 29: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/29.jpg)
Sep 4, 2008NVOSS 2008 29
Scavenger Hunt
7) Write a script that will show you the rough number of HST, Chandra and Spitzer observations at a given location.
Solution 2:
#!/bin/csh -fvodata -n hst,chandra,spitzer $1 $2 $3 |& grep -v -e
"#"
![Page 30: Sep 4, 2008NVOSS 20081 VO From the Command-Line Mike Fitzpatrick NOAO.](https://reader035.fdocuments.in/reader035/viewer/2022062618/5514d7d3550346935c8b51c0/html5/thumbnails/30.jpg)
Sep 4, 2008NVOSS 2008 30