Vanilla TCP?Vanilla TCP?Alasdair AllanAlasdair Allan

IVOA Interop Meeting, May 2006

2

Why TCP?

• Traditional and still the best

• Because we’ve always done it that way– not always a bad thing– everyone knows how

• Minimum buy-in for users

• Minimum buy-in for publishers

• Fast, little if any overhead

IVOA Interop Meeting, May 2006

3

User buy-in…

• 92 lines of Perl, including;– ACK messages– IAMALIVE messages– error handling– re-connection– message parsing

IVOA Interop Meeting, May 2006

4

Publisher buy-in…

• 130 lines of Perl, including;– ACK messages– IAMALIVE messages– error handling– re-connection– multiple clients

IVOA Interop Meeting, May 2006

5

What we’ve done so far?

• Convenience layer and parser (Perl, C++ & Java?)– building common cases (including GCN)– http://voevent.sourceforge.net/

• Prototype event network live and on sky• Limited (hard-wired logic) brokering

– using <What>• Archiving and persistent storage• RSS test feed(s)

– programmatic– human readable?

IVOA Interop Meeting, May 2006

6

TCP(V) prototypes

• VOEvent over “vanilla” TCP/IP

• Acknowledgement of passed message

• Basic health checks on the endpoints

IVOA Interop Meeting, May 2006

7

An architecture diagram showing; publishers, subscribers, filters (e.g. relays) and repositories, along with the more complicated aggregators and brokers.

“Atomic” services

• Publisher• Relay• Repository• Subscriber• Listener

• Author• Author

• Listener

IVOA Interop Meeting, May 2006

8

TCP(V) client/server

• Subscriber (client) opens a socket connection to a a Publisher (server) and listens for events on that port.

• Subscriber (client) will send an acknowledgement to the Publisher (server) on receipt of an event.

• Publisher (server) will periodically send an “iamalive” packet to all Subscribers (clients).

• Subscribers (clients) will reply with an “iamalive”.

IVOA Interop Meeting, May 2006

9

Event flow

Publishers Publishers BrokerBroker Subscribers

Subscribers

RepositoryRepository

# # # #<VOEvent role=“observation”> . . .</VOEvent>

# # # #<VOEvent role=“ack”> . . .</VOEvent>

IVOA Interop Meeting, May 2006

10

VOEvent Network

Caltech

SDSS SNe U Wash/Stanford

PairitelBerkeley

Palomar P60Caltech

Palomar-Quest Caltech

RAPTOR x 8LANL

Liverpool TelescopeLa Palma

Faulkes NorthHawaii

Faulkes SouthAustralia

OGLE IIILas Campanas

JACHawaii

Exeter

VOEvent

Other

Event Flow

Microlensing Survey Exeter

UKIRTHawaiiSkyDOT

(database)

PublisherFilterRepository

LANL

SWIFT etc

GCNNASA GSFC

GCN-2NASA GSFC

Backbone

Data MiningExeter

NOAO

SurveysCTIO/KPNO

Tools/ServicesCommunity

Key RolesAuthorSubscriber

ULTRACAMLa Palma

SuperNova SurveyExeter

IVOA Interop Meeting, May 2006

11

Alternative transport

• RSS 2.0

• eSTAR– SOAP– Plastic (over XMLRPC)

• Caltech– Jabber/XMPP

• NOAO– Java Messaging– Mule

IVOA Interop Meeting, May 2006

12

RSS 2.0

• Parallel (on the side?) development to TCP(V)

• The vanilla of the Web 2.0 era

• What RSS is not,– a transport layer– a protocol for document exchange

• It’s “just” a document format– envelope for VOEvent

IVOA Interop Meeting, May 2006

13

RSS feed

<?xml version="1.0" ?>

<rss version="2.0">

<channel>

<title>VOEvent GCN Notices</title>

<language>en</language>

<description>RSS feed for GCN notices</description>

<link>http://voeventnet.caltech.edu/GCN.html</link>

<item>

<title>MILAGRO_Source trigger 886-1</title>

<description>

<p>Possible GRB The event time is 2005-12-01T16:24:15 UT. Location RA 232.1248 Dec 62.9797 (J2000)</p>

</description>

<pubDate>Fri, 02 Dec 2005 16:09:52 PST</pubDate>

<link>http://voeventnet.caltech.edu/Notices/ivo_GCN_MILAGRO_58_886-1_2005-12-01T16:24:15.xml</link>

<author>Scott Barthelmy GCN scott@milkyway.gsfc.nasa.gov</author>

<comments>http://gcn.gsfc.nasa.gov/gcn3_archive.html</comments>

<enclosure url=“http://estar.org.uk/event.pl?id=886-1” length=“2440” type=“application/xml+voevent” />

</item>

.

.

.

</channel>

</rss>

IVOA Interop Meeting, May 2006

14

RSS feeds

eSTAR native feed (includes OGLE III feed)

http://www.estar.org.uk/voevent/eSTAR/eSTAR.rdf

RAPTOR/TALONS feed (includes GCN translation)

http://www.estar.org.uk/voevent/RAPTOR/RAPTOR.rdf

Caltech feed (re-published GCN translation)

http://www.estar.org.uk/voevent/Caltech/Caltech.rdf

See the eSTAR website http://www.estar.org.uk/ for more information.

IVOA Interop Meeting, May 2006

15

Where are we going?

• Event brokering (aggregators)– time critical– non-time critical (data mining?)

• Event archiving and persistent storage– REST (and SOAP) interfaces– persistence of data products

• Search service– REST interface

• Authentication– signatures

IVOA Interop Meeting, May 2006

16

We should not mandate transport!

• We should not tie our document or protocol standards to a single mode of transport;– bad design decision– RFC 1149, see http://www.blug.linux.no/rfc1149/– longer term problems

IVOA Interop Meeting, May 2006

17

Cathedral and the Bazaar

• Design by committee– cathedrals– slums

• Design by dictate– lottery

• Design by market pressure– betamax vs. VHS– market pressure

IVOA Interop Meeting, May 2006

18

Conclusions

• Probably need a backbone protocol

• Probably going to be TCP(V)– low buy-in for both publishers and subscribers– fully supported across different platforms– momentum is in that direction

• But other transport mechanisms are needed

IVOA Interop Meeting, May 2006

19

The 2nd HTN WorkshopJuly 23-26, 2006 in the Institut für Astrophysik, Göttingen, Germany

Aims• Establish the standards for interoperability between robotic

telescope networks.

• Work towards the establishment of an e-market for the exchange of telescope time.

• Establish the standards for interoperability with the Virtual Observatory (VO) for event notification.

See http://www.telescope-networks.org/ for details.