High Data Rate Transfer for HEP and VLBI
description
Transcript of High Data Rate Transfer for HEP and VLBI
![Page 1: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/1.jpg)
High Data Rate Transfer for HEP and VLBI
Ralph Spencer, Richard Hughes-Jones and Simon Casey
The University of ManchesterNetwrokshop33 March 2005
![Page 2: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/2.jpg)
HEP: The LHC Detectors
CMS
ATLAS
LHCb~6-8 PetaBytes / year
~108 events/year
![Page 3: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/3.jpg)
Radio Astronomy• The study of celestial objects at <1 mm to >1m
wavelength.• Sensitivity for continuum sources• B=bandwidth, integration time.• High resolution achieved by interferometers.
Some radio emitting X-ray binary stars in our own galaxy:
B/1
GRS 1915+105 MERLIN
SS433MERLIN and European VLBI
Cygnus X-1VLBA
![Page 4: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/4.jpg)
Earth-Rotation Synthesis
Need ~ 12 hours for full synthesis, not necessarily collecting data for all that time.NB Trade-off between B and for sensitivity.
Telescope data correlated in pairs:N(N-1)/2 baselines Merlin u-v coverage
![Page 5: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/5.jpg)
The European VLBI NetworkEVN
• Detailed radio imaging uses antenna networks over 100s-1000s km
• At faintest levels, sky teems with galaxies being formed
• Radio penetrates cosmic dust - see process clearly
• Telescopes in place …• Disk recording at
512Mb/s• real-time connection
allows greater – response– reliability– sensitivity
![Page 6: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/6.jpg)
WesterborkNetherlands
Dedicated
Gbit link
EVN-NREN
OnsalaSweden
Gbit link
Jodrell BankUK
DwingelooDWDM link
CambridgeUK
MERLIN
MedicinaItaly
Chalmers University
of Technolo
gy, Gothenbu
rg
TorunPoland
Gbit link
![Page 7: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/7.jpg)
eVLBI Milestones • January 2004: Disk buffered eVLBI session:
• Three telescopes at 128Mb/s for first eVLBI image
• On – Wb fringes at 256Mb/s
• April 2004: Three-telescope, real-time eVLBI session.
• Fringes at 64Mb/s• First real-time EVN image - 32Mb/s.
• September 2004: Four telescope real-time eVLBI• Fringes to Torun and Arecibo• First EVN, eVLBI Science session
• January 2005: First “dedicated light-path” eVLBI• ??Gbyte of data from Huygens descent
transferred from Australia to JIVE• Data rate ~450Mb/s
![Page 8: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/8.jpg)
• 20 December 20 2004• connection of JBO to Manchester by 2 x 1 GE• eVLBI tests between Poland Sweden UK and Netherlands at 256 Mb/s
• February 2005• TCP and UDP memory – memory tests at rates up to 450 Mb/s(TCP) and 650 Mb/s (UDP)• Tests showed inconsistencies betweeb Red Hat kernals, rates of 128 Mb/s only obtained on 10 Feb• Haystack (US) – Onsala (Sweden) runs at 256 Mb/s
• 11 March Science demo• JBO telescope winded off, short run on calibrator source done• Test needs to be repeated!
![Page 9: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/9.jpg)
![Page 10: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/10.jpg)
Jodrell BankUK
OnsalaSweden
MedicinaItaly
TorunPoland
EffelsbergGermany
WesterborkNetherlands
Telescope connections
JIVE
1Gb/s
1Gb/s
1Gb/s
155Mb/s
MERLIN
CambridgeUK
2.5Gb/s
1Gb/s
MERLINe
?????
![Page 11: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/11.jpg)
Tests on the Network, Manchester-Dwingeloo: investigation of packet loss
• 4th year MPhys Project at The Univertsity of Manchester Oct-Dec 2003, using campus network and SuperJANET 4 academic network in the UK.• UDPmon used to test throughput and packet loss• NB tune up the end machines – see http://grid.ucl.ac.uk/NFNN.html
Effect on the local traffic:
![Page 12: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/12.jpg)
Gnt5-DwMk5 11Nov03-1472 bytes
0
2
4
6
8
10
12
0 5 10 15 20 25 30 35 40Spacing between frames us
% P
acket
loss
Gnt5-DwMk5
DwMk5-Gnt5
Throughput vs packet spacing Manchester: 2.0G Hz Xeon Dwingeloo: 1.2 GHz PIII Near wire rate, 950 Mbps NB record stands at 6.6 Gbps
SLAC-CERN
Packet loss
CPU Kernel Load sender
CPU Kernel Load receiver
4th Year project Adam Mathews Steve O’Toole
UDP Throughput Manchester-Dwingeloo (Nov 2003)
Gnt5-DwMk5 11Nov03/DwMk5-Gnt5 13Nov03-1472bytes
0
200
400
600
800
1000
1200
0 5 10 15 20 25 30 35 40Spacing between frames us
Recv W
ire r
ate
Mbits/s
Gnt5-DwMk5
DwMk5-Gnt5
Gnt5-DwMk5 11Nov03 1472 bytes
020406080
100
0 5 10 15 20 25 30 35 40Spacing between frames us
% K
erne
l S
ende
r
Gnt5-DwMk5 11Nov03 1472 bytes
020406080
100
0 5 10 15 20 25 30 35 40Spacing between frames us
% K
erne
l R
ecei
ver
![Page 13: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/13.jpg)
Packet loss distribution:
( )t
p d
Cumulative distribution
Cumulative distribution of packet loss, each bin is 12 sec wide
Long range effects inthe data?
Poisson
![Page 14: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/14.jpg)
26th January 2005 UDP Tests
Simon Casey (PhD project) Between JBO and JIVE in Dwingeloo, using production networkPeriod of high packet loss (3%):
![Page 15: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/15.jpg)
The Future:
• eVLBI tests in EVN at ~ 1 per month• Network tests at up to 950 Mb/s – statistics of
packet loss• Investigate alternatives to TCP/UDP – tsunami,
VSI-E etc.• ESLEA tests on MB-NG and UKLight• EXPRes EU eVLBI proposal including
distributed processing, JBO-Onsala test link at 4 Gbps, submitted March 19th 2005
• eVLBI will be routine in 2006!!
![Page 16: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/16.jpg)
Processing Nodes
Controller/DataConcentrator
VLBI Correlation: GRID Computation task
![Page 17: High Data Rate Transfer for HEP and VLBI](https://reader035.fdocuments.in/reader035/viewer/2022062803/568147ab550346895db4e640/html5/thumbnails/17.jpg)
Questions ?