Role of Digital Packaging Processor
• To filter, average, partially calibrate and convert raw correlator output into a Miriad-compatible format that is written to Interim Data Base
• Real time, irreversible processing
1
1
DPP Interface Overview
DPP
State Frame
ACC
Correlator
Start / End Scan Commands
Scan-independent Calibration Parameters
Scan Parameters
Frame parameters
Frame status report
<P>, <P2>,
Correlations
Interim Data Base
Miriad
format
Internal RFI
Database1 s timing tick
0.02 s timing tick
RFI
results
DPP Task Timescales• Data frame (20 ms)
– filter, and frequency-average correlator output
• Spectral frame (1 s) – Assemble, pre-calibrate, reformat and write data to Interim database
• Scan initiation– Accept, store and preprocess scan-specific parameters
• Occasional – non operational– Accept, store and preprocess calibration parameters
• TBD – Format results and write to RFI database
DPP Software Architecture
ACC State Frame Correlator IDB
DPP_CONTROL
I/O, data assembly, Miriad formatting, no numerical processing
DPP_Process_Header
DPP_Process_Dataframe
DPP_Process _Spectral _Frame
Communication via Common Blocks
Conventional, time-independent processing tasks with “Clean” inputs
C1
C2 C3, C4 …. C2
Cn = core within a mullti-core processor or nodes in a cluster
DPP
RFI database
DPP_Process_Parameters
C2
DPP_CONTROL-- Jim McTiernan --
• Handles all external I/O• Passes reformated header and calibration data
on to processing tasks• Assembles data frame input from correlator and
calls processing task only if/when complete• Assembles processed 20ms interval data and
calls spectral frame processor only when all accumulations are available
• Converts output of spectral frame processor to Miriad format
DPP_PROCESS_DATAFRAME• Tasks
– Identify RFI-affected subbands as a function of frequency only.– Combine with pre-flagged subbands to generate a “destination
vector” for each subband– Save RFI statistics – details TBD– Average subband data into spectral channels
• Challenge: 450 MB/s – Each visibility is handled only once– Multiple processors (up to 6) handle successive dataframes
• Initial Restriction:– No calibration at the subband level
DPP_PROCESS_SPECTRAL_FRAMEEvery spectral frame (1s)
• Convert antenna-based flags (e.g. slewing) from state frame to baseline-based, frequency-independent flags
• Apply time-independent complex gains • Correct for attenuator settings• Fine delay corrections-------------------------------------------------------• Baseline corrections
– (not necessary)• Apply non-linearity corrections
– Refinement to be added later, if necessary• Correction for spectral simultaneity
– not needed if using Miriad wide-band mode• Conversion of visibility, uv and analysis-relevant state-frame
data to Miriad-compatible format– May be a included here, in DPP_Control or as a separate Process
DPP-Correlator Interface• 4096 x 2 x16^2 8-byte visibilities
• 4096 x 2 x 16 8-byte <P2> values 9 MB / data frame 450 MB/s
• Two Dedicated 10 Gb Ethernet link(s)
• Interface architecture driven by correlator design
Format of Correlator Output
• Format of interface based on 3-Feb-2012 EST memo from Nimish, as clarified in 12-March-2012 email
• Features:– Accumulation divided into labeled packets, each with an
identical header (except for sequence numbers)– Packets divided into ‘chunks’, each of which contain all power,
power^2 and correlated data for a given subband– Data for a given subband has both polarizations interspersed,– All non-header data are scaled 4-byte integers
• Refinement:– Agreed that correlator output should always correspond to 16-
antenna/2 polarizations. – DPP will discard unused antennas/baselines
DPP-Correlator Interface Open Issues
• Specifics of EST format generalization to EOVSA
• Byte order: – Correlator outputs Big-endian (MSB first)– Many machines (e.g. PC’s) use Little-endian (LSB first)
• How is the load to be divided between the two Ethernet interfaces?– Current format (and DPP software) assumes sequential input within a
given accumulation
DPP Status• Overall:
– Software architecture and tasks defined– Platform to be acquired
• Interfaces:– IDB – Miriad format feasibility demonstrated with EST data– Correlator - as defined by Nimish with a few remaining issues– State frame - will follow protocol defined by GN, details TBD– ACC i – definition at an early stage– Timing ticks – TBD– RFI –TBD
• Milestones – Detailed software definition, coding and testing (with EST data) is underway– DPP_CONTROL has successfully associated tasks and processors– EST data loads correctly into DPP input common block– DPP_PROCESS_DATAFRAME well-defined and being coded.– EST to Miriad format conversion has been successfully demonstrated.– Immediate goal: End-to-end testing with EST data
Some DPP Open Issues• Do we use spectral or wideband mode in Miriad?
– Probably either would work – Initiall implementation will use wide-band mode
• (simpler, more flexible but less efficient) – analysis timing tests needed to see if this is an issue– easy to switch to spectral mode
• Is GNU Fortran 95 the best compiler choice?– Drawback is lack of support for STRUCTURE statements
• Workaround available for DPP_PROCESS_DATAFRAME
• Options for software development platforms – Independently develop DPP_CONTROL and processing routines on
‘personal’ machines, then test at OVRO?– Duplicate platform located in Bay area (JM, GH or SSL?)– Duplicate platform at NJIT ?
• Need to identify and purchase a multi-processor platform for DPP in near future.
Top Related