SLHC Design Considerations for the CSC Track-Finder & An Asynchronous Trigger Proposal Darin Acosta and Alex Madorsky University of Florida Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 2 Outline n CSC Muon Trigger u Brief review of the CSC Track-Finder for CMS u CSC BX assignment issues u CSC occupancy estimates u Some obvious changes for SLHC operation n Interesting R&D Directions u Xilinx Rocket IO X technology u Proposal to go asynchronous at Level-1 l After BX assignment, of course Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 3 Strip FE cards Wire FE cards Muon Port Card (Rice) MPC Sector Receiver/ Processor (U. Florida) OPTICAL SR/SP SP CSC Muon Sorter (Rice) Global Trigger DTRPC FE Global L1 2 / chamber 3 / port card 3 / sector 44 44 44 44 LCT Trigger Motherboard (UCLA) Wire LCT card In counting house TMB LCT RPC Interface Module RIM On-Chamber Trigger Primitives 3-D Track-Finding and Measurement Combination of all 3 Muon Systems CSC Muon Trigger Scheme EMUTrigger Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 4 CSC Track-Finder Crate SR / SP SR / SP SR / SP SR / SP SR / SP SR / SP SR / SP SR / SP SR / SP SR / SP SR / SP SR / SP CCB SBS 620 Controller Sector Processor Clock & Control Board Muon Sorter From MPC (chamber 4) From MPC (chamber 3) From MPC (chamber 2) From MPC (chamber 1B) From MPC (chamber 1A) To DAQ MS 180 1.6 Gbit/s optical links: Data clocked in parallel at 80 MHz in 2 frames (effective 40 MHz) Custom 6U GTLP backplane for interconnections (mostly 80 MHz) Rear transition cards with 40 MHz LVDS SCSI cables to/from DT Single crate solution, 2 nd generation prototypes under test Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 5 SP2002 Main Board (SR Logic) Optical Transceivers 15 x 1.6 Gbit/s Links Front FPGA TLK2501 Transceiver Phi Local LUT Eta Global LUT Phi Global LUT PLL patch To/from custom GTLP back- plane SR Logic Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 6 SP Trigger Logic n Xilinx Virtex-2 XC2V4000 ~800 user I/O n Same mezzanine card is used for Muon Sorter n Track-Finding logic operates at 40 MHz u Frequency of track stub data from optical links n Easily upgradeable path SP2002 mezzanine card Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 7 Track-Finding Latency 11 25 ns, or 275 ns Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 8 CSC BX Assignment n CSC Time resolution u 60 ns maximum drift-time per plane, 6 planes per chamber 5 ns chamber resolution (ALCT takes time from 2 nd hit) u Not likely to change for SLHC n Centered peak: u For time distribution centered in BX interval: LHC SLHC l Probability to get LCT in correct BX LHC: 98.8% SLHC:80% (will clearly need to consider multi-BX) 98.8%0.6% BX 1 BX 2 BX 3 80%10% BX 1 BX 2 BX 3 Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 9 From Andrey: TB99 Data Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 10 Other CSC Time Scenarios for SLHC n Bifurcated peak: u Fit 98.8% of LCTs in 2 BX window, but ambiguity on which BX muon belongs n Offset peak by 3 ns: u Puts 97% of LCTs in 2 BX window u 70% in central BX u 27% or 3% in following BX (choice depends on BX algorithm for Track-Finder) 49.4%0.6% BX 1 BX 2 70%27% 3% BX 1 BX 2 BX %0.6% Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 11 Track-Finder BX Assignment n Current CSC Track-Finder takes earliest arriving LCT as definition of track BX over a multi-BX window n LHC: u 2-station tracks: l Centered peak: 98.8% correct BX assignment n SLHC: u 2-station tracks: l Offset peak: 87% l Centered peak: 80% u 3-station tracks: l Offset peak: 89% l Centered peak: 73% 2 BX window (25 ns) for these efficiencies n 88% BX i.d. efficiency with offset peak and taking earliest arriving LCT, two or more stations Offset peak gives best performance Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 12 Alternative Track-Finder BX Assignment n Take ALCT approach, and consider second arriving segment to define track BX n SLHC: u 2-station tracks: l Offset peak: 87% correct BX assignment l Centered peak: 80% u 3-station tracks: l Offset peak: 78% (82%) l Centered peak: 90% (95%) 2BX (3BX) windows n Can improve BX i.d. efficiency to 95% with centered peak, taking second LCT, requiring 3 or more stations Offset peak gives worse performance for 3 stations Same as before Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 13 Andreys Conclusion: Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 14 Improved CSC Performance? n Its worth considering what can be done to slightly improve CSC timing (change gas, increase high-voltage) u Suppose 5 ns resolution 4 ns n BX i.d. from first hit: u 2-station tracks: l Offset peak: 93% l Centered peak: 88% u 3-station tracks: l Offset peak: 96% l Centered peak: 83% 88%6% BX 1 BX 2 BX 3 77%22% 1% BX 1 BX 2 BX 3 BX i.d. from second hit: u 2-station tracks: l Offset peak: 61% l Centered peak: 88% u 3-station tracks: l Offset peak: 87% (87%) l Centered peak: 96% (98%) Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 15 CSC Occupancy (LHC) n Start from CMSIM trigger study u R.Cousins, J.Mumford, and V.Valuev: CMS Note 2002/007 u Dedicated ORCA trigger simulation, albeit with LCT logic that does not exactly match final production hardware & firmware n Correlated LCT Occupancy (ALCT+CLCT match) u Entire CSC system: 0.05 / pp collision or 0.9 LCTs per L = u MPC L = l ME1: / BX *rescaled to 30 subsectors l ME2 4: / BX l Recall that MPC can accept up to 3 LCTs / BX u Adding neutrons leads to 30% increase in ME1 ME3, 3X higher in ME4 l Ignore neutrons for now. Correlated LCT rate from neutrons probably doesnt scale linearly with luminosity since it is composed mostly of random hits. Hard to make projections. Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 16 Projected CSC Occupancy (SLHC) n SLHC u Assume L = 10 35, BX interval decreases to 12.5 ns, chambers stay the same n Correlated LCT Occupancy u Entire CSC system: 0.9 * 10 / 2 = 4.5 LCTs / 80 MHz u MPC L = 10 35, assuming 80 MHz operation l ME1: / BX (ME2 4 is 3X smaller) P ( 2) = 0.7% (spoils di- measurement in 1 MPC) u MPC L = 10 35, assuming 40 MHz operation l ME1: 0.25 / 25 ns (ME2 4 is 3X smaller) P ( 2) = 2.6% (spoils di- measurement in 1 MPC) u Occupancies are not huge, but we neglected neutrons, and LCT ghost probability might be higher. Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 17 Projected Occupancy in CSC Track-Finder n For optimum efficiency and decreased sensitivity to exact CSC timing, we plan to use a 2 BX (50 ns) window at LHC to accept LCTs from the MPCs u e.g. See A.Drozdetskis testbeam analysis talk from Oct.03 EMU meeting u LCT efficiency goes from 98% to 99.5% with 2 BX window, Track-Finding efficiency goes with square or cube of this u Earliest arriving LCT defines BX (but this may not be best choice) n SR occupancies for L = 10 35, 4 BX window u ME1: 0.5 / 50 ns (every other trigger BX!) u ME2 4 is 3X smaller u BX i.d. study suggests only 2 3 BX will be required n Need to perform detailed rate studies to see if we pick up fake tracks that trigger Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 18 Inclusion of Muon Data with Tracker n It would be extremely desirable to include tracker data at Level-1 n How it is planned to be used at LHC for HLT: u Attach tracker hits to improve P T assignment precision from 15% standalone muon measurement to 1.5% with the tracker l Will improve sign determination as well and offers vertex constraints u Find pixel tracks within cone around muon track and compute sum P T as an isolation criterion l Less sensitive to pile-up than calorimetric information if primary vertex of hard-scattering can be determined (~100 vertices total at SLHC!) n To do this requires information on the muons finer than the currently reported 0.05 2.5 u No problem, since both are already available at and 0.015 Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 19 Muon Rate at L = From DAQ TDR Note limited rejection power (slope) without tracker information Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 20 Preliminary Conclusions on CSC for SLHC n Will probably want to upgrade front-end trigger boards and optical links to send LCT 80 MHz u Simulated muon occupancy may be low enough to avoid this, but with strong caveats. Real data may be worse. n BX identification is about 80% correct at LCT level n BX identification is about 90% correct at Track-Finder using current algorithm u 2 BX acceptance window, 2 or more stations, offset timing peak n Can be increased to 95% correct BX i.d. at Track-Finder u 3 BX acceptance window, 3 or more stations (need ME4!) u Requires new logic in Track-Finder to take second LCT time n Trivial to add more finer eta and phi information to reported muon candidate for use by a muon-tracker match box More Generic R&D Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 22 Xilinx RocketIO X u Maximum speed: Gbit/sec u Latency is not yet published u Will be better than RocketIO according to tech support u Minimal latency calculation (in clock cycles) based on Rocket IO documents: Input register2.5 TX CRC (bypass)1 8B/10B encoder1 TX FIFO4.5 TX SERDES1.5 RX SERDES1.5 Realingment3.5 8B/10B decoder1 RX FIFO (reduced latency)2 Output register2.5 Total21 A lot, but Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 23 RocketIO X u Clock cycle at 10 Gbit serial speed: ~250 MHz u Total latency: ~84 ns (should be better for RocketIO X) n Thus, probably an interesting avenue to explore to bring huge data volumes into an FPGA Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 24 An Asynchronous Level-1 Trigger? n The high-speed data links that will be available for use at SLHC (10 100 Gbit/s), and the challenges of timing in a fully synchronous system and distributing a jitter-free clock at the sub-ps level, started us thinking about going asynchronous after the front-end BX assignment u We DO still need a clock synchronous with the machine frequency distributed to the FE boards for BX assignment u But unlike early LHC electronics, there will now be several orders of magnitude separating the machine frequency from the data communication frequency n We KNOW this has to work because HLT is already asynchronous u Moreover, we know that CSC trigger data collected synchronously at the CSC Track-Finder exactly matches the DAQ data collected asynchronously Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 25 Advantages of an Asynchronous Design 1.De-couple clocking requirements of high-speed data links from synchronous BX assignment nData instead is sent with a BX label when it is available, and trigger logic assembles event for processing nCan use on-board xtal oscillators for serdes reference clock, rather than multiplying an 80 MHz clock to multi-GHz 2.Maximize the utility of the available bandwidth nA synchronous system must keep a low occupancy on the data links, otherwise Poisson fluctuations in one BX will overflow the link wasted bandwidth by sending lots of 0s nSome trigger subsystems recognized this and already serialize data over multiple BX nRPC, DT nA step toward an asynchronous design already! Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 26 Advantages II 3.Respond more robustly to bursts nCould allow more segments/clusters in an unusually busy event by allowing transmission time to vary (Imagine a black-hole event) 4.Detector technology may not keep pace with shorter SLHC BX ne.g. The CSC FE may not have the ability to accurately determine the 12.5 ns bunch. Track-Finder might need a 50 ns (4 BX) window to trigger efficiently. nDT system may be in even worse shape with BX assignment nLess compelling why data must be sent synchronously when it is naturally distributed over several BX nMight it be possible to not have a machine BX at all, but one long train? (BX i.d. then becomes a time-stamp) Big question will be detector occupancies Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 27 Advantages III 5.Decouples clock frequency of algorithm from BX frequency nA shorter BX does not mean faster logic. Logic works at transistor switching speed. Too short of a clock means wasted overhead to allow signals to settle before latch nMight allow continued use of legacy 40 MHz boards 6.Might allow incorporation of DSPs and CPUs into trigger architecture nIs a high-performance DSP useful? nWhat about embedded PowerPC chip in FPGA? nIf not for triggering, very useful for slow control, DAQ, nPerhaps track-finding, jet-clustering, b-tagging could benefit nMight think of this as merging traditional L2 into L1, rather than L2 into L3 as CMS now does. Given that we want tracker at L1, maybe this is a way we need to go. Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 28 Constraints n Proper BX assignment is still required at front-end u Data is stamped with 12.5 ns bunch crossing time u Distribution of synchronous 80 MHz clock much less challenging (its done already) than one used to drive data links l TTC system may be overly complicated for what we need n Level-1 decision must be reached by a maximum latency or sooner u Must have a time-out mechanism on data transmission and algorithms u Still must keep latency short! n Event building circuitry needed in FPGAs u Should be small n Escape clause: u Its always possible to wrap up the asynchronous logic into a synchronous black box (re-align data at trigger output) l We considered this as an option for the MPC SP optical link transmission in the current system, for example, to solve clock jitter issues l A matter of determining how small or large an asynchronous block can be built Darin Acosta, University of Florida 13 Feb 2004 SLHC Workshop, Madison WI 29 Advanced Detector Research Proposal n Submitted a proposal to the DOE Advanced Detector Research program for FY04 to prototype such an asynchronous system n Based on existing CSC detector, with upgrade to ALCT logic and redesigned Processor board