THE BTeV/CMS PIXEL TESTBEAM AT FERMILAB Lorenzo Uplegger.

THE BTeV/CMS PIXEL TESTBEAM AT FERMILAB

Lorenzo Uplegger

Introduction to BTeVIntroduction to BTeV

The BTeV experiment was designed to investigate one of the most fundamental problems of elementary particle physic, the CP violation.

The Standard Model of Particle Physics shows matter-antimatter asymmetry in K meson and B meson decays, but it fails to predict the magnitude of the baryon-antibaryon asymmetry observed in the universe.

Looking among the B decays for new sources of matter-antimatter asymmetry was the goal of the experiment.

In order to achieve this goal a highly sophisticated trigger system had to be used in order to collect the high statistics samples to make precision measurements.

The challenge for the BTeV trigger and data acquisition system was to reconstruct particle tracks and interaction vertices for EVERY interaction and to select interactions with B decays.

b

p p

B-meson

In a B event, in addition to tracks at Interaction Vertex, there is a B that travels ~few mm and then decays. Only 1 event out of 500 is a B event and only a small fraction of them are interesting for CP violation

The idea was to trigger at the first level selecting events with a vertex detached from the interaction vertex, thus triggering on the peculiar characteristic of events containing heavy quarks.

THIS REQUIRES SOPHISTICATED TRACK AND VERTEX RECONSTRUCTION AT THE LOWEST LEVEL OF THE TRIGGER

To succeed in this challenge, it was necessary to provide to the trigger processor the cleanest, easiest-to-reconstruct input from the vertex tracking system HENCE THE SILICON PIXEL DETECTOR WAS THE KEY FOR THE SUCCESS OF BTeV

Characteristic of b decaysCharacteristic of b decays

Muon

EM Cal

Straws &Si Strips

SM3 Magnet

RICH

30 Pixel Detector Stations

BTeV detector layoutBTeV detector layout

b production angle

b production peaks at large angles with large bb correlation

b production angle

The higher momentum b’s are at larger ’s

-ln(tan )2

ppbb+X

Characteristic of hadronic b productionCharacteristic of hadronic b production

Given the importance of the pixel detector, an extensive testbeam program was necessary to demonstrate that the pixel detector developed by FNAL satisfied the BTeV requirements.

The testbeam effort started in 2001 but only in May 2004 the beam was available and just one month after we received the detectors with the latest version of read-out chip (FPIX2) to be tested.

The main goals that we wanted to achieve were:1. Measure the spatial resolution of the sensors before and after irradiation

3. Test the read-out chip (ROC) in the real BTeV working conditions(data-driven mode)

4. Build events using only the temporal information (time-stamp) associated to pixel cells without the aid of an external trigger

2. Study the temperature dependence of the spatial resolution

BTeV pixel testbeam goalsBTeV pixel testbeam goals

FPIX1 Y measuring planes at 0 deg

FPIX1 X measuring planes at 10 deg

FPIX2 Detector Under TestCan rotate from 0 to 25 deg

in steps of 5 deg

120GeV p

• 3 FPIX1 pixel detectors upstream and 3 FPIX1 pixel detectors downstream compose the telescope

• The FPIX2 detector under test is in the center. it can be rotated from 0 to 25 degrees a cooling system can lower the temperature to -5° C Y measuring

orientation

X measuring orientation

Experimental setupExperimental setup

Detector PMC (mezzanine card) PTA (PCI Card)

Host computer

The overall philosophyThe overall philosophyFPGA

It allows users to implement theirown code (FIRMWARE)

Detector BA mezzanine

card forDetector B

A generic

PCI card

Detector AA mezzanine

card forDetector A

A generic

PCI card

A host

computer

Detector CA mezzanine

card forDetector C

A generic

PCI card

Readout process

Data1

Data2

Data3

Data5

Data4

Data6

Data7

Data8

Data11

Data10

Data9

Data12

EVENT

DAQ main featuresDAQ main features

Readout process

noise noise


Readout process

noise

noise

noiseData

1Data

4Data

6Data11

Data10

Data7

Data8


Readout process

noise

noisenoiseData

1Data

4Data

6Data11

Data10

Data7

Data8

noiseData

1Data

4Data10

Data7

Data1

Data2

Data3

Data5

Data4

Data7

Data10

Data9

Data12

?????


CLK

particles

timestamp

2Mhz

123456

Data1Ts 2

Data2Ts 2

Data3Ts 2

Data4Ts 2

Data5Ts 2

Data6Ts 2

Data7Ts 2

Data1Ts 2

Data2Ts 2

Data3Ts 2

Data4Ts 2

Data5Ts 2

Data6Ts 2

Data7Ts 2

Data1Ts 6

Data2Ts 6

Data3Ts 6

Data4Ts 6

Data5Ts 6

Data6Ts 6

Data7Ts 6


The read out system works in absence of an external trigger

The data collection from the different pixel detectors is therefore asynchronous

The DAQ must assemble the events in asynchronous mode

Events are built using the time-stamp information


In order to balance the different acquisition rates between the detectors, PCI cards and the PC, we took particular care in the design of the

FPGA firmware and the read-out software

Each PTA card is provided with two memory banks (1 Mb each) and the firmware has been designed to make use of these memories as a data-rate compensation buffer.

Signals from thePMC mezzanine cardsThe Altera

programmableFPGA

Output to host PC

Globaldata path

Two memory banks (1 Mb)

The PTA design

MemoryMemoryfull!!!full!!!

The PTA design

The first task that I was given when I came to Fermilab was the programming of thePTA card in order to handle a continuous data flux from the pixel detectors.

Let’s see the strategy that I implemented in the FPGA

• This process of periodic memory swap and transfer to a shared memory continues indefinetely.

• A shared memory is a chunk of memory in the host computer which can be accessed by different processes.

• We have several PCI cards playing this swap game in parallel: in order to be able to build events at a later stage, we needed a syncronization mechanism to keep the event builder as simple as possible.

• By synchronizing the swapping of all the memories we can build events in a very simple and immediate way.

PCI card working mechanismPCI card working mechanism

Interrupt handler AInterrupt handler A

Interrupt handler BInterrupt handler B

Interrupt handler CInterrupt handler C

Interrupt handler nInterrupt handler n

0 1Banks

1. The PCI card C redirect immediately the data flux to the other empty memory

Readout working mechanismReadout working mechanism

2. The interrupt handler of the PCI card C forces the other cards to swap and then starts flushing its content to the host PC

Each PCI card has its own interrupt-handler process listening for the memory-full signal

Let’s suppose, for instance, that the PCI card C is the first being filled up.

3. The other cards start flushing their (partially) filled memory banks to the host PC.

0 1Banks Readout working mechanismReadout working mechanism

1. The PCI card C redirect immediately the data flux to the other empty memory

2. The interrupt handler of the PCI card C forces the other cards to swap and then starts flushing its content to the host PC

Each PCI card has its own interrupt-handler process listening for the memory-full signal

Let’s suppose, for instance, that the PCI card C is the first being filled up.





0 1Banks Readout working mechanismReadout working mechanism

A0C0 … n0B0

• This architecture guarantees that events with contiguous time-stamps belong to buffers which are also contiguous in the read-out process.





0 1Banks

A0C0 … n0B0

Readout working mechanismReadout working mechanism

A1C1 … n1B1

• This architecture guarantees that events with contiguous time-stamps belong to buffers which are also contiguous in the read-out process.





0 1Banks

BUFi BUFi+1

A0C0 … n0B0 A1C1 … n1B1

Readout working mechanismReadout working mechanism• This architecture guarantees that events with contiguous time-stamps belong to buffers which are also contiguous in the read-out process.

• Events with the same time-stamp are contained within the boundaries of this overall buffer (BUFi), or at least in the next one, BUFi+1, but not in BUFi+2, making the event-builder an implementation of a sorting algorithm.





Event

Builder

Shared Memory

(unordered data)

Event buffer

(ordered data)

Timestamps:

Event

• Every hit with a new timestamp starts a new event (column) in a buffer.

• Other hits with the same time-stamp are appended to the right column in the buffer

• When the analysis of the BUFFER i+1 is over, it is reasonable to assume that there are no more data related to an event that begun in BUFFER i.

Event builderEvent builder

7 plane coincidences

FPIX1 Telescope

FPIX2 DeviceUnder Test

XY tracks distribution on the pixel detectorsXY tracks distribution on the pixel detectors

Resolution studiesResolution studiesThe resolution is calculated taking the RMS value of the distribution obtained subtracting the projected point of the track reconstructed by the telescope and the point measured on thedetector under test.

• Linear charge sharing applied

• Threshold set at 1800 e-

• The distribution of the residuals decreases until 15deg and then increases again.

• The resolution (without removing telescope resolution) varies from 12 to 7.5 m

• The change in temperature doesn’t change significantly the resolution.

Temperature dependence at different anglesTemperature dependence at different angles

• Linear charge sharing is applied

• Threshold set at 1800 e-

• The resolution (without removing telescope resolution) varies from 12 to 7.5 m

• The same behavior is observed for three different Voltages applied to the sensor.

Sensor bias dependence at different anglesSensor bias dependence at different angles

The angular dependence of the resolution can be easily understood observing the percentage of clusters sizes for different angles. The resolution improves when the number of hits in the cluster is 2 and the charge sharing can be used to determine the position of the hit.

Cluster size as function of angleCluster size as function of angle

TotalCluster size = 1Cluster size = 2Cluster size = 3

Results SummaryResults Summary

1. Measure the spatial resolution of the sensors before and after irradiation

3. Test the read-out chip (ROC) in the real BTeV working conditionsdata-driven mode

4. Build events using only the temporal information (time-stamp) associated to pixel cells without the aid of an external trigger

2. Study the temperature dependence of the spatial resolution

Most of the goals that we specified were achieved.

The study of the irradiated detectors has been postponed when I’ll find some free timeto analyze the data collected last March with one of the detectors that we irradiated last year.

From BTeV to CMSFrom BTeV to CMS

After BTeV cancellation, I was particularly attracted by the perspectives offered by the CMS experiment.Here, not only the physics program was very appealing, but we could even profit from our recent experience on BTeV to help on the forward pixel detector.

There were several problems which we could work on to help the CMS collaboration andone of them was the low data acquisition rate their system could handle at the testbeam.

The CMS telescope is based on silicon strip detectors which have a very slow integration timeand a trigger system that has to wait for a confirmation from the software before releasing the veto signal.

The result was the collection of ~300 events for each spill ( 0.6 second of beam every ~6 seconds) while, with the same beam structure, we had a rate of ~50000 events.

With the new spill structure that is given right now (4 seconds of beam every 2 MINUTES)the collection of high statistic samples is very time consuming.

On the other hand the BTeV DAQ has a much higher bandwidth and can collect many more events as the beam intensity increases.

The net result is an improvement of the number of collected data, which can be increased by an order of magnitude.

Furthermore, the flexibility given by the FPGA micro-programming allow us to adapt the system quickly in case of necessity.

From BTeV to CMSFrom BTeV to CMS

CMS testbeam setupCMS testbeam setup

CMS detector layoutCMS detector layout

D:\CMS Outreach\CMS Outreach\CMSmedia\CMSmovies\CMS_Slice.swf

Pixel detectorPixel detector

Barrel

Forward pixel (USCMS)

UC DavisFermilab

Johns HopkinsMississippi

NorthwesternPurdueRutgers

PSIETH

U. ZurichU. Basel

IHEP WienRWTH AachenNew Zealand

Pixel forward detector Pixel forward detector

1/2 disk

Panels

BTeV and CMS pixel detectors comparisonBTeV and CMS pixel detectors comparison

• Pixel dimensions 50x400 µm2

• BCO is 132ns (~8Mhz)

• Readout clock is 70Mhz

• Max readout speed is 840Mbits per seconds over 6 output LINES

• Binary digital output

• ADC internal charge conversion

• Data driven read-out

• Registers read-back operations

• Few important adjustable parameters, like thresholds, leakage current compensation and reference voltage.

• Pixel dimensions 100x150 µm2

• BCO is 25ns (40Mhz)

• Readout clock is 40Mhz

• Max readout speed is 240Mbits per seconds over 6 LEVELS

• 6 levels digital output

• Analog charge information

• Triggered read-out

• No registers read-back

• Many important adjustable parameters, like threshold, leakage current compensation, analog output gain and offset, levels gain and separation, analog and digital voltages.

BTeV CMS

The two detectors are designed with completely different philosophies.

The first problem that I had to solve to integrate the CMS pixel detector in the BTeV telescope system was the synchronization of the events of the two systems.

• The BTeV detector does not use any trigger signal. The read-out is data driven and events are reconstructed using the timestamp information provided by the chip.

• The CMS detector, instead, requires a trigger signal to start the read-out. Events are the collection of the hits associated to a trigger.

CMS synchronization with the telescopeCMS synchronization with the telescope

All the BTeV detectors on the beam are synchronized in order to get the same timestamp from the hits generated when a particle passes through them.

Data1Ts 6

Data2Ts 6

Data3Ts 6

Data5Ts 6

Data6Ts 6

Data7Ts 6


26 bits counter

2Mhz Clock

26 bits counter

RST

CMS FPIXBTeV FPIX1

6 bits counter Analog output

Telescope Timestamp

20 most significant bits mezzanine counter 6 FPIX BITS

CMS Timestamp

26 bits counter

Master PMC card Slave PMC card


Data1Ts 6

Data2Ts 6

Data3Ts 6

Data5Ts 6

Data6Ts 6

Data7Ts 6

Coincidence TBM

CMS DataTs 23

The Timestamp difference between the CMS pixel and the FPIX detector is CONSTANT dependent by the time taken by the scintillators to form the signal, by the cable length, by the TBM and PSI response to a trigger.

Scintillators

Coincidence DATA



00110010

One of the main problem that we faced was the reconstruction of the data output fromthe detector. Data are not sent out in binary format but they are encoded on 6 levels.

Analog output ADCBinary digital output

Original configurationOriginal configuration

Telescope Host PCPCI bus expander

Sharedmemory

Massstorage

Modified configurationModified configuration

Telescope PCI bus expander

………………………………………………… ………

CMS

COSMOPC in the hut

Custom built ADC

Host PC

Sharedmemory

Massstorage

Modified configurationModified configuration

7 planes coincidences measured on the CMS pixel detector

Proof of conceptProof of concept

Once we proved that the synchronization scheme was working we started immediatelyto work on the initialization and control of the chip itself to give us more flexibilityin case we needed to study some parameters related to the readout clock.

The task was not easy given the big number of parameters that can be tuned for eachdetector

Total integration in the DAQTotal integration in the DAQ

Summer testbeamSummer testbeam

Although the effort to fully control the detector was not as simple as we were expecting, after a month, the full integration of the CMS pixel detector in thetestbeam DAQ was completed.

We requested some beam time in mid July and we started to take data with the new system.

We were able to characterize at different angles and different temperatures3 detectors, before we took them last week to be irradiated.

The adapted DAQ was able to collect around 25000 events per spill which reducedby big factor the time needed to collect high statistics for precision studies.

This was the first time that the new chip with the analog information was tested on the beam.

Here are the very first results of this summer data taking…

First results of the summer data takingFirst results of the summer data taking

Charge collection efficiency measured along the pixel dimensions

µm µm

Results summaryResults summary

The new DAQ system proved to be faster than the one that CMS was using before.

The analysis is still in progress since the last run was taken few weeks ago,nevertheless I showed the preliminary results of the charge collection efficiencyin the CMS pixel detector.

In the next future we are going to study the charge collection efficiency of the detector that we have just irradiated.

After the testbeam performance in May we were asked to evaluate the possibilityto develop a new test-stand for the CMS pixel.

Based on the experience accumulated in these years I contributed to the design of a new PTA card which will improve considerably the readout bandwidth in order to keep up with higher rates.

The new PTA will play an important role in the new test-stand which will be used for the imminent CMS pixel production.

This PTA will be also very useful to test the detector in a high intensity beam to simulateas much as possible the real condition of operation in LHC.

A new test-stand for the CMS pixel detectorA new test-stand for the CMS pixel detector

The new PTA card

Why there is the need for a new test-stand?Why there is the need for a new test-stand?

The problem that the CMS collaboration was facing was again a bandwidth problemsince the full characterization of a plaquette ( a module containing more than one chip required 117 hours. After a lot of improvements of their test-stand, now the time needed is reduced to 2 HOURS.

This time makes almost impossible a full scale testing of all the components to be installedin the final detector.

Based on the recent experience in the testbeam we believed that a new DAQ based on our PCI cards could reduce this time by a lot.

A first release of this new test-stand has been delivered to SiDet last week and the total time for a full calibration of a single chip is around 2 MINUTES.

This considerable reduction of the time needed for the characterization of the readout chipswill make possible to test all the components before being installed in the final detector,making the CMS pixel detector a successful project.

Test-stand GUITest-stand GUI

A highly efficient vertex detector is necessary in modern experiments to performstudies of events containing heavy quarks and to achieve the high precision measurementsnecessary to challenge the standard model.

With the latest development in the pixel detector technology the idea of a triggercapable to select events with heavy-quark secondary vertices is now conceivable also for the LHC experiments where the track density is very high.

Fermilab has a key role in the construction of the forward pixel tracker of the CMS experiment and it will have an important role in the upgrade of the detector too.

With the expertise accumulated in the BTeV experiment, Fermilab can become thecenter for the development of a much more efficient trigger system than the onecurrently designed for CMS.

A final remark before the conclusionsA final remark before the conclusions

ConclusionsConclusions

• I have been responsible for most of the hardware and software for the BTeV testbeam DAQ for the last 2 years, solving all the problems that we had to face during the data taking.

• I have successfully integrated the CMS pixel detectors into the BTeV DAQ at the testbeam.• We are currently running to characterize them.• It took us a few months to produce results.

• We developed a new test-stand that has been delivered to SiDet and will be crucial to characterize the detectors during the imminent CMS forward pixel production.

• I have shown the testbeam results for the BTeV and CMS pixel detectors

• I presented the BTeV results at the last IEEE conference

A lot of work still needs to be done…

• From the preliminary results the new PSI readout chip seems to work very well.• This is a great result for all of us.

THE BTeV/CMS PIXEL TESTBEAM AT FERMILAB Lorenzo Uplegger.

Documents

Transcript of THE BTeV/CMS PIXEL TESTBEAM AT FERMILAB Lorenzo Uplegger.