Controls Activities & O rganisation during LS1

Controls Activities &

Organisation during LS1

Eugenia Hatziangeli BE/COLS1 Day – 12th June 2012

Input from Machine Controls Coordinators (S. Jensen, M. Pace, M. Cattin, F. Locci) LS1 Controls Coordinators (C. Dehavay, M. Draper), M. Vanden Eynden, P. Charrue

E.Hatziangeli - BE/CO

Outline

2

• Organisation, Planning and Objectives• Work to be done• Summary


BE-CO Organisation for LS1

3

• The LS1 work planning is done by two planners– Injectors and Facilities

• Claude Dehavay– LHC

• Claude Dehavay & Mick Draper

• They are responsible to– Represent BE-CO in all LS1 coordination meetings– Collect all BE-CO LS1 objectives and include them in the high-level LS1

planning (plan.cern.ch)– Identify, for each Accelerator, the control system functions that are

required during the LS1 with the associated level of service– Identify the impact of LS1 objectives proposed by all equipment

groups which require specific BE-CO participation (field installations, hardware procurement, software modifications)


Machine Controls Coordinators

4

• Direct links with the Equipment groups– One person for one or more machines

• In the context of renovation– They propose a renovation plan taking

into account • the control system obsolescence

problems • the operational constraints

– They participate actively in• the initial deployment • the commissioning phase of the new

technical solutions


Work Information Sources

5

• Scheduled CO shutdown upgrades, removal of legacy HW & SW• Machine Controls Coordinators• ACCOR

– Injector controls renovation project– Strong coupling with LIU Project

• LS1 Meetings– LS1 LHC Coordination (F.Bordry, K.Foraz)– LS1 Injectors Coordination (S.Baird, V.Chohan)

• PS & TT2 (R.Brown)• LS1 SPS (D.Mcfarlane)• LS1 PSB & AD (N.Gilbert)• LS1 Linac2 (C.Mastrostefano)• LS1 CNIC (B.Copy)• Linac 4 installation (J.Coupard)• SPS North @ East areas (I.Eftymiopoulos)• CTF/CLEX (F.Tecker)

6E.Hatziangeli - BE/CO

Planning Tools• BE/CO work planning (MS Project)

– Task list/machine is provided by MCCs– Detailed planning of work (start/end, duration) & resource per machine

• Major activities (plan.cern.ch)– Requests are coming in from equipment groups continously– BE/CO is entering an abstracted level of its own MS Project planning

• Must ensure there will be no duplication of requests• Not all requests are inserted yet

– Status in plan.cern.ch• Total 91 requests to BE/CO

– TE/ABT, TE/EPC– TE/MPE, TE/CRG– EN/STI– BE/ABP, BE/BI, BE/RF, BE/CO

• Resources– TE/ABT ~3FTE confirmed – 16 FTE pending– BE/CO for LS1

» HW team people @ 100% » Installation FE & IN teams: @100% + 6 FSUs @100% + additional ones if needed (know by end June)» All MCCs @40% and ACCOR + LS1 coordinators @100%» All CO Software teams working on upgrades, new functionalities, consolidation, layout, DBs

# of Requests State Groups Status

65 Not Filled Checking with requesters

4 Pending TE/ABT, EN/STI, BE/ABP

More details needed

20 Yes BE/BI, TE/ABT Completed

2 Incomplete ?? No contributing group


LS1 Controls Renovation Objectives• Renovation of the Injectors control system

– Replacement of Front-ends (~450)• Outdated VMEBus racks replaced by ELMA• VME PPC CPUs replaced by MEN A20• LynxOS replaced by Linux with RT

– Replacement of old HW modules (Tg8, GFAs, BC1553,..)– Replacement of legacy software

• GM software (125 classes) by FESA• Samplers by OASIS• Rationalization of high-level software by InCA

(LSA, Acquisition core, CBCM,)

• Upgrade of the LHC control system– Replacement of Front-ends (~450)

• VME PPC CPUs replaced by MEN A20• Industrial PC replaced by Kontron PCI-760• LynxOS replaced by Linux with RT

• Unify the responsibility model with the equipment groups– The LHC responsibility model will be used

as reference • installation, configuration, software, support

Machines– LINAC 3– LEIR– LINAC 2– PSB– CPS– SPS– REX-ISOLDE– AD

EDMS documents with formal approval process (Eq. groups, OP, CO)

– Renovated systems– Time frame & resources– Separation of responsibility


Linac3

• 6 systems remain to be renovated– BE/CO: upgrade crates, OASIS– BE/RF: RFLNP (PLC)– BE/BI: SEM, Faraday Cups (MPV908-> VD80)– EN/STI: strippers (PLC based solution)

• Remarks– Major upgrades already done in 2012 (timing, PC,…)– All remaining FEs will be renovated in LS1– No outstanding issues

Linac3 MCC M.Pace

# FRONT-ENDS # RENOVATED # DURING LS1 # AFTER LS1

14 8 6 0


LEIR

• 11 systems to be renovated in LS1– BE/CO

• upgrade crates + HW modules, • SAMPLERS to OASIS [18 devices] • FunctionGenerator GFAS to CVORB: [15 devices]

– BE/RF: DIG & ANALOG beam ctrl– BE/BI: BTC, Ring PU– TE/ABT : Kickers, bumpers

• Remarks– TE/MSC: B-TRAIN

• No confirmed date for the new TE/MSC based solutionLEIR MCC M.Pace

# FRONT-ENDS # RENOVATED # DURING LS1

# AFTER LS1

30 17 12 1


Linac2

• 2 systems to be renovated– BE/CO (1 FE)

• Wes->ELMA, Rio2->Rio3, Tg8->CTRx,Mil1553->CBMIA, OASIS, COMPAR->SIS

– EN/STI (1 FE)• Beamstoppers/Slits -> PLC based solution (EN/STI)

• Remarks– Low priority due to arrival of Linac4– Few FEs remain, but high concentration of old hardware

• BE/ABP: Source control ICAN -> PLC based solution– Responsibility with ABP (R. Scrivens)– Possible renovation end 2013

• TE/EPC: Power converters (MIL1553 – 1FEC)– Partial CO renovation BC1553 -> CBMIA

Linac2 MCC S. Jensen


9 1 2 (8*) 6* Partial CO renovations (Rio2->Rio3, Tg8->CTRx)


PSB

• 35 systems to be renovated– BE/CO: Wes->ELMA, Rio2->Rio3, Tg8->CTRx, Mil1553->CBMIA, GFAS->CVORB,

OASIS, COMPAR->SIS, AIOX, SPEM– BE/BI : BLM, BCT, BPM, Semgrids, Ring beam diagnostic, Tune measurement, Fast wirescanners, – BE/RF: New digital LLRF, Tomoscopes, Analogue control, Transverse feedback, Cavity control– TE/EPC : CAMAC->FGC– TE/MPE: Interlocks– EN/STI: Beamstoppers

• Remarks– Timing renovation requires CTRx/LTIM everywhere => Equipment groups must adapt– TE/ABT: No kicker renovation in LS1 (3 FECs) – SW/HW responsibility under ABT => CO stock OK– TE/MSC: B-TRAIN (measured & synthetic)

• No confirmed date for the new TE/MSC based solution– BI, RF: No concrete planning available other than “during LS1”– BE/BI: BLM location unknown, i.e. no planning/installation possible at present– Cabling by EN-EL: Determines CAMAC->FGC (TE/EPC) and injection semgrids (BE/BI)

PSB MCC S. Jensen


46 8 35 3/ABT (+7?*)* current RF crates, hot spares


CPS

• 33 systems to be renovated– BE/CO : upgrade crates + HW modules, SAMPLERS->OASIS, Timing [TG8>CTR], FunctionGenerator

[GFAS>CVORB], PC [BC1553>CBMIA]– BE/RF: cavity controls– BE/BI: BLM, BCT, SEMS stepping motors– TE/VSC : Vacuum– TE/ABT : Injection Ion Kicker KFA28 (1) & CT Extraction fast bumpers (CAMAC eradication)

• Remarks– BE/CO: not clear if resources are available with all planned upgrades

• some residual systems till TS 2014-15 or LS2 ? : old crates, HW modules, xenericSamplers, TG8, GFAS, BC1553– BE/BI: ring BLM renovation in LS2 if there is an issue with EN/EL cabling

• Constraint on BE/CO to support CAMAC beyond LS1– TE/MSC: B-TRAIN No confirmed date for the new TE/MSC based solution– TE/ATB:

• P+ Inj kicker KFA45 (1) renovation in LS2 (together with the LIU upgrade)– Constraint on Timing renovation

• P+ Ej kicker KFA71: renovation in TS 2014/15 – validation via Inj Kicker for ions– EN/STI: MTV positioning (1 crate): Low priority (PLC solution) -> LS1 planning under discussion

• Constraint on BE/CO to support CAMAC beyond LS1

CPS MCC M.Pace

# FRONT-ENDS # RENOVATED # DURING LS1

# AFTER LS1

78 30 33 5


SPS

• ~120 systems to be renovated– BE/CO : eradicate SLEQUIP old FE SW, upgrade OASIS & Timing– BI: BCT, BPM, North Area BI– TE/EPC: migration to MENA20– TE/MPE: migration to MENA20– BE/RF: all BA3 + BA2 systems– TE/ABT: all kickers (based on new FMC fine delay boards)

• Remarks– BE/BI

• SEMS IN/OUT (4 FEs) renovation during TS 2014/15 – MIL-1553 CO support required

• BLM and BPM (28 FEs) renovation during LS2 – LynxOS CO support until LS2

– BE/CO: All SPS FEs are in the process to be fully described in Controls configuration DB


220 ±70 (TBC) ±120 (TBC) ±30 (TBC)

SPS MCC M.Pace


AD

• 15 systems to be renovated– BE/BI : Orbit meas., DCCT, Fast transformers – TE/VSC : All vacuum controls (PLC, PVSS)– BE/RF : Stochastic cooling (PLC, IEPLC)– TE/EPC, BE/CO : Power converter controls (MIL1553, GFAS)– BE/CO

• New central timing, TG8 replaced by CTRx• OASIS VXI crates (VME and/or PCI).

• Remarks– Kichers (TE/ABT) and RF (BE/RF) :

• Renovation (5FEs) not confirmed for LS1

– TE/MSC B-TRAIN: • No confirmed date for the new TE/MSC based solution

AC MCC M.Cattin


31 10 15 6


ISOLDE & REX

• 6/8 systems to be renovated– EN/STI : Stepping-Motors and Slits (Collimators, Deflectors, Scanners, etc.)

• Should be done during LS1 (EN/STI std. PLC solution) -> Work-plan still to be defined– BE/CO + BE/OP + TE/EPC : REX-EBIS (Charge breeding system)

• 75% generic non-standard controls• EPC agreed to renovate – exact date & technical details to be finalised• Renovation should done in LS1. Exact schedule to be finalized

– Participation of BE/OP in 2012 in the design and spec of the new system (4-5 weeks) not possible

• Remarks– EN/STI + TE/EPC : HRS and GPS Magnet controls (GM class and obsolete FE)

• Could be renovated during LS1 under these conditions (*)

– HRS regulation process taken over by EN/STI– Technical decision to be taken for GPS regulation (HW obsolescence problem)– Responsibility limits to be clarified for Power equipment (STI & EPC)

– TE/ABT : High-Voltage converters (GM class and obsolete FE)• No renovation in LS1. ISOLDE is not the first priority – no impact on CO for support

# FRONT-ENDS # RENOVATED # DURING LS1 # AFTER LS122 13 6 (STI + EBIS)+2 (ISOLDE magnet)(*) 1 (TE/ABT)

REX-ISOLDE MCC F. Locci


Provisional Status after LS1

LINAC3

LEIR

LINAC2

PSB CPSSP

S

REX-ISOLD

E AD0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

FECs renovated after LS1FECs renovated during LS1FECs already renovated

IEFC, June 8th, 2012 – M.Vanden Eynden


LHC 1/3• LynxOS-PPC platform eradication

– > ~200 FEs• BE/BI : 125 RIO3 FEs + 17 FESA Classes• TE/ABT : 7 FEs + SPS (6 FEs)• BE/RF : 56 FEs• TE/MPE Interlocks : 24 to 48 FEs• BE/CO : 5 FEs

• Work involved– Retain the existing Wiener crates– Replace the CES PPC CPUs by Intel MEN A20– Re-compile and test all FESA software on Linux (32 or 64

bits)– No remaining LHC LynxOS systems


LHC 2/3

• Replacement of Elonex platform by Kontron PICMG1.3 – ~250 machines

• Power converters, Cryogenics, QPS, Beam Instrumentation, Beam Transfer,…

• Work involved– Equipment groups are currently validating their

applications on the Kontron platform (WorldFIP)– BE-CO will do the installation– FESA software re-qualification required by equipment

groups• Elonex already on Linux so no big issue

E.Hatziangeli - BE/CO 19

S12P1 S23P2 S34P3 S45P4 S56P5 S67P6 S78P7 S81P8Nov.

Dec.Jan.Feb.Mar.Apr.MayJun.Jul.Aug.Sep.Oct.Nov.

Dec.Jan.Feb.Mar.Apr.MayJun.

2013

2014

PT PTPT PT

PT PTPT PT

Cool-downCool-down

Cool-downCool-down

Cool-downCool-down

Cool-down

Powering Tests Powering

Tests Powering Tests Powering

Tests

Powering Tests


TestsPowering

Tests

Warm-up Warm-up Warm-up Warm-upWarm-up Warm-up

Warm-up Warm-up

SplicesSplices

SplicesSplices

SplicesSplices

SplicesSplices

Cool-down

E2K

E2K

E2K

E2K E2K

E2K

E2K

E2K

Powering Tests Splices

Warm-up Preparation for cool-down

Cool-down

R2E

BE/CO

Planning Elonex to Kontron


LHC 3/3

20

• Doubling of the QPS infrastructure– Detailed EDMS Doc Document 1223709 describing

the BE/CO work– Material to be provided by BE/CO

• 20 Gateways• 41 FIP PMC cards 1Mhz• 20 Timing cards• 66 Repeater Copper / Optical• 41 FIPDiag (or NanoFIPDiag)• 106 Connectors FIP (41 attenuators)• 41 TAPs Fip• ……

– BE/CO manpower needed• Installations SR crates + FEC : 8 days in pairs• Installations RE : 8 days in pairs• Installations tunnel, UA : 21 days in pairs• Qualification 41 new segments + 47 old segments: 44 days in

pairs


LS1 activity in CCRComplete renewal of the electricity & HVAC infrastructure• Impact - Electricity

– A few ½ days cuts in June 2013 plus reception tests during October 2013

• Impact – HVAC– The CCR ProLiants racks will have to be moved therefore will be unavailable

for ½ day one after the other, during April/May 2013

• Impact CCR– Both electricity and HVAC will run in a degraded mode during most of 2013

• Loss of some of the current redundancy• HVAC will provide temporary movable device to be used in case of failure

of the current HVAC installation

• Remarks– Work will be distributed over the 2013– Difficult work phasing to ensure the availability of the plant as long as possible– The dates of the ½ day cuts will be planned with more precision later in 2012,

in collaboration with OP, EN/CV and EN/EL– BE-CO will supervise the cabling activity– Intense transport activity: loading/unloading

* Aout-Sept 2012: Civil Engineering work: slabs, roof reinforcement

CO Coordinator P. Charrue

21


BE-CO in LS1 as OwnerJob Description Concerns

R2EMove of Timing, Remote reset,

WorldFip repeaters BE-CO, EN-EL

R2E Move of Power converters (1, 5, 7) BE-CO, TE-MPE (TBC*)

Radmon Additional devices BE-CO, EN-EL, TE-STI

Linac 4 FE installation BE-CO, EN-EL

FE maintenance Cleaning dirty crates, fans BE-CO

FE Wes to Elma Replacing old Wes by Elma crates BE-CO

…. … …

• To do before LS1 – During 2nd half 2012, monitor the front end temp., current and fan speed to check if preventive

maintenance must be done during LS1

• Cabling quality assurance– BE/CO will provide training video and QA procedures to assist EN/EL with the training of cablers– Qualification of WorldFip installations by BE/CO

E.Hatziangeli - BE/CO 23

S12P1 S23P2 S34P3 S45P4 S56P5 S67P6 S78P7 S81P8Nov.

Dec.Jan.Feb.Mar.Apr.MayJun.Jul.Aug.Sep.Oct.Nov.

Dec.Jan.Feb.Mar.Apr.MayJun.

2013

2014

Tentative Planning R2E

PT PTPT PT

PT PTPT PT

Cool-downCool-down

Cool-downCool-down

Cool-downCool-down

Cool-down


Tests Powering Tests Powering

Tests

Powering Tests


TestsPowering

Tests

Warm-up Warm-up Warm-up Warm-upWarm-up Warm-up

Warm-up Warm-up

SplicesSplices

SplicesSplices

SplicesSplices

SplicesSplices

R2E

R2E

R2E

Cool-down

Powering Tests Splices

Warm-up Preparation for cool-down

Cool-down

R2E

BE/CO

?

?


BE-CO in LS1 as Service Provider

• All control services should run until Jan. 21st 2013• Identify the necessary operational controls facilities and the required level of service (best effort,

core infrastructure, timing,..) … In progress• Consolidate network installation

– Replace 500 n/w switches (2/day) -> FSU support to IT/CS by BE-CO– Concerns all technical network in Meyrin, Prevessin, LHC– Interruption 1h/day

• Layout changes validation by the data management team

Machine/Equipment Description Concerns

CTF3 Controls operational BE-CO, TE-ABT

CCR HVAC upgrading To maintain service in CCR BE-CO, EN-EL, EN-CV

Linac2 Linac3 Controls infrastructure needed

Source Controls Upgrade BE-CO, BE-RF, BE-ABP

3Mev Test StandControls infrastructure needed

Source Controls Upgrade BE-CO, BE-RF, BE-ABP

Linac4 FEs commissioning – LS1-LS2 BE-CO, BE-RF

LHC LHC Powering tests BE-CO (LHC WorldFip Gateways)

SM18Operational support

(data base and data handling) BE-CO, TE-MSC

… … …

25

BE-CO in LS1 as HW & SW supplierJob Description Concerns

ELMA VMEBus crates (500 units) Production of 500 crates

BE-RF, BE-BI, TE-MPI, TE-ATB, EN-STI, BE-OP,IT-CS, EN-EL

HW modules Available in stock or to purchase BE-RF, BE-BI, TE-ATB,…

QPS (doubling infrastructure) WoldFIP HW components BE-CO, EN-EL, TE-MPE

Radmon WoldFIP HW components BE-CO, EN-EL, TE-STI

Diamon, CMW New version of C2MON, MW All accelerators

Data Management Services Upgrading services All accelerators

InCA, LSA Consolidation Injectors, LHC

CBCM Timing Renovation Injectors

• WorldFip additional agents for RADMON– Taps, repeaters, power socket, bus arbiters,

Need to know # of devices and location


Summary

26

• Planning for BE/CO has just started – BE/CO implicated as owner and provider for services and HW&SW– Critical paths, resources and constraints will be identified and realistic plan will be done– Not clear yet if enough BE/CO resources are available for all required upgrades

• Large control system renovation effort during LS1– ~90% of the Injectors FE systems will be renovated– Large upgrade of the LHC control system– Hardware procurements related to this renovation are in place (Kontron, ELMA, MEN, …)– The core control components (HW modules, device drivers, FESA classes, InCA,.. ) will be ready for LS1– Spares will be accumulated for the remaining non-renovated systems

• Collaboration with equipment groups – Agree on objectives, resources, responsibilities and documented in EDMS documents

• Issues– Getting all information in with firm dates commitment– Coordinate the availability of control facilities for (CTF, Linac3, 3Mev,..) with the renovation activities (CCR,

control system)– Linac4 equipment commissioning lowers work priority for Linac2– Cabling overload (EN/EL) will push some consolidation activities after LS1– Support of non renovated HW and SW must be properly organised with Eq. groups– Controls re-commissioning is MANDATORY for all accelerators (implication of MCCs, OP group, EQ groups)


Controls Activities & O rganisation during LS1

Documents

Transcript of Controls Activities & O rganisation during LS1