December 2nd, 20209th Virtual MicroTCA Workshop for Industry and Research

Design and status of MTCA.4 based LLRF control system for the J-PARC MRYasuyuki Sugiyama, Fumihiko Tamura, Masahito Yoshii J-PARC Center

1

4

J-PARC (Japan Proton Accelerator Research Complex)

• High intensity Proton Accelerator with a LINAC and two synchrotrons.

• Started the beam commissioning from 2006 and achieved the high intensity proton delivery.

• LINAC: 400MeV negative hydrogen (H-) with 40mA (25 Hz)

• Rapid Cycle Synchrotron (RCS): 3 GeV proton with 1 MW (25 Hz).

• Main Synchrotron Ring (MR): 30 GeV proton with 500 kW (2.48 s cycle) for ν experiment.

Tokai

J-PARC facility

2

Joint Project between KEK and JAEA

LINAC(400MeV)

RCS(3GeV)

MR(30GeV) Hadron 51kW

Neutrino500kW

µ,n 1MW

Frf

• J-PARC MR has 8 bunches accelerated with h=9 in the ring.

• RF frequency for Acceleration: FRF=1.67~1.72MHz (Frev=185~190kHz)

• Synchrotron Frequency Fs:30Hz~300Hz

• 9 RF cavities in use: 7 (h=9), 2 (h=18)

Main Ring (MR)

Synchrotron frequencyFs

RF cavities

3

LLRF system for the J-PARC MR• J-PARC synchrotrons use the wide-band MA

loaded cavity systems.

• Beam loading compensation is necessary.

• Current LLRF system uses RF FeedForward method for the beam loading compensation.

• Not perfect enough at the beam power of 500kW and Coupled Bunch Instabilities (CBI) were observed.

• Current system with VME modules

• Use several racks connected with many cables…

• Modules with more than 10years old =Difficulty in the maintenance

• Need more functionalities for higher intensities and stability.

4

=> MTCA.4 as next platform!

Beam Longitudinal CB oscillation

• More powerful beam loading compensation

• Monitoring and Feedback for the CBI.

• Communication between main and sub LLRF system.

• Current 9 RF cavities are in the same straight section, will be used for acceleration (h=9)and will be controlled by main LLRF system.

• We are planning to add more RF cavities for h=9 and h=18 in the other straight sections.

• Separate LLRF system will be needed for additional cavities. It will need information from/to main LLRF.

Requirement for the new LLRF system

MR

Current RFFuture

Futur

e

5

New Main/Sub LLRF system Configuration• Main system:

MTCA.4 shelf with RF back plane

• Design based on J-PARC RCS LLRFpresented in the past MTCA workshop.

• AMC/RTM for each function

• Common Function module for Freq. management and beam FB.

• Cavity Driver module for cavity gap voltage FB control.

• Special MCH:High Speed Serial Communication module (HDS)

• Sub System:2U BOX with a pair of AMC and RTM for Cavity Driver

6

Cavity Driver

Common Func. Module

High SpeedSerial Com.

ModuleMain LLRF

Sub LLRF

System Component

7

• Developed by Mitsubishi Electric TOKKI System Co.,Ltd.

• AMC:

• 8 ADC and 2 DAC

• EPICS-IOC running on embedded Linux on Zynq.

• Clock, trigger distribution and high speed serial communication via MTCA backplane (Zone1).

• RTM:

• Used for signal transition from/to the RF cavities and the other accelerator control systems.

• Different design for Common Function module and Cavity Driver module.

されている。現在、前面のディジタル信号用のバッ

クプレーンとは別に、μRTM 用の背面からアクセ

スできる RF バックプレーンの開発が進められている。また、Euro-XFEL などで MTCA.4 を、LLRF などへ適用が進められている [11]。 例えば、LLRF のフィードバック制御システムの

場合、μRTM の基準信号発生モジュールに加速器

のリファレンス信号を入力して、PLL 等で LO 信号およびサンプリングクロック信号を生成する。これ

らを RF バックプレーン上の配線を使ってダウンコンバータμRTM へ伝送する。ダウンコンバータモ

ジュールではパネル面から入力される各ピックアッ

プからの RF 信号を IF 信号に変換して、Zone3 のコネクタ（ZD コネクタ）を介して伝送され前面のAMC に実装している A/D 変換回路でデジタイズされる。その後、同じ AMC 上の FPGA で信号処理をするもしくは、別の CPU-AMC で演算処理をしてD/A 変換したベースバンド IQ 信号を Zone3 経由でアップコンバータのμRTM へ入力しパネル面から

RF 信号を出力する。そして、アンプを経由してクライストロンをドライブする。

2.2 新しい A/D・D/Aボードの構成

私たちが開発した新しいボードを Figure 3 に示す。MTCA.4の広い RTMに RF回路を実装して利用するために、DESY が推奨している Zone3 のコネクタピンアサイン「ClassA1.1」を採用した。 私たちが以前開発した MTCA.4 規格準拠の FMC

キャリアボードと同じ SoC FPGA「Zynq」を採用して開発期間の短縮を図った。2つの FMCを実装できる機能をなくした代わりに A/D・D/A 変換 IC を直に実装した。FPGA 内蔵の CPU（ARM Cortex-A9）で使うワークメモリ（DDR3-SDRAM）は 1GiB であ

り、ブートメディアは、SD Card および QSPI Flash ROMを実装した。FPGA内のロジック回路から直接制御できるメモリ（DDR3-SDRAM）も 1GiB 実装した。前面パネルには、離れた場所のユニットと高速光通信できるように SFP モジュールを 2つ実装した。バックパネルに接続される AMC コネクタにはGigabit Ethernetと PCI Express×4用の高速シリアルインタフェイスを接続した。諸元を Table 1 に、機能ブロックを Figure 4に示す。

A/D・D/A ボードの単体性能評価ができるように、ダウンコンバータや帯域制限フィルタ機能のあるμRTM の代わりに、パネル面の同軸コネクタから RF信号を入力し平衡信号に変換後、Zone3 のコネクタに接続する延長ボードを準備した。

AMC

ZDｺﾈｸﾀ(Zone3)

A/DD/A

AMCｺﾈｸﾀ(Zone1)

μRTM

RFｺﾈｸﾀ

SFP

FPGA

Figure 3: MTCA.4 A/D, D/A Board Zone3 Class A1.1.

Table 1: Specifications of New Control Board

FPGA Zynq XC7Z045-1FFG900C

OS Xilinx Linux (EPICS-IOC)

RAM DDR3-SDRAM 1GiB×2 (PL, PS)

FPGA Configuration QSPI FLASH-ROM 16MiB, SD Card, Remote Update

ADC 8ch, 16bit, 370MSPS max., BW 800MHz

DAC 2ch, 16bit, 500MSPS max.

Zone1 (AMC Connector) Port[0:1]:1000BASE-BX, Port[4:7]: PCI Express Gen2

Port[17:20]:M-LVDS, IPMB: IPMI v1.5 support

Zone3 (ZD connector) Class A1.1(RFin×8ch,DCout×2ch,CLKin×1,DIO×6pair,TCLKout)

SFP 2ports

Switch 8bit DIP-switch

Front Panel LED Hot swap status (blue), Error status (red), Running status (green)

Size PCIMG MTCA.4 Double-Width Full Size 148.5*28.95*181.5 [mm]

Zone 1AMC connector

Zone 3ZD connector

RF I/O connector

Main LLRF system• Single MTCA.4 shelf for 9 RF cavities in use.

• Cavities for h=9 and neighbor harmonics.

• System Clock (144MHz) from eRTM is distributed via RF backplane.

• Common Function Module

• Frequency pattern management, Trigger/Clock distribution via MTCA backplane

• Beam Feedback

• Baseband demodulation for Phase FB,Sideband demodulation for Coupled Bunch FB.

•

8

MTCAbackplane

RFcavity

High-SpeedSerial

Communicationmodule (MCH)

Common Functionmodule RTM

Cavity Drivermodule RTM

Cavity Drivermodule AMC

Common Functionmodule AMC

RFbackplane

RFcavity

Cavity Drivermodule AMC

Port 17-20Clock

TriggerFrequency

Port 1High-Speed

SignalTranser

PM(RTM)

MCH

PM

eRTM(Generate

System Clock)

RFcavity Cavity Driver

module RTMRFcavity

144MHzClock

12MHzClock

BeamSignal

Trigger

AC100V

AC100V

1GbEthernet

RTM side AMC side

Common Function module

ADC BasebandDemod.

Phase FB

Cavity Gap Voltage IQfrom Cavity Driver

SingleSidebandDemod.

CoupledBunch

FB

BeamSignal

RFfrequency

Synchrotronfrequency

FeedbackPhase

CoupledBunch

FBSignal

Main LLRF system• Single MTCA.4 shelf for 9 RF cavities in use.

• Cavities for h=9 and neighbor harmonics.

• System Clock (144MHz) from eRTM is distributed via RF backplane.

• Common Function Module

• Frequency pattern management, Trigger/Clock distribution via MTCA backplane

• Beam Feedback

• Baseband demodulation for Phase FB,Sideband demodulation for Coupled Bunch FB.

• Cavity Driver Module

• Multi-harmonic Vector IQ FB for cavity voltage.

• Coupled-Bunch Feedback signal as a set point of the voltage feedback.

9

MTCAbackplane

RFcavity

High-SpeedSerial

Communicationmodule (MCH)

Common Functionmodule RTM

Cavity Drivermodule RTM

Cavity Drivermodule AMC

Common Functionmodule AMC

RFbackplane

RFcavity

Cavity Drivermodule AMC

Port 17-20Clock

TriggerFrequency

Port 1High-Speed

SignalTranser

PM(RTM)

MCH

PM

eRTM(Generate

System Clock)

RFcavity Cavity Driver

module RTMRFcavity

144MHzClock

12MHzClock

BeamSignal

Trigger

AC100V

AC100V

1GbEthernet

RTM side AMC side

Cavity Driver module

ADC BasebandIQ Demod.

Coupled Bunch FB signal fromCommon Function module

FB ModulateCavityGap 

Voltage

IQ pattern

SUM

revolutionfrequency

harmonicnumber

Feedback block foreach harmonic

component

DAC

RF signal

++

+

High-SpeedSerial

Communicationmodule

CommonFunctionmoduleCavity Voltage IQ

Feedback Phase

CoupledBunchFeedBack signal

Vector Sum of Cavity Voltage IQ

CavityDrivermodule

PhaseFeedback

CoupledBunchFeedback

Vector Sum &Normalization

BeamPickup

Beam Signal

RFcavity

Cavity GapVoltage 

• Use Port1 in MTCA backplane as a high-speed serial communication path.

• Xilinx Aurora as data format.

• Special MCH2 “High speed Serial communication module (HDS)” as a data accumulator and distributer.

• Cavity Driver module → HDS→Common Function module

• Cavity Voltage IQ signals are summed and normalized at HDS.

• Common Function module→HDS →Cavity Driver

• Phase FB signal

• CB feedback signal (sidebands of beam signal)

Data flow in Main LLRFhigh speed serial communication

FPG

Alo

gic

• There are no trivial star-like connections among AMCs

• Idea: putting FPGA logic in MCH2 slot and using Port1, although it sacri� ces redundancy of MCH

ESS-J-PARC workshop, Fumihiko Tamura MicroTCA.4 based LLRF control system of the J-PARC RCS: design and status 18

10

Sub LLRF system

Main LLRF systemHigh-Speed

SerialCommunication

module

Common Functionmodule

CavityVoltage IQ

Feedback Phase(fundamental)

CoupledBunchFeedBack signal

(fundamental)

Vector Sum of Cavity Voltage IQ(fundamental)Cavity

Drivermodule

PhaseFeedback

CoupledBunchFeedback

Vector Sum &Normalization

BeamPickup

BeamSignal

RFcavity

CavityGap

Voltage  SFP

SubLLRF(h=9)

SFP

CavityGap

Voltage 

Fundamental RF system 

RFcavity

CavityVoltage IQ

11

• 2U BOX with a pair of the cavity driver module (AMC/RTM) for additional cavities.

• Self Management of Clock/Trigger/Frequency.

• For beam FB information,use SFP ports on AMC as high-speed serial communication port.

• Sub system for Fundamental RF (h=9): Connect with HDS module.

• Used as the extension of MTCA backplane Port1

• Sub system for 2nd harmonic RF (h=18): Connect to Common Function module.

• Cavity voltage and Beam IQ signals of the second harmonic RF are processed in the Common Function module.

• 2U BOX with a pair of the cavity driver module (AMC/RTM) for additional cavities.

• Self Management of Clock/Trigger/Frequency.

• For beam FB information,use SFP ports on AMC as high-speed serial communication port.

• Sub system for Fundamental RF (h=9): Connect with HDS module.

• Used as the extension of MTCA backplane Port1

• Sub system for 2nd harmonic RF (h=18): Connect to Common Function module.

• Cavity voltage and Beam IQ signals of the second harmonic RF are processed in the Common Function module.

Main LLRF systemHigh-Speed

SerialCommunication

module

Common Functionmodule

CavityVoltage IQ

Feedback Phase(fundamental)

CoupledBunchFeedBack signal

(fundamental)

Vector Sum of Cavity Voltage IQ(fundamental)Cavity

Drivermodule

PhaseFeedback

CoupledBunchFeedback

Vector Sum &Normalization

BeamPickup

BeamSignal

RFcavity

CavityGap

Voltage  SFP

SubLLRF(h=9)

SubLLRF(h=18)

SFP

CavityGap

Voltage CoupledBunch

FeedBack signal(2nd harmonic)

Vector Sum &Normalization

FeedbackPhase(2nd

harmonic)

CavityGap

Voltage Fundamental RF system  2nd harmonic RF system 

RFcavity

RFcavity

CavityVoltage IQ

CavityVoltage IQ

12

Sub LLRF system

Performance Evaluation with prototype module.• With prototype cavity driver module,

we evaluated the performance of the beam loading compensation. • With current LLRF using FeedForward method,

wake voltage of 1~3kV remained in each cavity.• Cavity Voltage FB system suppressed wake voltage for h=8,10

down to less than 0.1kV

13

Detected Cavity Voltage(Single Cavity)@480kW

Effect of wake voltage suppression on the CB oscillation

• We installed the prototype cavity driver modules for all cavities.• With FB (h=8,10),

no significant CB oscillation.• We achieved the user operation with

500kW thanks to FB at Jan. 2020.

14

with FBCB Oscillation@480kW

with FF

• MTCA.4 based new LLRF system for the J-PARC MR is under development.

• Will be completed in JFY2020.

• Beam test with the prototype Cavity Driver module successfully demonstrated its performance to suppress the beam instability caused by the wake voltage at the RF cavities.

Conclusion

15