Controls Strategy- Lehman Review – Feb 2004 1 Controls Strategy Joel N.Butler Talk to Department...

Controls Strategy- Lehman Review – Feb 2004 1

Controls StrategyControls Strategy

Joel N.ButlerTalk to

Department of Energy Assessment of the Run II Luminosity Plan at the Fermilab Tevatron

Accelerator Integration SessionFeb. 24, 2004


What is the Concern?What is the Concern? The Control System works well but could work better

and contains obsolete components The character of the Run 2 upgrades puts a lot of pressure

on the system. We are also for the first time running Fixed Target and Collider simultaneously. The system must not only control and monitor the complex but must provide data acquisition and archiving, support data analysis and trending, provide more automated control and feedback, and support complex accelerator studies and calculations

It must support/facilitate development of new applications It must operate efficiently and be easy to maintain It must continue to do these tasks for many years

There are needs that are not optimally addressed by the current system, so process improvement is worth pursuing.We are undertaking a comprehensive review of the system


Goal of the Current ReviewGoal of the Current Review We want to understand the requirements

of the system and support departments in order to optimize the long term strategy

We aren’t trying to break things We aren’t trying to make extra work for

people We are very aware of the need to

maintain the ability to operate even while we undertake improvements.


ConstraintsConstraints No long shutdowns to replace large blocks of functionality

Precludes fundamental redesign of the system Parts of new system must be plug compatible and

communicate with old Must be able to easily switch new things in and back out

and/or run them in parallel No big pile of money to replace everything

Base on inexpensive commodity computing equipment Many small PCs/unix boxes rather than few large systems

No big influx of new people for a parallel effort Work must be done by existing lab personnel.

No long break to learn a new system New languages, tools etc. should have modest learning curve A lot of code written by non-Computing Professionals System department people are busy with accelerators


Control System OverviewControl System Overview

Applications

CentralServices

Front-Ends

Applications Applications

Front-Ends Front-Ends

CentralServices

…..

…..

…..

Field Hardware

ethernet

field bus

…..


Overview (VMS)Overview (VMS)

VAXes

Computer Room

Or elsewhere

The VAX 4090’s

are obsolete and

unsupported

Front Ends

(anywhere)………………

Datapool

Manager

Datapool

Manager………..

Applications Applications ………………

Front-End Front-End

Shared Memory

ACNET/Ethernet

Ethernet

X WindowsDisplay(PC)

X WindowsDisplay(PC)

…

Mature libraries and services to facilitate applications development. System is written in C (assembler) and relies heavily of VMS operating system

>500 “Console Applications”, those with user interfaces, written in C and FORTRAN. These do much of the work in support of operations


Overview (Java)Overview (Java)

Sun Netras

(computer room)

PCs

(anywhere)

Front Ends

(anywhere)

DAE DAE

………………

(16 consolidators)

DSE----------------OAC/Servlet

DSE----------------OAC/Servlet

………..

Application Web Application ………………

Front-End Front-End

RMI

ACNET

ACNET

http

This system could be a complete replacement on a more modern, operating system independent, platform. It is used for many non-interactive task (OACs), for SDA, Autotune for MiniBooNe and Ecool, and a few interactive applications.

From here, same protocoland database tables as VAXs


IssuesIssues VAXes

obsolete and constraining• Too little CPU power, memory, disk storage• There are many workarounds to solve problems

Have the most complete application support, but not easily portable

Familiar environment. AD people know how to program in C and FORTRAN. The applications are mostly portable if the new system’s infrastructure was made to “look like the VAX”.

JAVA Modern computers and programming language with

plenty of resources and easy expansion capability Incomplete support for applications Unfamiliar programming paradigm and language.

“Porting”in the usual sense of the word is not possible

Migration was assumed to involve a rewrite Because of the nature of Java, has some

performance and tuning issues as well


Approach: Controls Working GroupApproach: Controls Working Group We need to find out

What we are running now and what problems there are

What we would like to be running now if only we had the resources (especially programmers)

What new applications we need to develop in the future and what resources it will take to acquire them and run them to full advantage

What specialized and embedded systems we support, how they are supported and how they integrate into operations

What specialized systems and capabilities we imagine supporting in the futureWe set up a group consisting of two representatives from

each department and a few at large members to answer thesequestions – a.k.a the Controls Working Group (CWG).


Schedule of CWG TalksSchedule of CWG TalksOct. 29 Controls (Patrick) Booster(Kasper)

Nov. 5 Tevatron (Moore)

Nov. 12 Controls (Patrick)

Nov. 19 PBAR (Derwent) PBAR (Peterson)

Dec. 10 Controls (Hendricks_ Controls(Hendricks)

Dec. 17 Main Injector (Capista) Taking stock

Jan. 7 External Beams (Lucas) RF (Chase) ES&H (Zimmerman)

Jan. 14 Electrical (Martin) CR YO (Norris)

Jan. 21 Instrumentation (Voy) Labview devices(Lorman)

Inst/Introduction(Pordes)

Jan. 28 Recycler(Gattuso)

Feb. 4 ECOOL (Warner) SDA (Slaughter)

Feb. 11 Linac(Allen) Operations(Rohde)

Mar. 3 Operations (Kissel)


Future NeedsFuture Needs A major challenge for the Booster is to achieve

and MAINTAIN peak performance …

Day to day and even hour to hour scatter is large!

Maximum p/hr allowed by BLM’s

Protons/pulse


Monitoring ChangesMonitoring Changes Need to continuously monitor and record

changes in the machine’s behavior Complicated by the rapid cycling nature of the

Booster Would like to monitor hundreds of devices on

minute time scales but are limited by bandwidth issues

… lots of potential for improvement.. Can provide alarm capability for snapshot

devices Change analyses Auto tune


Key General ThemesKey General Themes MORE BANDWIDTH! MORE DATA STORAGE!

We are storing more and more stuff in the data-loggers

We would like it to stay around for a much longer time (years not months)

MORE AUTOMATION “It’s the 21st Century. Why not think more about

smart systems.”• Automated troubleshooting• Automated beam tuning

These, together with the desire for a more convenient andeasier application development environment were thekey general themes we heard from the CWG


Migration to JAVAMigration to JAVA We did a survey of VMS Console Applications:

Not counting obsolete or redundant programs Estimating what could be gained by application consolidation

either within departments or across departments We were pleased to learn that there was a fairly high degree of

consolidation, even across departments We have now established that there is a base of VMS console

applications that is greater than 500 programs Most people claimed that they did not have time to learn a new

programming language. Nor did they have time to “rewrite” their programs in this new language

There has not been an aggressive program to teach people JAVA nor were there good tutorials anchored in the actual tasks people were trying to do

There are pieces of the JAVA support infrastructure missing. Many needs are too pressing to wait for JAVA to catch up so even some new APPS are written for the VAX


The Emergence of a New ProposalThe Emergence of a New Proposal The Controls Group has proposed to attempt to

“port” the VAX/VMS CONSOLE INFRASTRUCTURE to a LINUX/INTEL platform.

Then, most of the Console Applications should simply port

The much more modern, powerful processors should solve a number of problems and applications can be extended in a familiar language.

This proposal makes sense ONLY if the Console Infrastructure can be rewritten QUICKLY The goal is for enough work to be done to port some

applications by the summer and To have most of the Console Applications migrated by

Oct 2005 These are just goals at present – planning/scheduling is

just beginning There will be a review of the proposal, including its

implementation, in March. A charge has been written.


What Happens to JAVA?What Happens to JAVA? The JAVA environment has proven to be very

useful and many modern Accelerator controls systems are based on it. In the end, it is believed to provide the best path to an easier application development environment

We will try to overcome some of the”acceptance” problems by Developing JAVA tutorials Producing examples of how to write common

types of applications Completing missing infrastructure Improving documentation Establishing a JAVA applications writer group


How will it come together?How will it come together?

The JAVA system already runs on LINUX. Then, we will have one platform that can run C and JAVA

Somewhat farther down the road, one could see that a single set of common services could support either C or JAVA-based applications

The Controls Department will be supporting at least two systems for some time. The Department will have to devote resources to develop the infrastructure and some of the applications support will have to revert to physicists and others This requires the Department to play more of a leadership role and less of a support role. They will have to guide people to use the most efficient and appropriate tools for their applications. This will require management and the Department to work together to get the correct result. Priorities and rules will need to be set and enforced.


Other Lessons from the CWGOther Lessons from the CWG Communication on controls issues is not great (for

many good reasons) Consequently, “best practices” are not uniformly

applied across the Division Knowledge of the capabilities and support available

from the current system is not good Needs and problems are not well-communicated Priorities and schedules are not always well-

established We will continue the CWG (perhaps biweekly) and

turn it into a forum for addressing controls issues across all departments. Examples are hardware initialization Save and Restore Writing of non-interactive tasks (OACS)


• The Control System is not broke, but can work better to serve Fermilab’s goals.

• We will work towards improving it • We hope that it will lead to easier operations and

maintenance, more consistency, more automation, and more efficient studies and in that way contribute to the Run 2 luminosity improvements.

ConclusionConclusion


ConclusionConclusion

The Control System is not broke, but can work better to serve Fermilab’s goals. We will work towards improving it. We hope that it will lead to easier operations and maintenance, more consistency, more automation, and more efficient studies and in that way contribute to the Run 2 luminosity improvements.

Controls Strategy- Lehman Review – Feb 2004 1 Controls Strategy Joel N.Butler Talk to Department...

Documents

Transcript of Controls Strategy- Lehman Review – Feb 2004 1 Controls Strategy Joel N.Butler Talk to Department...