INTRODUCTION O Military Operations Research Society …faculty.nps.edu/gbrown/docs/MOR Vol 16...

INTRODUCTION

Oral Histories represent the recollec-tions and opinions of the person inter-viewed and not the official position

of MORS Omissions and errors in fact arecorrected when possible but every effort is madeto present the intervieweersquos own words

Dr Gerald G lsquolsquoJerryrsquorsquo Brown is currentlya Distinguished Professor of Operations Re-search and Executive Director of the Centerfor Infrastructure Defense at the Naval Post-graduate School (NPS) Monterey Califor-nia In 1976 NPS students elected him bestteacher in 1982 NPS faculty elected himbest researcher in 2005 he was elected anInstitute for Operations Research and theManagement Sciences (INFORMS) Fellowin 2008 he was elected to the National Acad-emy of Engineering (the first NPS facultymember to receive this honor) and in 2009the Secretary of the Navy awarded himthe Navy Distinguished Civilian ServiceMedal Dr Brown also won the INFORMSMilitary Applications Societyrsquos KoopmanPrize in 1990 the MORS Barchi Prize in2007 and the MORS Clayton ThomasAward in 2010 This interview was con-ducted in Jerryrsquos office at NPS on June 202011 Notes at the end identify Jerryrsquos cita-tions most of which can be downloadedfrom his NPS homepage httpfacultynpsedugbrown

MORS ORAL HISTORYInterview with Dr Gerald G BrownJune 20 2011Naval Postgraduate School (NPS) MontereyDr Kirk Yost and Dr Bob Sheldon FSInterviewers

Bob Sheldon Jerry let me first ask you togive us your parentsrsquo names

Jerry Brown My dadrsquos name is GeraldBrown Irsquom the second And my motherrsquosname was Ruth

Bob Sheldon Tell us about your parentsand how they might have influenced youWhat did your dad do for a living

Jerry Brown My dad worked at NorthAmerican Aviation He was one of the devel-opers of the F-86 F-100 A-5 XB-70 X-15and the original B-1 Early on I lived in ElSegundo California in a house on the beachthat later was razed to make way for LosAngeles International Airport (LAX) runway

25 Irsquove been involved with our military allmy life

I lived in Southern California for a whilebut then I was moved around quite a bit be-cause of family disruptions and so forth Ispent time in Idaho on a ranch in Ohio ona farm and at Fort Knox Kentucky

I went to high school for four years inVicenza Italy and received what I later ap-preciated to be a very very excellent educa-tion at the Vicenza American High Schoolpreparing me for university

Bob Sheldon Did you get to travel aroundItaly while you were there

Jerry Brown I traveled all over EuropeBob Sheldon You said you had some re-

ally good teachers were any of those inmath or science

Jerry Brown They were I had excellentteachers in mathematics and science andin English exposition I was really blessed

Kirk Yost Despite growing up in so manyplaces you decided to return to California

Jerry Brown I decided to come back andIrsquom currently living in Pebble Beach aboutas far from the ocean as Irsquove ever lived inCalifornia

I should admit about my California ed-ucation that I failed kindergarten in ElSegundo California and I assume Irsquom stillrecorded in El Segundo as being lsquolsquomentallychallengedrsquorsquo or whatever the currently ac-ceptable euphemism is

This had to do with a congenital prob-lem called trigger thumbs that physicallyinterfered with my speed manipulation ofblocks on an IQ test This was diagnosedand surgically corrected But in CaliforniaIrsquom still officially lsquolsquomentally challengedrsquorsquomdashjust ask my spouse

Kirk Yost How did you decide where togo to college from Vicenza

Jerry Brown I came back from Vicenzaduring my senior year so I could get a highschool diploma from a California schooland not have to explain to colleges whatwas going on

I graduated from Fullerton Union HighSchool in Southern California and had hadsuch a good preparation in Vicenza that Iwas allowed to enroll in the junior collegeeven though I was still in high school AndI applied to I think about six universities

I was accepted to a number of schoolsand I ended up going to the University ofCalifornia at Berkeley But that was not agood time to be in Berkeley with the Free

MilitaryOperationsResearchSociety(MORS)Oral HistoryProjectInterview ofDr GeraldG Brown

Bob Sheldon FS

Group W Incbsgroup-w-inccom

Kirk Yost

MITREkyostmitreorg

MILITARY OPSRESEARCH HERITAGEARTICLE

Military Operations Research V16 N4 2011 doi 1057111082598316457 Page 57Military Operations Research V16 N4 2011 doi 1057111082598316457 Page 57

Speech Movement and the riots and tear gasand so forth And not having a scholarship Ihad to work for a living

Between the demonstrations and workingat night I had had enough of that in about threequarters and transferred I ended up getting twocollege degrees from California State UniversityFullerton and then I went to UCLA for my PhD

Kirk Yost Both your bachelorrsquos and masterrsquosdegrees are from Cal State Fullerton

Jerry Brown They are And are you ready forthis My bachelorrsquos degree is in business market-ing (quantitative methods) my masterrsquos degreeis an MBA (quantitative methods) and my PhDis in management

Bob Sheldon I want to ask a question aboutperception Therersquos often a perception of two-way snobbery between engineering schools andbusiness schools Many folks in the engineeringschools assume that if somebodyrsquos too weak intheir mathematical rigor to survive the engineer-ing school they go to the business school And inthe business schools therersquos a perception that ifa personrsquos too weak in their personality skills tosurvive in the business environment they go tothe engineering school Given that you survivedquite well you excelled in business school andyou are quite rigorous mathematically whatrsquosyour comment on that

Jerry Brown It depends on the business schooland on the engineering school My recollectionis that if you could understand the Black-Scholesequation you were no slouch mathematically Ifyou talk to a mathematical economist or to some-one mathematical in finance and so forth theydonrsquot have much to apologize for in terms ofmathematical rigor You need to establish foun-dations in mathematics and English expositionEquipped with these skills you can do about any-thing you want The engineering and business op-tions you mention just emphasize one or the otherbut cannot (or should not) be attempted with seri-ous deficiencies in either area

Irsquom not a keen fan of undergraduate businessprograms I prefer to see MBA students whohave completed undergraduate studies in someconventional academic area

I donrsquot think therersquos any particular weak-ness on either side here If you choose a good uni-versity a good curriculum and if you seek rigorand follow accomplished scholars yoursquoll find it

Kirk Yost Although all of your degrees are inmanagement or business administration yourearly publications are on statistics Did you thinkyou were going to be a statistician And how didyou eventually turn to optimization

Jerry Brown One of my early mentors wasHerb Rutemiller and yoursquoll see his name as aco-author in some of those papers He was veryinfluential and showed me how to do researchand how to publish He was a very accomplishedstatistician I followed his lead and got exposedto the literature of operations research (OR)But I donrsquot regret the business degrees becauseI can read operating statements and speak thelanguage and thatrsquos been useful in consultingI had worked for some time at North AmericanAviation Beckman Instruments and HughesSatellites on computers and so I was lucky tobe an early computer guy when such were stillrare

I was at that time hired out as a general-pur-pose mathematical and scientific digital pro-grammer This was when there werenrsquot thatmany large-scale machines and before the ad-vent of the current numerical recipes and suchOn my bookshelf here Irsquove still got basic refer-ences like Abramowitz and Stegun with tablesof Bessel functions and the like It was computerexperience that opened up early research to mepaid my bills and also got me working on somedefense problems

I became interested in optimization whileworking with Rutemiller but also while work-ing at UCLA with Art Geoffrion Glenn Gravesand company And theyrsquore the ones who reallypersuaded me to move from descriptive statis-tics to prescriptive optimization

I became besotted with the leverage thatthis sort of modeling gave us and I was fortunateto be involved in some early practical applica-tions at large scale including one at RAND Cor-poration you may remember Kirk that helpedthe Air Force decide what bombs to buy everyyear for two decades (Brown et al 1994)

Kirk Yost Can you outline how you went fromCal State to UCLA and subsequently into theNavy

Jerry Brown When I got my masterrsquos degreeand was accepted into UCLA I was hired at CalState Fullerton as an assistant professor of quan-titative methods

MORS ORAL HISTORY PROJECT DR GERALD G BROWN

Page 58 Military Operations Research V16 N4 2011

Bob Sheldon What year was thatJerry Brown 1969 I taught basic OR and com-

puter classes and the computer classes wereattended by various people from the engineer-ing school and the sciences because we ownedthe biggest baddest machine on campus (anIBM 1620 with 20000 decimal digits of memory)I noticed a couple of excellent students with realshort haircuts compared to the fashion at thetime

I eventually learned that they came from anorganization called the Naval Security Group(NSG) They were taking my computer classesbecause I was teaching one of the few sequencesthat actually taught programming on a large-scale computer while also showing how to inte-grate mathematics to design and implementuseful systems

And a quarter after that there were aboutsix guys with nice haircuts In parallel to thatI was having exchanges with my draft boardand I was single My draft board was harassingme as was their due and one evening in myclass one of my short-haired students came upand introduced himself as a Navy Commanderand said lsquolsquoDo you think you could pass the ap-titude test for the Navyrsquorsquo

I guess I did okay on the aptitude test be-cause I ended up in basic training at San DiegoCalifornia to be issued as a freshly minted Sea-man Recruit E1 with a masterrsquos degree This wasnot too unusual because NSG was pretty fast com-pany and the typical commissioned officers andpetty officers had advanced degrees some ofthem with very impressive backgrounds

In 1969 I got my first top secret clearance(something I find amusing when I fill out myforms today because there canrsquot be anythingthey donrsquot know about me by now) and startedserving with a reserve unit of the NSG

Then I contacted my Navy detailer andsaid lsquolsquoHerersquos the deal Irsquove been accepted to UCLAfor their doctoral program Therersquos no guaran-tee of success but if you give me the time to tryto earn my doctorate theremdashI donrsquot want youto pay for it and Irsquoll go on inactive duty withno paymdashwhen I finish the degree I might be ofmore value to the Navy and at that time Irsquoll comeback on active dutyrsquorsquo

Some lieutenant commander in the Penta-gon concluded this was a good deal and made

it happen On graduation I was supposed to geta direct commission as a Lieutenant One thingled to another and the Navy got impatient fill-ing their quotas and said lsquolsquoCome join us nowyoursquore going to like Newport Rhode Island andyoursquore going to like Officer Candidate School(OCS)rsquorsquo

In fact I did like both For me it was a vaca-tion I was the oldest guy there at OCS And inJune 1973 I was commissioned as an Ensign inthe (regular) United States Navy I was expectingto ship out to Vietnam because of some otherbackground I had with French and earlier workas a diver doing insurance and salvage workand diving for abalone This meant I could readnavigation charts printed in French And so myorders were essentially for the job you saw inthe movie Apocalypse Now

Kirk Yost You were going to be a swift boatcommander

Jerry Brown Yes of a Vietnamese river boatdetachment My detailer really played this up aslsquolsquoa Lieutenantrsquos billetrsquorsquo

I put my modest affairs in order Then I gota call from a guy named Jack Borsting at NPSand he said lsquolsquoAccording to this IBM card yoursquovegot advanced graduate education Tell me aboutyourselfrsquorsquo I explained to him what I had doneand what I was interested in and he saidlsquolsquoWould you like to come to Monterey and bea military instructorrsquorsquo I said lsquolsquoWell let me thinkthat overrsquorsquo And as you know you have to findsomebody else to take your orders in a situationlike this Otherwise you just canrsquot step back

But it turned out there were 30 guys eager togo over and be heroes so they had no troublefilling my billet and I ended up back in my Cal-ifornia and in Monterey

Kirk Yost Were you done with your PhD atthat time or were you still working with the peo-ple at UCLA

Jerry Brown I was still working at UCLAKirk Yost You ended up going to Monterey

as a professor not having finished yet at UCLAHow did that work

Jerry Brown I arrived here as a newly com-missioned Ensign and I was made a militaryinstructor but because of my publicationsand other activities they gave me an academicappointment as an assistant professor Theydesperately needed my computer skills and so


Military Operations Research V16 N4 2011 Page 59Military Operations Research V16 N4 2011 Page 59

I was dual-appointed to computer science andOR and given a double teaching loadmdasha full-time load in each department

I had a pilotrsquos license That put me within twohours of UCLA so I commuted down to UCLAon Friday and came back Sunday night After ayear of this people realized the sorts of research Iwas doing were of interest to the Navy and theyactually cut me some slack and gave memdashwhichthey were supposed to do anywaymdasha quarteroff I was able to finish up the degree

Bob Sheldon What was your dissertation onJerry Brown My dissertation was on nonlin-

ear programming used to find maximum likeli-hood estimates for very difficult distributionssuch as the three-parameter Weibull The moti-vation was exigent statistical problems but thebase was developing some rather abstruse non-linear programming theory

Kirk Yost How long were you in uniform atNPS

Jerry Brown Three years During those yearsI was promoted to associate professor as an En-sign then to Lieutenant Junior Grade (LTJG)

Kirk Yost Were you the only Ensign to everbe an academic associate professor at NPS

Jerry Brown I think so My computer sciencecolleague Gary Kildall was a full Lieutenant Iwas also dead last on the NPS accession listfor 36 months In other words for me to have as-sumed command here everyone else wouldrsquovehad to be gone

Bob Sheldon Letrsquos back up to your PhD pro-gram Any other notable professors that you re-call from your academic studies

Jerry Brown Too many to name These wereheady times at UCLA in my departments ofmathematics computer science and my homeschool of management I also met some very im-pressive and famous (just ask them) medical re-searchers One of the ways I paid my bills wasconsulting with the medical school A medicalschool as you can imagine has a number ofsmall-sample statistical puzzles and their re-searchers would show up and put a small sam-ple on my desk and say lsquolsquoI need to write apaper about this What can you prove from thisdatarsquorsquo

Instead of trying to school them on statisticalpropriety I scoured the literature and developeda library of the best small-sample tools I could

conjure This was quite remunerative it wasgood pay for not too much work

I recall a case where I had to tell a physicianresearcher lsquolsquoThis is just too small a sample sizersquorsquoAnd he replied lsquolsquoOh Irsquoll go get more datarsquorsquoWhen he came back later I finally thought toask this guy lsquolsquoWhatrsquos this data about Whatare you measuringrsquorsquo

He described a painful procedure that wasbeing inflicted on unknowing undergraduatecontrols who were patients at the UCLA medi-cal facility I immediately declared lsquolsquoWersquove gotplenty of data What do you want to proversquorsquo

I guess the most impressive guy who toler-ated me at UCLA was Jacob Marschak He wasa Russian polymath He was just brilliant Hewasnrsquot an optimizer but he was just so doggonesmart that yoursquod spend half an hour with himover a cup of coffee and hersquod end by askingsome question Yoursquod think that over and itwould occur to you about a week later whathe meant Sometimes it would take you a monthto prepare for the next cup of coffee but youdidnrsquot want to go back into his office againand be ignorant of what his direction had been

I am very grateful for having had his ac-quaintance and it was a great honor a coupleof years ago when I was invited back to UCLAto give the recurring lsquolsquoJacob Marschak Interdis-ciplinary Colloquium on Mathematics in theBehavior Sciencesrsquorsquo (httpwwwandersonuclaedux1094xml) with an audience coming fromall over Southern California

Bob Sheldon Could you comment a bit moreabout your early work in statistics

Jerry Brown The initial work we did was infixed sample acceptance testing (Brown andRutemiller 1971) and in reliability testing thatwas required by the US military to monitor pro-curement quality (Brown and Rutemiller 19731974 1975) Civilian entities also use these mili-tary quality controls There are military specifica-tions for acceptance tests where you have a lot ofa given number of items and you sample fromthe lot and subject that sample to testing andbased on the number of successes and failureseither accept the entire lot or not

My colleagues and I agreed these were ade-quate tests statistically (I believe Jerry Liebermanat Stanford at the time was one of the most in-fluential proponents) But we felt the tests



could be unnecessarily expensive in the follow-ing sense

If you take a sample and you begin testingit and your results are inordinately successfulyour instincts tell you that you could probablystop testing save a lot of money and make anearly conclusion that things are all right Con-versely if you have early results that are veryvery bad it might be cheaper to just stop andback up to try to find the cause

We did some early work in revising thesemilitary specifications to do sequential samplingso that we could get better decisions earlierat less cost That was my early involvement instatistics

Kirk Yost Where did your extraordinarilyhigh level of entrepreneurialism come from

Jerry Brown I donrsquot like being hungry Anditrsquos part of my background with my familywho were products of the Depression that Irsquovenever accepted money from anyone for any-thing I never took a dime in scholarship or sup-port of any type never had a fellowship andnever accepted any tuition assistance from theNavy And I graduated debt-free with each ofmy degrees I worked in parallel and paid mybills as they came up

Bob Sheldon What were the first coursesthey assigned you to teach here at NPS

Jerry Brown Basic computer programmingfor the masses Lots of that Databases and digi-tal simulation

Bob Sheldon Was that in FortranJerry Brown Yes some of it was in Fortran

We had a couple of other teaching languagesand special languages for artificial intelligenceAPL graphics list processing simulation andso on but Fortran was in widest use It was onpunch cards We had civil service civilians whotypically had masterrsquos degrees in mathematicsas daytime duty consultants in our computercenter But at that time with punch cards andbatch processing the only way you could getyour work done was to come in at night Thestudents would go home for dinner and thencome back and spend the better part of the nighthere and if you were their instructor it was agood idea to do the same

I ended up spending virtually every nighthere with students Of course if students knowyoursquore here and they know you know how things

work then they come by and ask you questionseven if theyrsquore not your students I ended upworking on a lot of things like orbital dynamicsand engineering acoustics and learned a greatdeal along the way

Scott Redd recently reminisced as our NPSgraduation speaker about these night shifts withme and we laughed with the recollection that Iwas neither his advisor (Al Washburn was) norsecond reader I was just there And I learnedfrom working with Scott and still learn workingwith Al

Kirk Yost One of your first breakthroughsin the OR community was the network optimi-zation paper you wrote with Gordon Bradleyand Glenn Graves What led you into that area

Jerry Brown That actually started on the highside for me I consulted for the Joint Strategic Tar-geting Planning Staff Omaha My early classifiedwork was on planning the Strategic IntegratedOperations Plan the SIOP replacing pins andstrings with optimization You can see how thatleads to the necessity for large-scale network op-timization given the large number of weaponswe had at the time I worked with another agencyin the Pentagon reckoning the Red SIOP orRISOP

On the unclassified civilian side GlennGraves had a consulting contract with GeneralMotors to determine how to distribute automo-biles to dealerships and customers And thecombination of those two challenges plus theinteresting nature of the problem led GordonBradley and me to spend quite a bit of time andeventually publish (Bradley et al 1977) We wereoffended as scholars that a competing networksolver being published at that time in our openliterature was being sold as a proprietary productby the authors We gave away our superior prod-uct for free and undermined the market

Kirk Yost Putting the GNET code in the pub-lic domain eliminated most of the competition

Jerry Brown No not eliminated We werenrsquotfielding salesmen We changed the market Thesecompetitors were using our open scholarly lit-erature as commercial advertising And thesweet part here was our algorithm is actuallymuch more efficient To this day despite a lotof advances in network optimization and eventhough our network solver is a network simplexalgorithm with a bad theoretical worst-case



runtime in live benchmarks against live dataitrsquos still the fastest code

Kirk Yost Was Gordon Bradley at NPS whenyou arrived and was he already working in thenetworks area

Jerry Brown Gordon and I arrived on thesame day in 1973 He had been a tenured asso-ciate professor at Yale He had taught optimiza-tion doubtless including networks but I thinkthe two of us launched off on our network initia-tive at the same time

Kirk Yost Although from radically differentbackgrounds I would suspect

Jerry Brown Thatrsquos true and the good for-tune for me was that Gordon had done a postdoc-toral fellowship at Stanford with George Dantzigand that opened up another set of doors for meGordon was very gracious to introduce me toGeorge and Phil Wolfe and other people he knewand who I had not encountered during my oddcareer

Kirk Yost Up until the 1970s the optimiza-tion community seemed to be divided betweentheoreticians and implementers You and Gordonseemed to be at the forefront of people whoworked on both the theory and the coding aspectsof these problems Can you comment on that

Jerry Brown Going back to that era and look-ing at the literature you wouldnrsquot see much thatyou would recognize today as an algorithm Pro-cedures were described in a rather hand-wavingimprecise way because we just hadnrsquot developedour way of thinking about such things Theoremswere well-defined but algorithms not so muchHowever those early literature articles are beau-tiful to read If you go back to the earliest issuesof Operations Research and Management Scienceyou will find some lovely military OR really wellthought out and eloquently expressed

In that era there were a lot of professors whowere well-trained mathematically (recall thatour OR discipline is a descendant of mathemati-cians and physicists in World War II) who lookeddown on those of us who dirtied our hands do-ing real computer implementation But we alsohad a parallel discipline in computer science thatwas just sorting out things like data structuresand algorithms The academics who kept tomathematics vigorously defended their theoret-ical journals from mere applications Inevitablythose of us fortunate enough to have a foot in

OR computer science and experience with cut-ting-edge applications developed new theory

One of the offshoots of this for Gordon andme was that we were two of the three foundersof what is today the INFORMS Computing Soci-ety (ICS) We founded the Computer ScienceSpecial Interest Group and Gordon and I servedas two of the three first presidents The early at-tendance of our fledgling interest group meet-ings was helped by me smuggling in cheesecrackers and wine I was told by the poobahsat the Operations Research Society of Americaand the Institute for Management Science atthe time lsquolsquoYou canrsquot do thatrsquorsquo This evidently vi-olates contracts with meeting hotels and theirunions Well I did it anyway and guess whosemeetings were standing room only To this dayone of the traditions of ICS and now of otherINFORMS special interests groups is an infor-mal cheese crackers and wine meeting

Kirk Yost Can you talk about one of yourmajor philosophies the notion of elastic pro-gramming Itrsquos central to much of your workbut rarely addressed in the mainstream optimi-zation literature

Jerry Brown Some contemporary textbooksnow mention elastic programming I credit theoriginal idea to Glenn Graves I was just quickto grasp its charm We were building a large-scale optimization system from the ground upat the time and we developed theory and algo-rithms with the elastic feature intrinsic We weredissatisfied with the commercial products thenand thought we had some better ideas One ofthe difficulties we had was with some standardbenchmark problems rogue problems that hadbeen developed precisely because theyrsquore so per-nicious We were trying to find ways of solvingthem much faster than the competition And itturns out that if you can relax constraints you donrsquotlike at least temporarily this is a good thing to do

One thing led to another and we began tothink lsquolsquoYou know this elastic business with lin-ear penalties is equivalent to bounding the dualvariables so that more fully defines the modelYou state the model you specify the constraintson your courses of action and along with eachconstraint you specify exactly how importantthis constraint is to you You specify how muchat most yoursquore willing to spend to satisfy thisconstraint That had a rather compelling ring to



it and when we looked further it turns out that ifyou implement an algorithm that incorporateselasticity as a fundamental intrinsic functionyou get some very elegant results and a very ef-ficient algorithm

Kirk Yost Are you the only practitioner thathas written a code that incorporates thosemethods

Jerry Brown I donrsquot know for sure but I sus-pect the root node integer enumeration roundingin CPLEX uses crude penalties And certainlymany people write elastic models but theyrsquoresolving them with traditional codes that treatthe elastic variables as explicit logicalsmdashslacksartificial and surplusesmdashand this is not as effi-cient as it could be

Kirk Yost Yoursquore the only professor Irsquoveheard who not only talked about the notion ofelasticity but talked about it as a fundamentalpart of an optimization problem

Jerry Brown Itrsquos absolutely fundamental Iwas told by academics early on that elastic con-straints lsquolsquocheatrsquorsquo But a manager policy makeror a general officer understands immediatelywhat elastic constraints mean They can controlwhatrsquos going on in a way they understand

If you like you can use conventional model-ing and declare lsquolsquoall my constraints are immuta-ble and infinitely importantrsquorsquo Good luck withthat in the real world and especially in the De-partment of Defense (DoD) where objectivesand constraints are rather fungible and wheremere whims by senior policy types become hardconstraints for junior analysts

Kirk Yost Another central idea yoursquove intro-duced is the notion of persistence in optimiza-tion Do you feel that yoursquove made headwayin the community with those ideas

Jerry Brown I think in most cases such fea-tures arise because if a model without any per-sistence feature gets used repeatedly say overtime itrsquos pretty hard to brief a solution that hasamplified some inconsequential data changeinto a wholesale revision of plan some of whichmay have already been promulgated (Brownet al 1996) When I find persistence features ina model this is a telltale that the model has ac-tually been used and is not merely some math-ematical confection

As you know Kirk any model ignorant ofits own past advice is really an ignorant model

And yoursquore not going to be able to use an opti-mization model very long in reality if the modelhas no feature to recall and heed decisions thathave already been advised and advertised Thatidea is not yet in textbooks and thatrsquos too bad(Brown et al 1997)

Kirk Yost Can you talk about your involve-ment with the Karmarkar algorithm for linearprogramming Its introduction and the sub-sequent efforts to control it as a proprietarymethod were very controversial

Jerry Brown When we first saw Khachianrsquosalgorithm Al Washburn and I took a look at itcomputationally and found it to be interestingbut not very efficient (Brown and Washburn1980) Certainly the theoretical resultmdashthe poly-nomial worst-case bound on the number of iter-ations to solve a linear programmdashwas valid butnot efficiently implementable Karmarkarrsquos algo-rithm was potentially more efficient althoughthere are a couple of missing steps in terms oftransitions from the interior points to what wecall basic solutions

My initial concerns with the Karmarkar re-sults were twofold

One was that our open academic literaturewas being used (here we go again) to promoteand sell a commercial product and presumingto publish papers about algorithms that werepatented trade secrets That is they successfullypublished results without showing how the re-sults were obtained This is not science They alsocreated a custom-design supercomputer to runthis algorithm and were trying to sell it to majorcompanies in the United States to solve planningproblems I believe Delta Airlines bought one

We were at the same time solving the samecrew scheduling problems for another largerUS airline with our own algorithm These prob-lems are not linear programs but rather integerlinear ones Lacking an integer feature somehowyou have to deal with fractional crew assign-ments You canrsquot assign half a pilot here and a thirdof a flight attendant there yoursquove got to assignwhole people The Karmarkar implementationhad no integer procedure at all so I was at thetime wondering what Delta Airlines was doing

I believe this was a commercial disaster forthe proponents I donrsquot think they sold morethan a handful of these and they only sold thoseto people who were rather innocent of what was



being inflicted on them Another thing that dis-turbed me was a presentation by Karmarkar atStanford hosted by George Dantzig A bunch ofnumerical results were displayed purporting tocompare the new algorithm against IBMrsquos MPS360 at that time a well-regarded commercial-quality optimizer Apparently no one else in theaudience knew MPS 360 had a limit on the num-ber of model constraints The reported results farexceeded that limit and therefore were concocted

Kirk Yost Did that eventually get exposedJerry Brown I exposed it only by asking a

question from the audience but I donrsquot recallthat anybody ever retracted a paper or publisheda correction or explanation Itrsquos too bad these in-terior point methods got off to such a poor startOthers have independently developed the the-ory and implementations since and mated thesewith conventional simplicial optimization Forsome problems this works well

Kirk Yost Was there any substantive changein the community with respect to dividing sci-entific discovery and marketing products

Jerry Brown A few journal editors steppedup but generally the Operations Research Soci-ety of America and The Institute of ManagementSciences today merged as INFORMS are prettypassive in that regard Despite a case I made asa plenary address before an annual meeting ofINFORMS and another plenary address by SethBonder with the same subject INFORMS stillhasnrsquot even defined what OR is as a professionThere are no standards Anybody can hangout a shingle And so theyrsquove been rather pas-sive and ineffectual at fencing off behaviors thatyou would consider unprofessional We havenrsquotdefined what the profession is

By contrast the uniformed military servicesdo have educational skill degree and experi-ence requirements for OR billetsmdashwe shouldbe proud of this

Kirk Yost On a different subject can you talkabout why you chose to stay at NPS as a profes-sor once you left the active-duty Navy

Jerry Brown I thought yoursquod never ask Irsquovedelivered seminars at many universities workedwith their students and remotely advised thesesand dissertations Therersquos nothing like teachingat NPS

For starters our students are paid full sal-aries with their sole duty to be our students

and to graduate During tenure here studentsget to catch their breath during a military careerNothing the student does here will appear ina service record or on a fitness report other thanlsquolsquoattended and graduatedrsquorsquo Imagine that Manystudents who were lackluster undergraduatesreturn to our graduate program after some timeand experience in uniform having learned howto allocate time effort and attention and abso-lutely bloom as analysts

I walk into classes on Tuesday which is uni-form day here and the one day a week that thestudents donrsquot wear just business casual attire Iadmire their decorations and qualification in-signia and ask myself lsquolsquoWhere do we find peo-ple like this Where do we find people who dothe things these young people do so willinglyably and even heroicallyrsquorsquo

Itrsquos humbling My students may not haveever noticed but out of respect my uniform onTuesday includes a tie and I always begin bycomplimenting them on their sharp appear-ance and thanking them for their service andfor making me proud

I think of my thesis student CPT Tom Whitethen already having earned two Silver Starswhose thesis led to the redesign of our main bat-tle tank CDR Mike Mullen [later Admiral andChairman of the Joint Chiefs of Staff] (who stillcalls me lsquolsquoEnsign Jerryrsquorsquo) a section leader whosethesis under the Navyrsquos preeminent tacticianWayne Hughes presaged the employment ofAEGIS combatant ships with new-generationphased array radar and interceptor missilesLCDR Steve Tisdale who completed two com-pletely independent degrees in OR and spacesystems and developed a space junk trackingalgorithm still in use today and Scott Reddwho retired as Vice Admiral and then directedthe formation of our National CounterterrorismCenter The list goes on and on and there areechelons of more junior officers rising I havebeen pleased and proud to see their accomplish-ments both in uniform and after

I also have to express my admiration for ourinternational students Although we try our bestto be good hosts I canrsquot imagine how hard it is tomove a family to Monterey get established andculturally aligned while at once engaged ina graduate study program that assumes the stu-dent is available full-time without qualification



My spouse volunteers teaching English tointernational student spouses and family mem-bers as part of a very important program sup-ported by NPS and our local school districtThis course involves daily mixing of all interna-tionals with a master teacher and qualified vol-unteers This cultural exchange in the long termmay prove as valuable as the academic achieve-ments of the international students Our interna-tional students come from professional upperclasses of their home countries and the spouses in-clude very accomplished professionalsmdashdoctorslawyers architects engineers and so onmdashwhoare not allowed to practice their professions inthe United States while their spouses attendNPS (This is by the way a nutty US policy)

Wersquore spoiled by the fact that when we givehomework to our students itrsquos considered or-ders And they respond in kind You have to bevery careful If you give a bogus homework as-signment at the end of a week you may findout later the students spent all weekend tryingto complete it

So NPS is a great place to be Therersquos noth-ing like it anywhere else I wouldnrsquot trade mymasterrsquos students for PhD students at any uni-versity anywhere

The pay is better elsewhere but wersquove gotall the computers and all the toys you can imag-ine and if we come up with some idea involv-ing blowing something up firing some roundsshooting a missile dropping some bombs orsomething less kinetic but no less interestingwe have the means to get such experimentsaccomplished

Kirk Yost Have you ever been tempted toleave and assume another position

Jerry Brown There have been a number ofoccasions including recently when Irsquove receivedunsolicited offers significant enough that I had totake them up with my spouse To her credit shehas advised lsquolsquoYoursquore happy at NPS Donrsquot worryabout itrsquorsquo

Kirk Yost Can you talk about the commer-cial consulting you do and how that compli-ments your duties at NPS

Jerry Brown NPS is a military school butadministered by scholars The distinction hereis key NPS wants me to know everything I needto know within DoD at all levels of classifica-tion and NPS also wants me to know whatrsquos

going on in civilian industry They want me toknow whatrsquos going on in the United States andinternationally They want me to be ready whencalled to be able to advise on and with the globalstate-of-the-art

NPS encourages us to do commercial con-sulting on a not-to-interfere basis We have to filepaperwork with the Judge Advocate Generaland the work canrsquot involve any client who doesany business with the federal governmentwhich rules out a lot of organizations but it hasbeen a way for us to find out in the private sectorwhatrsquos going on with a good portion of the For-tune 50 if not the Fortune 500

Kirk Yost Many senior people in DoD be-lieve that the commercial sector has better ideasand the DoD should be employing them Givenyour significant experience in that world whatis your opinion

Jerry Brown I think the analysts and profes-sionals I deal with in DoD including the deci-sion makers those analysts support are equalto anything that you would expect to find inthe private sector if not better Irsquove never founda more admirable or harder-working cohort ofprofessionals

Of course there are exceptions in allorganizations

I have to refer to Carl Buildersrsquo great bookThe Army in the Strategic Planning Process WhoShall Bell the Cat Builder hilariously adviseswith deadly accuracy that when it comes toOR lsquolsquoGod created the Navy and all else fol-lowsrsquorsquo Our Air Force (Brown et al 2003) Army(Brown et al 1991) and Marines (Bausch et al1991) embrace OR and use it well but I admitmy Navy is well not as willing a client as Iwould wish

We have had some successes but the Navyratio of success per attempt is not as high as wewish Much Navy OR emphasis is on programplanning because our OR degree sponsor isOPNAV N81 Assessment Division Howevereven though I always advise following the moneymilitary OR is about a lot more than just programplanning (Brown et al 2004 2005 2007 Brownand Carlyle 2008 Newman et al 2011)

NPS is a joint institution and this is a goodthing for NPS OR for DoD OR and for DoD

Kirk Yost Do you think that there are effectivecommercial OR methods that DoD isnrsquot using



Jerry Brown No I donrsquot In fact there aresome fashionable things in industry Irsquom gladDoD is not using for instance Enterprise Re-source Planning (Brown 2003) ERP has madesome modest inroads into DoD but the cost ofthese systems is just enormous and for a coupleof applications I have seen that will remain name-less the legacy software was better than the ERPthat replaced it This is a situation where seniorofficers and senior executives make decisionstoo expensive to fail and theyrsquore not aroundwhen the implications follow

Kirk Yost You donrsquot think itrsquos true that pri-vate industry is quantitatively much smarterthan the DoD

Jerry Brown No I donrsquot No private enter-prise is planning at anywhere near the scalethe potential consequences the long planninghorizon or the myriad exigent scenarios weare duty-bound to deal with in DoD Even ourlimited NPS OR contributions have been flat-tered by an external review that assessed ouradvice to have influenced more than a trilliondollars of defense investment

Whether or not we always have the influ-ence we seek at the right levels of policy withinDoD it is structured and organized and we un-derstand which levers to pull So if people askthe right questions and we come up with theanswers we can at least make a pitch

I have always felt even as an Ensign that Ihave had advantaged access and audience any-where in DoD I have on occasion exercised thatleverage and gotten myself invited to talk topeople when I thought there were emergentproblems worthy of our analysis and to whichwe could contribute Irsquove always been grantedan audience Every time Sometimes itrsquos been in-fluential and sometimes not

Unlike civilian corporate bureaucraciesDoD is much more deeply layered with levelsof authority But setting aside whether this or-ganization depth is necessary I only care if itis effective In my experience it is

When you know yoursquore right never give upBob Sheldon Jack Borsting recruited you here

and Irsquove done an oral history interview with himHersquos noted for being one of the founders of themodern OR curriculum at NPS Do you haveany comments on the formative years of the ORcurriculum here

Jerry Brown I was a latecomer Current Pro-fessors Washburn Gaver and Schrady predateme Jack Borsting at that time built a large orga-nization that was the combined OR and Admin-istrative Sciences Department Think of this asa combined military business school and OR or-ganization I forget how many mailboxes therewere but it was a lot of people

Jackrsquos a remarkable guy in the sense that ourorganization chart was completely flat We hadthe entire facultymdashand we had Jack Jack was(and still is) very good at making you feel likeyou have a valued opinion but as he always ad-vised lsquolsquoYou all get to vote But I get to count thevotesrsquorsquo

I would credit Jack with the formation of thedepartment He cultivated the connections heneeded He served in executive positions profes-sionally had a good nose for talent and workedthe phone tirelessly If he could find some ob-scure Ensign in Newport Rhode Island he couldferret out talent at Johns Hopkins or GeorgiaTech He was really remarkable in that respectSince Jack Irsquove worked for other chairmen Iguess a total of eight and wersquove been fortunateto have a deep bench and really good leadershiphere through some tough times

The key thing about working here is thatIrsquom absolutely shielded from the normal politicsthat is a preoccupation and distraction at otheruniversities I can stay in my office do my workwork with my students work on their theseswork on research projects and I donrsquot have toworry about any politics at all Well except oc-casionally when we are threatened with a BaseRealignment and Closure action and are askedlsquolsquoWhat have you done for us latelyrsquorsquo Thatrsquos aneasy question to answer but you never knowif your answer carries any weight in the politicalmilieu of that epoch

Bob Sheldon In your career yoursquove avoidedpositions such as department head dean andso on Yet you have given considerable supportto professional societies Can you talk about that

Jerry Brown My career is distinguished inthat I have never had a major administrativeposition of any kind and I hope to completemy career that way With INFORMS (then theOperations Research Society of America) myonly contribution work was helping set up thecomputer science interest group and an early



publication that started as a newsletter and isnow one of their flagship journals

Irsquove done a fair amount of editorial work forINFORMS Risk Analysis and the Military Oper-ations Research (MOR) journal Irsquove served ona number of committees For instance I re-cently chaired a committee to choose a new ed-itor for the journal Management Science Irsquoveserved for a three-year cycle and chair for a yearof the INFORMS Fellows selection committee Iserve on the editorial board for the MOR jour-nal I lack administrative ambition I did chairthe OR PhD committee here for 20 years andhave been our associate chair for research Icanrsquot think of much else Irsquove done besides men-tor junior faculty advise students and do re-search I could let the National Academy ofEngineering (NAE) become another unpaidfull-time job Unfortunately NPS doesnrsquot haveendowed chairs like other major universitiesso NAE work is lsquolsquoadditional dutyrsquorsquo

Irsquom currently serving on a National ResearchCouncil (NRC) Army board on explosives andsurvivability and Irsquom on the NRC Board ofMathematical Sciences and their Applications(BMSA) that sets the agenda in these fields onwhat studies will be conducted I review reportsfor the academies and have the advantage of fa-cilities to review classified reports without hav-ing to travel to Washington

The payback is access via the academiesrsquolegislative affairs office to policymakers This istwo-way access and we get calls from them forexample the Government Accounting Office andcongressional staffers with technical questions

Kirk Yost Does your future include writinga textbook or at least collaborating on one

Jerry Brown I donrsquot think so Irsquom having toomuch fun doing research The sorts of workwersquore doing involves groups sometimes largegroups of people Wersquore trying to write seminalpapers that introduce these new things suchas attacker-defender (or defender-attacker so-called bi-level optimization) models For in-stance the Bastion paper appearing elsewherein this issue optimally merges activities of allantisubmarine warfare (ASW) platforms some-thing never done before (Brown et al 2011)

Wersquore trying to write these pieces so they aretheoretically innovative with exposition of asgood quality as we are permitted within the real

estate we are allowed Whenever possible weprovide numerical examples that readers can re-produce independently And we provide oursoftware free of charge at least to DoD and itscontractors Al Washburn maintains a publichomepage full of free software (httpfacultynpseduawashburn) These papers are likemini-textbooks and they may end up beingchapters in compendia of military OR andorcivilian OR Itrsquos just not my nature to sit downand spend two years of my career writing a bookon completed past work Irsquod be pleased to helpsomeone else and I really admire my colleaguesAl Washburn Moshe Kress Wayne Hughes andothers who are not only scholars of the first mag-nitude but skilled wordsmiths who can writeclean first drafts that make sense Irsquom a lot slowerthan that A recent paper of ours went through39 iterations over several months for a single re-vision if you can imagine that (Alderson et al2011) Writing is hard work for me and takesa long time My production rate is slow

Kirk Yost I will press you on the textbookquestion one more time because the most im-portant ideas you teach are not in mainstreamtexts

Jerry Brown Thatrsquos very flattering But whenI look in the mirror in the morning shaving Irecognize that I might be able to contribute asa co-author to such a text but Irsquom not likely tofinish a monograph like that

We have published pieces to fill in what weview as gaps in textbooks and the open litera-ture (Brown 1997 Brown and Dell 2007 Brownand Rosenthal 2008) Kirk these are full of thesort of tidbits you seem to have come to valueand canrsquot find in textbooks I donrsquot want to slightany of my professional colleagues but thosewho have time to write textbooks may not alsohave time to gain the sorts of experience thatyou were exposed to here in Monterey as a doc-toral student It takes a lot of time figuring outwhat not to do

Kirk Yost Can you talk about the explosionof improvements in optimization software inthe 1990s when most people thought it wasa mature field with little left to be exploited

Jerry Brown It has been faster hardwarebut more importantly better optimizationmethods I just signed a purchase order for a16-gigabyte laptop with eight processors In a



typical evening at home I use more computerpower than it took us to get to the moon and back

Kirk Yost Dr Robert Bixby the principal au-thor of CPLEX says in his presentations that thetheory was there but wasnrsquot being imple-mented in the products Do you agree

Jerry Brown Yes I agree with thatKirk Yost Do you think thatrsquos still true todayJerry Brown The main advances in linear pro-

gramming came about because a few researcherstook the time and trouble to build a linear pro-gram package from scratch It turns out therersquosa little more involved in doing this than youmight think when you walk out of your first op-timization class

Integrating new ideas with a commercialoptimization product is hindered by lack of di-rect access to internals Open-source productssuch as the Computation Infrastructure for Op-erations Research (COIN-OR) permit this butthe overall performance of COIN-OR is unevenWhat you need is a unified design scrupulouslydebugged and tested core routines and featurespurpose-built for your design Bendersrsquo decom-position does not work very well as a bolt-on op-tion but delivers spectacular performance asa unified feature Hundreds of researcher-yearshave gone into the development and efficientimplementation of cuts for integer program-ming Now we can solve these mixed integer lin-ear programs at large scale with what 10 yearsago would have been astonishing speed

Kirk Yost Whatrsquos your philosophy about heu-ristics such as genetic algorithms versus classicaloptimization

Jerry Brown I have two concerns with theseheuristics First as we read too often lsquolsquothe com-putational complexity of this problem meanswe have to use a heuristicrsquorsquo More often thannot there is no reduction proof to support thisdefensive complexity speculation Second ourbusiness is solving hard problems on laptopsin seconds Using a complexity justification tojustify less sophisticated methods without firsthaving at least tried traditional mathematicaloptimization is well disappointing We havesome very powerful software to try and whenyou donrsquot even try you give up a bound onthe achievability of a better solution

It surprises me that so few people workingon heuristics spend the same amount of time

developing bounds in the objective quality oftheir solutions as they do developing better so-lutions The developing-better-solutions part isquite fashionable and the developing of boundsfor those solutions seems to be not quite so fash-ionable if not rare The compelling appeal ofthese heuristic techniques is theyrsquore easy to teacheasy to motivate and easy to implement Noth-ing could be easier than tabu search

But I would be very uncomfortable bettingmy professional reputation on a PowerPointslide based on a too-easy heuristic I get verynervous that someone in the audience can geta qualitatively better solution because I didnrsquotdo my work with traditional methods or workvery hard at developing an objective bound onhow good my solution is or could be I owe myclients better than that I need to find out howmuch of their money I might be leaving on thetable

Every year as an anonymous reviewer I en-counter a few papers immediately adoptingheuristics using the lsquolsquowe have to do this becauseof complexityrsquorsquo argument I customarily ask theeditor to ask the authors to provide their dataIf they refuse to do this as a scientist (and a re-viewer) this gives me pause If they provide thedata I rummage around my hard drive for some-thing I might use to try to solve their problemYoursquod be surprised how often a common com-mercial optimization package can solve theseproblems exactly and much much faster thanthe heuristic proposed

Kirk Yost Can you talk about the issue ofgetting a planner to pay $7000 for industrial-quality optimization software when hersquos usedto being issued a spreadsheet for free

Jerry Brown The providers of this state-of-the-art optimization software offer their bestpackages free of charge to universities Theseagreements typically require that we credit theprovider when we use their packages on researchand certainly require that if someone walks offcampus with one of these models they get afull-up commercial license which we make surethey do In many cases this puts you in a situa-tion where you can test the software free ofcharge during a research phase and pay for itonly if it works and you decide to use it Weare a major profit center for these software pro-viders Regardless can you imagine any problem



thatrsquos worthy of you working on it for evena week that doesnrsquot justify a $7000 softwarelicense

Kirk Yost I bring that up often and fail oftenwhich is why Irsquom interested in your views

Jerry Brown Itrsquos just nuts Irsquove encounteredfolks who think nothing of spending hundredsof thousands of dollars on analyst labor yet balkat buying a single seat with powerful modelingand optimization tools Even more ridiculousI have periodically heard lsquolsquoWersquoll save a lot ofmoney by writing our own modeling and opti-mization packagersquorsquo Whew

Kirk Yost Didnrsquot you confront this issuewhen you worked on routing C-130s aroundIraq and it became a problem

Jerry Brown It was not just the cost it wasthe availability We had to take to theater a lap-top with all the software we needed at that timeand we left it there for the planners at the Com-bined Air Operations Center (Dell et al 2006) Inparallel we developed a heuristic on a togglesomething wersquove done many times with ourdeployed software We have a toggle on thedashboard that says lsquolsquoDo you want an optimalsolution If you do yoursquove got to spend 7000bucks to have the software Or do you want afast solution and instant gratification and herersquosthe fast solutionrsquorsquo The Air Tasking and EfficiencyModel (ATEM) has been gifted to HeadquartersUS Air Force and to US Transportation Com-mand Yoursquoll have to ask them how they haveused ATEM to address exigent problems but Ido observe that some results include email listswith a lot of names you would recognize

We provide reach-back in our secret and topsecret laboratories so that planners can tell uslsquolsquoListen things have changed here in theaterCan you have a look at this to make sure yourfast solution is still as good as we hope it isrsquorsquoWersquore keenly aware that for instance the opti-mization software we desperately need to dooptimization-based decision support is notallowed to be used on Navy Marine Corps Inter-net (NMCI) computers I am the custodian fora number of laptops wersquove bought and loanedpermanently to victims of NMCI I donrsquot wantto see my property list of mission-essential gearwe have had to purchase and loan to our ana-lysts I know I have personally monogrammedlinens waiting for me at Leavenworth Federal

Prison but rather than request permission(which with NMCI these days would take thebetter part of forever and more money than Ican muster) Irsquom counting on forgiveness forgetting the job done

Kirk Yost Does anyone in DoD have a ratio-nal policy for this

Jerry Brown Are you talking about the samefolks who have prohibited jump drives eventhough there are absolutely secure ones available

The Air Force is pretty good but I think theArmy has perfect pitch When they send an ana-lyst to theater they ask lsquolsquoFrom this checklistwhat do you want on this laptop wersquore buildingfor yoursquorsquo And the analyst deploys with a full-upround The poor Marine analyst (or Navy indi-vidual augmentee) has to find an Army analystor buy his own laptop out of pocket to actuallyget any work done that requires the tools of ourtrade Those defending NMCI seem to viewa computer as an email appliance with a spread-sheet and slide maker A computer for an ORis a tool a weapon Denying Navy and MarineORrsquos access to full-up computers is a stupidand wrong information technology (IT) policyI say again this is a stupid and wrong IT policyHave I made myself clear enough

Therersquos going to be some debate but youcan go back to first principles about whetherthis NMCI thing has made any sense at all eco-nomically At one point NPS was scheduled toconvert to NMCI and I learned I would haveto donate all our high-end optimization com-puters (and we have a lot of these in our labs)and after some undetermined time for our soft-ware to be certified at some undetermined costbuy them back for a lot of money I went ballis-tic and called in a lot of chips (so to speak) To-day NPS is in the edu domain and not subjectto (but has full communication with) NMCIand the argument that saved us that our formerIT director (and NPS MS-OR) Tom Halwachsmade was lsquolsquoWho else do you have in the Navyto tell you what the next NMCI should looklikersquorsquo Whew Had we been forced to NMCI Idonrsquot think I would still be working here

Kirk Yost In the early 2000s you startedworking on two-sided optimization Can youtalk about how that came to you

Jerry Brown I have to credit DistinguishedProfessor Kevin Wood for that Kevin was



working in the early 1990s with US CentralCommand planning drug interdiction effortsOne of the early insights he contributed was thatinterdicting relatively small quantities of re-fined drugs is hard but interdicting 55-gallondrums of precursor chemicals is much easierThese travel in canoes on the rivers He cameup with some models of network flows describ-ing drug operations and how to interdict theseand it soon became clear with Special Opera-tions Forces that the tactics these people were us-ing were very adaptive These smugglers wereintelligent and observant We couldnrsquot hide ourinterdiction efforts and when we did succeed insnagging a shipment they just changed their tac-tics which led us to ponder lsquolsquoGee shouldnrsquot wemodel this so that we actually have the adversaryrepresented in a more realistic wayrsquorsquo

And then we suffered 911 saw the crea-tion of the Department of Homeland Security(DHS) and the emergence of probabilistic riskassessment as their recommended way to repre-sent terrorist threats In DoD we plan for adver-sarial intent (akin to probability assessment) andfor terrorist capability But we rarely dependupon intent That DHS was exclusively relyingon terrorist intent electrified me into action

In 2007 I was asked to serve on an NRCcommittee evaluating the DHS Bioterror ThreatRisk Assessment DHS produces a report everytwo years consisting of a small classified set ofPowerPoints to show to the President indicatinglsquolsquoHerersquos what wersquore worried about and here arethe potential consequencesrsquorsquo but backed up byan enormous technical appendix Our NRC as-sessment was not pretty Even after DHS com-plained and sequestered our report for manymonths lsquolsquofor security concernsrsquorsquo when it was fi-nally released National Public Radio called itlsquolsquoharshly criticalrsquorsquo NRC didnrsquot find much to likein overly complex models with obvious mathe-matical errors lacking any standard model lex-icon and depending on millions of probabilitiesguessed by subject matter experts (SMEs) basedon facts not known to science Unfortunatelythe NRC report was released on lsquolsquofinancial melt-down dayrsquorsquo in 2008 (National Research Council2008) A group from this NRC committee wrotea paper with a plea for DHS to come to reason(Brown et al 2008b) Responding to the nuancedDHS use of the terms probability likelihood

propensity and so on we also wrote a tongue-in-cheek paper that should give you a chuckle(Brown et al 2008a) These nuances of probabil-ity terminology are completely bogus

Probabilistic risk assessment of adversarialrisk is still spreading in DHS and DoD This isnot a good thing As Tony Cox and I argue youcannot know what a terrorist knows or willknow in the future (Brown and Cox 2011) Youcannot reckon the probability he will take anyparticular action SMEs do not render consistentadvice between themselves on terrorist intentnor do they give the same estimates for the sameconditions on repeated trials SME estimatesnever assess zero (never) or one (always) Yetan adversary will make a decision that is equiv-alent to zero or one and nothing else This is notscience this is voodoo magic

I have never encountered a lsquolsquosubject mat-ter apprenticersquorsquo Have you A subject matterjourneyman These SMEs seem to appear byself-declaration and I know of no other statedqualification

We view modeling of intelligent observantadversaries as a core competency for our stu-dents I believe ours is the sole curriculum onthe planet that requires every student to com-plete an adversarial modeling case study Weask them to prepare both sides of the action at-tacker and defender where one opponent has tomove first anticipating how his adversary willrespond to that move Wersquove got about 11 fac-ulty researching these topics with our studentsranging from missile defense to ASW

You might wonder how ASW becomes adefender-attacker optimization A ship is visibleand noisy and canrsquot be hidden from an enemysubmarine which will adjust its evasive track ac-cordingly A nuclear attack submarine (SSN) cansearch passively or by active pinging The lattergets a better fire solution but exposes the SSN

We have added a third level to the sequen-tial adversarial decisions Our tri-level modelstarts with deciding what to defend what to for-tify what to harden and so on We let the badguys see this because we canrsquot hide it Theseare huge commitments that will appear in theWall Street Journal Theyrsquove got cellphone cam-eras they can purchase satellite images andthey can use Google Earth Once they observeyour defensive preparation they get to plan



and carry out their attack(s) Once they attackwe respond by operating the surviving infra-structure as best we can

We have a viable large-scale high-fidelitymodeling technique using nested Bendersrsquodecom-position that optimizes this complete decisionportfolio at once advising the best worst-caseoutcome Wersquove demonstrated this for instanceworking with the Office of the Assistant Secre-tary of Defense for Homeland Defense andAmericarsquos Security Affairs (ASD[HDampASA])looking at the resilience of the electrical infra-structure and how that might influence missionassurance at places such as Vandenberg AirForce Base California Wersquove also demonstratedit with the roads and bridges of San FranciscoBay Wersquove looked at many other infrastructuresincluding about 150 case studies of infrastruc-tures ranging from gas or oil pipelines to pro-tecting meetings of heads of state to securingnuclear stockpiles to traffic systems Wersquove mod-eled just about everything in terms of critical in-frastructures except for banking and financeAnd if we find someone whorsquos willing to partnerwith us and is a domain expert in banking andfinance which we are not wersquore eager to help

Kirk Yost Your work analyzes a range of op-tions for both sides but the prevalent method isto rely on estimates provided by SMEs Are youmaking any headway

Jerry Brown Wersquove had some success al-though we have to separate this out Wersquove gotDoD concerns DHS ones and the private sectorIn DoD we have a very apt audience because weunderstand what intelligent adversaries areabout and how not to do things and get our-selves hurt However we have not had as muchsuccess as we would like changing the wordingof many DoD guidance documents We believethatrsquos just a matter of time Itrsquos not an error ofcommission that these documents have beenwritten with unfortunate language itrsquos just anoversight The typical directive says for instancethou shalt prioritize your targets and begin pros-ecuting them in decreasing priority until you runout of resources We know from just basic knap-sack problems that yoursquore not going to get a reli-ably good plan that way

Wersquove also had an opportunity to demon-strate this Our Professor Jeff Kline set up abenchmark in which we competed ourselves

against a well-known missile defense planningsystem We emulated find your best defenderfirst fix that in position then find your next-best defender fix that and continue until youhave no more defensive assets to fix We as-sume our opponent can detect our defensiveplatforms and change his plans accordinglyAEGIS puts out a lot of radar energy and termi-nal defenders such as surface-to-air Patriotmissile batteries are collocated with their de-fended asset so you can see them on CNN Therelative effectiveness of the sequential fixing heu-ristic for our scenarios was zeromdashall the attack-ing missiles leaked through our defenses Usingthe same set of defensive assets and a defender-attacker optimization we defended two thirdsof the same defended asset list (Logan 2007)

Wersquove had a couple of occasions within DoDto present these demonstrations and I think itrsquosjust a matter of time before these defense guid-ance documents get reworded

In DoD we do plan for enemy intent whichis the equivalent of probabilistic risk assessmentright Whatrsquos the bad guy likely to do But wealso plan for enemy capabilities where his coursesof action are limited only by his resources Whatrsquosthe worst thing he can do Wersquore better off in DoDusing intent only if we have very good intelligenceand if the planning horizon is very short Other-wise we always use enemy capabilities

Recalling WWII we had about the best intel-ligence you can imagine We were reading Japa-nese Admiralty code messages at the same timetheir ships were decoding these And wersquod re-verse-engineered the German Enigma encryp-tion machine with our Ultra emulation We hadabsolutely wonderful intelligencemdashfor examplewe were sure the Japanese were going to attackMidway If Chester Nimitz had acted on enemyintent he wouldrsquove pulled our forces out ofHawaii and far forward advantageously posi-tioned to engage the Japanese and defend Mid-way but he did not He held back because hewas cautious that if he deployed our forcesthe Japanese could still attack Hawaii and thiswould have been a disaster He waited until hehad sightings then he fully committed his shipsThatrsquos not intent thatrsquos capability If you look backin the annals of military history I think yoursquollfind very few examples of any forces committedbased on planning in terms of enemy intent Well



any good planning George Custer may havebeen an exception

Letrsquos move from the DoD across the Potomacto DHS Letrsquos ask a couple basic questions After911 why didnrsquot DHS go to DoD to learn how toplan against intelligent adversaries Why didthey instead decide to go to National Laborato-ries Physicists of course can do anything Andin 2001 National Laboratories had run out ofwork because we arenrsquot building new nukesnor testing them Our National Labs are hungrylooking for work Congress is looking for workfor the National Labs in their districts DHS isformed Congress allocates money to DHS andsays lsquolsquoGo hire National Labs and do somethingabout terrorismrsquorsquo And they did

So what did the National Labs come upwith They looked back in the archives andfound lsquolsquothe Rasmussen Reportrsquorsquo from the NuclearRegulatory Commission Rasmussen was a pro-fessor at the Massachusetts Institute of Technol-ogy who chaired the committee that issued thisreport and it is universally referred to with hisname The Rasmussen Report in 1975 made theincredible claim that engineers could predictthe outcome of extremely rare events of high con-sequence namely the probability that a light wa-ter nuclear reactor would suffer some fault thatwould cause a casualty leading to a major eventThis got a lot of press at the time with the prob-ability of a major nuclear event said to be compa-rable to lsquolsquobeing hit by a meteor while walkingdown the streetrsquorsquo Subsequent to the release ofthis report we witnessed the Three Mile Islandevent And then the Chernobyl disaster

The Nuclear Regulatory Commission calledanother committee together in 1989 to lsquolsquolook atthis Rasmussen Report and see whatrsquos wrongrsquorsquoThe Rasmussen Report was reviewed intenselyIt was slightly revised and reissued with no sub-stantive change The National Labs were wellaware of this Rasmussen Report because itrsquosled over the years to what we call today lsquolsquoprob-abilistic risk assessmentrsquorsquo And they dusted thisoff and said lsquolsquoWell clearly this is the way weshould describe terroristsrsquorsquo

As a side note Rasmussen himself warned intestimony lsquolsquoOne of the basic assumptions in the(Rasmussen report) is that failures are basicallyrandom in nature () In the case of deliberatehuman action such an assumption is surely

not validrsquorsquo Neither DHS nor its contractors seemto have noticed this

What has evolved is a large number of plan-ning systems funded by DHS and its constituentCoast Guard that in various ways assess thepossibility (that is the probability) of variousbad things happening to us Many of these arewhat we call TVC modelsmdasha probability thata terrorist will attack something lsquolsquoTrsquorsquo a vulnera-bility to that attack lsquolsquoVrsquorsquo and lsquolsquoCrsquorsquo the conse-quence of that attack typically described eitherin fatalities injuries or economic costs TheseTVC models have become widespread Al-though I had read (and frankly dismissed) acouple of papers on this appearing in the liter-ature soon after 911 I first became aware of thescope and influence of these TVC models whenI served on the NRC Bioterror committee

I have already mentioned that our evalua-tion was lsquolsquoharshly criticalrsquorsquo There have beenother NRC committees formed to study othersystems and to date when you bring in scholarswho know something about modeling adversar-ies you can expect harsh criticism and wirebrushing of these TVC models Theyrsquore just in-appropriate

So a long answer to a short question wemdashthe gang who agrees with memdashhave not yethad any discernable influence on DHS otherthan DHS now says theyrsquore aware of our con-cerns and have addressed all of them We haveno idea what this means because they havenrsquotasked us for help These systems still have nodocumentation suitable for independent techni-cal review and theyrsquore not yet cataloging data es-sential for substantive systemic analysis DHSis very defensive of very large investments onmodels based on questionable fundamental as-sumptions with answers presumably used toguide allocation of grants to state and localagencies

There are also a lot of boots on the groundgathering data describing our infrastructureThatrsquos a good thing Itrsquos necessary to know whatyour infrastructure is where it is and how it oper-ates DHS obviously doesnrsquot want to hear whatwersquore trying to tell them This is unfortunate

Because you asked letrsquos go a little furtherThese TVC models are applied to individual com-ponents of infrastructure not on infrastructuresystems But infrastructure systems have function



The electric grid has componentsmdashtransformersgenerators bus bars and transmission linesmdashbut its function is to provide power to its cus-tomers It makes no sense at all to apply a TVCmodel to individual components if you donrsquotknow how each component functions as part ofits system What we have advised is if yoursquore go-ing to plan things about an infrastructure firstyou should understand that infrastructure andhow it works (Does this sound reasonable toyou) You may be surprised to find that damageto or loss of some particular component has noinfluence at all on system function

Another component might also have no in-fluence at all But if both these components failat once say the only two exits from the buildingyou die That means you have to understand howthe system functions as a whole Thatrsquos not as easyas myopic component-wise TVC But it turns outif you look at this as we have these systems aremanaged or can be with OR models If you lookat natural gas distribution systems theyrsquore con-trolled by optimization models describing the op-eration of pipelines storage facilities and pumps(Avery et al 1992) The same thingrsquos true for crudeoil The same thingrsquos true for traffic management(Alderson et al 2011) Same thingrsquos true in virtu-ally every infrastructure system where yoursquoll findtherersquos a system operator (or regulator or eco-nomic motive) whose job it is to make sure noth-ing bad happens to guide infrastructure functionand perhaps beneficially motivate system users

For instance with the electric grid therersquos anindependent system operator (ISO) Wersquove talkedwith the ISO in California He has 40 million cus-tomers and must appear before our legislatureevery time some of these customers suffer apower interruption He cares very much aboutserving his customers reliably and well Hehas some extremely high-resolution engineer-ing models that are used to continuously advisehow to manage generation and spinning re-serves to maintain load balance for his 40 millioncustomers He controls all of our generating facil-ities here on the West Coast and contracts forpower imports Across our country every elec-tric grid has the same sort of ISO manager

Do these ISOs plan for coordinated attacks byintelligent terrorists who have studied the basicsof electrical power No they donrsquot The industrystandard is to plan for a full-up system that

can suffer any single component failed and ina limited way maybe any pair of componentsSome of these components are very vulnerableremotely located and unguarded and expensiveto replace But they are very very reliable Whyworry

When we discussed this with the CaliforniaISO we suggested we might be able find smallsimple sets of components whose loss wouldhave much more drastic effect on his grid thanhis engineering models predict He was ofcourse quite skeptical of that We pointed totheir operations map in the ISO control roomand asked lsquolsquowhat if we take out these two com-ponentsrsquorsquo This got his attention because he real-ized that it was going to be very dark in a largepart of California for a very long time And hesaid lsquolsquoHow did you know thatrsquorsquo We repliedlsquolsquobecause we have the same model you doand we embedded it in an attack planner thatfinds the worst case you can respond torsquorsquo

My points are simply these

1 You cannot predict what a terrorist will doYou cannot know what he knows or predictwhat he will be thinking in the future Thusyou cannot guess what he is going to doYou can try and perhaps gain insight by roleplaying but in the end you cannot guess hislsquolsquoprobabilityrsquorsquo (that is his decision)

2 You cannot assess system vulnerability orresilience by myopic component-wise anal-ysis ala currently fashionable TVC models

3 You can assess system function You canlearn how an infrastructure system oper-ates its management protocols and how itis used by its customers More importantyou need to model this operation to be ableto reasonably predict how the infrastructurecan respond to any injury to its components

4 You can assess the level of adversary effortrequired to damage or destroy an infra-structure component We do this for a livingin DoD and have cataloged massive data-bases for example joint munitions effec-tiveness manuals

5 You can assess or parametrically evaluatethe amount of adversarial investment (man-power money and so on) required to mountan attack We also do this for a living in DoDespecially in Special Operations



6 An operator model can reveal sets of com-ponents which might individually be un-distinguished in any particular way butwhose simultaneous damage or destructionhas catastrophic consequences

7 The economic replacement cost of a criticalinfrastructure component is irrelevant Ifa damaged or destroyed component is crit-ical it will be replaced regardless of cost

8 Effective defensive measures for critical na-tional infrastructure systems are expensiveand will be visible to those who wish to dous harm Adversaries will adapt their plansin response so we are well-advised to as-sume they will know about our defensivepreparations when we decide what to do

9 TVC models have motivated gathering dataabout our critical infrastructures and thisis a good thing Now we need to go furtherand specify how these systems of compo-nents function and are managed in the eventof failures or attack

10 Donrsquot be fooled by synonyms for the termprobability used to imply something otherthan probability

Wersquove demonstrated how to do such analy-sis by examples For instance wersquove just fin-ished two student thesis studies by invitationof the US Coast Guard Captain of the Port ofHonolulu one on the operation of the container-ized cargo imports into Hawaii (de la Cruz2011) and the other on Hawaiirsquos import stor-age refining and distribution of fuel oil and re-fined products (Ileto 2011) These students metwith the refiners electric utility commercialshippers and so on Wersquore very grateful to theUS Coast Guard for making these officialsavailable to us to reduce required travel Eachstudent built an operator model of his systemThe logistics of containers and fuel is well un-derstood Then they each looked for ways to in-terdict their system to see what the bestresponse to the worst case could be They foundparticular sets of components that are extremelyimportant to the continued function of thesesystems and these systems are vitally impor-tant to the Hawaiian Islands

We hope these case studies and manyothers like them will eventually have influenceat DHS

And by the way before the DoD readers ofthis snicker I am sorry to report that TVCmodels have bled from DHS over into DoDFor instance I have seen one example dealingwith vulnerability of Navy shore facilities Allthe criticism and warnings above apply equallyhere

Tony Cox shows by simple numerical exam-ples that you can get using these TVC modelsnot only the wrong answer but the reverse ofthe priorities you should be using (Cox 2008) As-suming the terms are statistically independentwhich defies common sense leads you to griefFor instance if V increases significantly youwould expect this to influence T wouldnrsquot you

(As I teach all my students the independenceassumption can get you killed The most stunningDoD case I recall was a model of an integratedenemy air defense system that assumed inde-pendence between all radar returns)

But I do understand how my containers arehandled I do understand how my refinery isrun (with a linear program) I do understandhow oil and gas are transported (with linearprograms)

The electric grid is also controlled in realtime by optimization models I want to usethings that I do understand such as how the sys-tem operator responds to casualties and mis-chief How does he keep the system runningHow does he plan this

That I understand And I do understand howterrorist and military actions take place Wersquovegot the Al-Qaida training manuals Wersquove gotintelligence We train Special Operations Forcesto do the same things to our enemies We havemanuals unclassified manuals on explosivesand demolition We know how many people ittakes and exactly where and how to take downthe Golden Gate Bridge We know this becausea student Red Team showed us how The sortof modeling that wersquore doing (bi-level or tri-level) we feel is based on things that we doknow or should know

I donrsquot want to guess what an adversary isthinking I canrsquot I care about defending mycountry our society and our way of life fromthe worst-case thing that could possibly happento our infrastructure If I can do that I may alsomake that infrastructure more resilient againstengineering failures and Mother Nature



Finally letrsquos move to the private sector Con-gress in its infinite wisdom passed and extendedthe Terrorist Risk Insurance Act indemnifyingprivate sector organizations from losses inflictedby terrorist actions in excess of private insurancecoverage Business has responded reasonablyenough by doing almost nothing except per-haps naming a Director of Corporate Continuityand establishing a back-up data center Theyrsquorewhistling in the dark

Kirk Yost When do you think the two-sidedmethods will become mainstream OR topics

Jerry Brown The tutorial we wrote on thisis the most highly cited one in the history ofINFORMS so something good is happening(Brown et al 2005)

Kirk Yost Can you talk about two unpleas-ant areas where optimization was heavily usedthe financial crisis of 2008 and challenge of mod-ern air travel

Jerry Brown Serving on the NRC BMSAboard Irsquove learned more than I ever wanted toknow about our monetary financial and invest-ment systems We took testimony from Treasuryofficials from major investment banks fromtraders and so on Days of this

There are some very sophisticated modelsbeing used for trading including trading deriv-atives and other exotic investments I donrsquot thinkthis was a failure of modeling These are smartpeople and theyrsquore influential This was an egre-gious failure of investment institutions and Fed-eral regulation It was also a failure in the sensethat people motivated by making a lot of moneyput a lot of lipstick on a lot of pigs and got awaywith it and to this day havenrsquot been brought tothe dock But we havenrsquot found any generallyagreed mathematical smoking gun BMSA founda couple of topics that NRC might look at if Con-gress asks I donrsquot anticipate any Federal regula-tor will ask But these topics do not includestochastic modeling or the underlying optimiza-tions still being used by for instance portfoliomanagers

Kirk Yost You did not see errors in the port-folio models that probably were all sourced inthe OR literature I would think

Jerry Brown Not as much of that appears inliterature as you might think Thatrsquos considered tobe a proprietary advantage by the people who arepaying the bills I have met some ex-students

whose suits cost more than my first car This isa sophisticated business

We have people on the BMSA panel who areexperienced very senior very accomplishedeconomistsmdashfor instance mathematicians andmodelers Wall Street typesmdashand they wouldrsquovebeen on this like a cat if they thought somethinghad been done incorrectly

Kirk Yost One of your colleagues wrote anarticle that noted optimization seeks extremesolutions Airline travel nowadays is extremein the sense that the airlines have downsizedto the minimal possible size airplanes minimalpossible seat spacing and so on And I waswondering what you have to say about that

Jerry Brown Thatrsquos a result of deregulationand Adam Smithrsquos hidden hand This is happen-ing because the market will bear it If people arewilling to pay more money to travel in greatercomfort therersquoll be more such seats available

We have a mass market that wants to paythe minimum possible to get from City A to CityB and is willing to put up with a few hours ofdiscomfort to do it If you work for the govern-ment like me yoursquore expected to use the cheap-est lowest-class service available to this massmarket so your last-minute travel will be inthe last available seat that doesnrsquot recline inthe back middle of the five-across seats Just suf-fer with it

My advice for US airlines if they want tosave a lot of money is to dissect their proformalabor contracts with their pilots and cabin atten-dants Over years the sheer length of these con-tracts has grown to far exceed the impressivevolume of Federal Aviation Regulations Thereare reasonable credits for working at night lay-overs and so forth However letting your flightcrews live wherever they want and fly (often atno cost) an arbitrary distance and time to get totheir official domicile to begin a duty periodneeds adult intervention The Federal AviationAdministration is looking into crew fatigue asa result of this Letrsquos cross our fingers that theNational Transportation Safety Board doesnrsquothave to join this hunt after another incident

Any industry that lets its high-paid execu-tives work for the first part of each monthfor a specified number of hours then take therest of the month off partitioning such labor re-cords in strict monthly buckets needs its head



examined And thatrsquos exactly what we have inthe US airlines these days

Bob Sheldon You have a lot of former stu-dents who will be reading this oral history aswell as current and future students who willbe reading it Any comments you would maketo them about their profession

Jerry Brown I have enormous respect andadmiration for our students and I am gratefulto have had the opportunity to work with theseamazing people

I am trying to make a few changes here atNPS I want our local junior college to staff andteach a basic English exposition class for usAmerican secondary education has collapsedand even some inputs wersquore getting from theservice academies have managed to keep it a se-cret from themselves and others that they canrsquotcompose a complete paragraph in English Wefinally confront this here when it comes timeto review the mandatory MS-OR thesis draftWhew Thatrsquos too late We need to screen earlyin our curriculum and help these junior officersget squared away

This canrsquot happen to you as an OR OR isabout describing a problem back to the clientso the client declares lsquolsquoYes thatrsquos what I meantto sayrsquorsquo Exposition is all and clarity of exposi-tion is a symptom and a concomitant with clar-ity of thinking These are innocent victims butthey really really need to take a remedial lsquolsquobone-headrsquorsquo English exposition class until they canpass a test writing a complete clear paragraph

Irsquove also advised a number of my studentsand colleagues to participate in Toastmastersas I have done This is a very effective way to in-vest one lunch hour a week learning how to im-prove verbal exposition It can be a lot of fun andit works Our junior officer students who havenot yet discovered their exposition problemsare soon going to be the go-to experts when theygraduate Theyrsquore going to be expected to writepoint papers for Monday briefs about materialthat gets dropped on their desk at 1700 on Fridayafternoon and they can expect to be appointed tomake presentations to senior executives Theyneed to know how to speak They need to knowbody language They need to know when not toput their hands in their pockets how to dresshow to face an audience moderate voice andconduct themselves This is an important part

of our profession This is an important part ofour education here and I want to enhance this

Irsquom pleased that MORS has prizes for ex-pository excellence both written and verbaland I encourage MORS to continue that Itrsquos im-portant MORS and NPS OR have the MORS-Tisdale competition among members of eachMS graduating class started by Rick Rosenthalwho named it for his late student Steven TisdaleThis is our most important single award de-cided by an expository competition in front ofan audience of all students and faculty includ-ing senior executives invited for the occasionThe winner is judged to be the best expositionof the best analysis

Kirk Yost Rick Rosenthal did a lot to bringoptimization tools to the masses Could you com-ment on Rickrsquos contribution to optimization in-struction at NPS

Jerry Brown Rick introduced us to optimiza-tion modeling languages in particular to GAMS(httpwwwgamscom) This reduced the ef-fort to build a new model from days to minutesRick had boundless enthusiasm for teachingstudents and mentoring young scholars andmany readers of this will recall his charm andwarmth (Bausch et al 1991 Brown et al 2007Newman et al 2011 Brown and Dell 2007 NavalResearch Logistics 2011)

Kirk Yost I have heard that the classified ver-sion of MOR was your idea Is that true and howis it coming

Jerry Brown That is true NPS ProfessorRobert lsquolsquoBobrsquorsquo Koyak is the editor (rakoyaknpsedu) The MORS National Security Operations Re-search journal is now in operation I believe this isgoing to be put out as an electronic paper thatrsquoson a push from SIPRNet This is currently theonly way we should be publishing real stuffwhile we still have our boots in theater Wewould like to be able to publish for the consump-tion of others within the MORS community someof the things wersquove done over there and we canrsquotWe also think we have a lot to learn from othersThe only opportunity we have had is to give andlisten to talks at our MORS meetings and thatrsquosnot as satisfying or instructive as having a com-plete archival document

Bob Sheldon Is it natural to take some of thoseclassified papers and sanitize them and makethem unclassified or does that prove too difficult



Jerry Brown Wersquore not a classifying author-ity here And how do you lsquolsquosanitizersquorsquo an ideaOf course this stuff is secret It involves protect-ing our personnel and making them more po-tent Once wersquore out of there it might bereasonable to go back and make a case lsquolsquoOkayherersquos what wersquove done and how it workedWhat part of this is operationally still a matterof some sensitivity and what part of this isnow so routine and well-known we can publishit and make our taxpayers proud of usrsquorsquo

Bob Sheldon Anything else from you KirkJerry Brown Kirk you never asked me the

hardest problem I ever solvedKirk Yost Everything you work on looks

hard to me But I will ask the question what isthe hardest problem you ever solved

Jerry Brown It was at UCLA A guy handedme an x-ray crystallography problem And Iworked on that for about five months I justrediscovered my notes here a couple weeks agoI donrsquot even understand my own notes any-more But the x-ray crystallography problem isfascinating Itrsquos what we now call reverse opti-mization What you have is a bunch of digitalevidence of what the answer is and you seekthe question most likely leading to that answerTo this day I have to admit that was the hardestthing Irsquove ever done

Kirk Yost Are you satisfied with the prog-ress you made on it

Jerry Brown Yes we made good progressand now therersquos a whole area in physics thatstudies this and quite a bit more theory proba-bly beyond my ken There are a lot of relatedareas in microscopy and things like optical iden-tification of fingerprints eye prints facial recog-nition and so forth Itrsquos fascinating stuff Irsquomglad we have smart guys who can work on it

Kirk Yost In your view what is OR and whyshould we care

Jerry Brown Fundamentally OR is aboutdiscovering simplicity in complexity discover-ing clarity in confusion Let me read to youhow Mike Mullen put it well during his inter-view with INFORMS (Horner 2010)

lsquolsquoOne of the great things that the graduate educationin OR taught me was how to think much more crit-ically than I had before and really to frame a prob-lem And where that really helps me in this job is

being able to still frame a problem in my mind and tolook at it differently than many people who bringthose problems to mersquorsquo

And then I have an opportunity to ask the rightquestions Itrsquos become a pretty natural part ofhow I do business the ability to frame a problem and then ask hard questions that push the systemin a direction of an answer that clearly wasnrsquot forth-coming by the time it got to mersquorsquo

People most frequently come to us espe-cially in military OR not because they want tobut because they feel they ought to or have toTheyrsquore typically faced with complex problemsand feel for whatever reasonmdasheither our reputa-tions from prior successes or just sheer hopemdashthat somehow we might be able to help withthese problems

If you look at the way we do help you willfind a common pattern for successmdashone not de-scribed in any textbook or manual And I thinkthese standard things help explain why ourgraduates have been so successful in senior pol-icy positions

The first thing wersquore trained to do is to de-fine and use a standard lexicon and we employthat language carefully We then use that lexiconto write down in our native language in ourcase English but in any native language writedown our understanding of what the problemis (Brown 2004) Thatrsquos step 1

1 What is the problemWe usually write two versions of this We

write one version that is the executive versionsuitable for a general officer without such train-ing as ours to read and to understand And an-other aimed at our OR colleagues

Irsquove also advised to have someone who isuntrained in OR read your problem descriptionback to you A spouse will work If when theyrsquorereading this back to you they hesitate or stopand look up at you or they need terms definedthat they donrsquot understand such as lsquolsquoalgorithmrsquorsquo(this term is the third rail of such descriptions)then you need to edit and rewrite

Next you have to decide

2 Is this problem importantIn government we can sometimes get in-

volved in studies about differences that canrsquotmake a difference The earlier you conclude



you are engaged with a problem not worth solv-ing the better This conclusion shows how youfinish your problem description because the po-tential client who owns the problem has to recog-nize this If the potential client just wants to useyour imprimatur to burnish some routine deci-sion itrsquos up to you whether to cooperate

Next you have to find out

3 How will this problem be solved withoutyour help

It turns out tribal wisdom can be pretty ef-fective especially in DoD where people haverisked much to learn how to solve problemsand how not to solve them Here it is wise to ac-tually visit the operations center office com-mand or wherever these decisions are madeA personal visit and interview of participantscan be most enlightening It reveals the tone ofthe organization the level of sophistication ofparticipants and perhaps most important thewillingness to actually participate No matterhow much senior officers and managementmay admire some nifty possible new improve-ment if their subordinates arenrsquot convinced in-centivized and on board you are dead in thewater

Irsquom keen on looking for any post-it noteshandwritten spiral binders or other paper refer-ence materialsmdashthese are probably not on anycomputer system and very likely contain gemsof wisdom Irsquom also aware that a phone callcan beat a clever decision support system everytime because such a call can relax a requirementmodify a mission change an objective and so onYou canrsquot get a sense from afar of whether suchhorse trading is used When we take our problemdescription back to the people who brought theproblem to us and if they read that descriptionand say lsquolsquoYes thatrsquos what we meant to sayrsquorsquothatrsquos a good sign wersquore making progress

(Irsquoll bet a number of MOR readers have ex-perienced some technical briefing when the au-dience starts to get lost and all turn to the soleknown OR in the room for clarification Doesthis ring a bell)

Surprisingly this is the successful end ofmany engagements Having merely clarifiedthe problem statement sorted out distractorsstated what courses of action are available andexamined current practice you may be finished

For example I have been asked more than onceto schedule the Navyrsquos fleet of executive air-craft After due diligence I have always con-cluded this is not worth the effort On-calldemands and shifting priorities with no likelyway to forecast these makes an operations cen-ter whiteboard a pretty effective tool for visibil-ity and decision making And besides myunderstanding is that any lsquolsquoscheduledrsquorsquo air trans-port is the exclusive business of our Air ForceNah

If we decide to continue the real fun beginsWe may get to do some preliminary mathemat-ical modeling But regardless we next have toplan design and formally commit to answer

4 What do you propose to doThis is the meat and potatoes of our profes-

sion but we have to hesitate to jump to this stagebefore passing all prior qualifications And a keycorollary question we must answer up front is

5 How will we all know when you have suc-ceeded or failed

Remember OR is (or should be) scienceThroughout our mantra is lsquolsquoif itrsquos not writtendown it never happenedrsquorsquo PowerPoints canhelp but never substitute for writing Thesewritings are the key step in design beforeyou do anything else This brings us to the laststep

6 Is the documentation of your success suffi-cient for external professional and technicalreview

I have encountered important decision sup-port systems in DHS and DoD notably recentones using probabilistic risk assessment for in-telligent adversaries that are documented ex-clusively by PowerPoints if at all And eventhese scant materials are held in confidence(Not classified just held back) This is shamefulIf I am asked to evaluate such a system you canexpect strenuous objection On occasion I havedetermined that the lack of documentation is anunambiguous telltale that the proponents donrsquotknow what theyrsquore doing This is dangerous

These six guidelines suggest for either sideof the table how to structure engage manageand conduct a defense contract involving OR



An operations researcher should be prepared toseamlessly and comfortably transition from theverbal description of the problem to the mathe-matical description of the problem to perhapsany modeling thatrsquos required to address theproblem and especially be effective in ex-plaining lsquolsquothe whatrsquorsquo and more importantlylsquolsquothe whyrsquorsquo of the results that you get from suchmodeling

Kirk Yost Yoursquove been elected to the NationalAcademy of Engineering (NAE) the first NPSfaculty member to receive such an honor andmost military operations researchers donrsquot fol-low the activities of that organization Can youexplain why that should be important to us

Jerry Brown The National Academy of Sci-ences was created by President Lincoln in1863 during the Civil War to seek help from ci-vilian engineers and scientists to advise theUnion on issues that were vital to the conductof the war For instance lsquolsquois metal cladding ofnaval ship wooden hulls worth the time andexpensersquorsquo The NAE was created a hundredyears later in 1964 as a sister organizationnot so much devoted to the pure sciences as tothe new engineering sciences such as (today)aerospace bioengineering chemical civil com-puter science electronics operations (thatrsquosus) materials mechanical and earth resourcesengineering

I learned of my election by the members ofNAE via a cellphone call from my chairman atthe time Jim Eagle while standing in line withJeff Kline at Dulles waiting to board the firstof flights home from Washington Jeff and I wereseated separately in the lousy coach seats we arerequired to occupy and we had completed ourmission save writing a report we could not doin public A cabin attendant showed up with acold beer from Jeff Thanks Jeffmdashbest beer Iever enjoyed

The mission of the NAE is to objectively ad-vise on questions about technology and policyThere are currently about a dozen memberswho have contributed directly to military oper-ations research Among us for instance thelate Seth Bonder Peter Cherry Don Gaver DaveMaddox Bill Perry Steve Pollock Steve RobinsonLarry Stone and Al Washburn Typically NAEgets involved with the National Academy of Sci-ences and the Institutes of Medicine the three

sister organizations through the NRC which isthe coordinating organization The NRCrsquos fore-most frequent client is the US Congress withDoD a close second Studies are commissionedto advise on technological questions that bearon emergent policy issues

In such a case NRC will form a committeetypically numbering a dozen or more composedof members of the Academies as well as aca-demics scholars and other domain experts Acommittee will meet maybe four to six timestake testimony for a day or two at a time fromexperts and deliberate (either with a press gal-lery or in closed session) Between meetingsmembers have homework to do correspond ex-tensively and plan with NRC staff who arrangesinvitations for the next meeting Eventually thecommittee writes a lsquolsquoconsensus reportrsquorsquo whichis anonymously reviewed by about 10 reviewersand an editor reviewed with the client organi-zation or the subject of the study as directedand then released to the public (httpwwwnationalacademiesorgpublications) All thisis aimed to respond to inform and advise leg-islators and administration officials our bestadvice The idea is to influence policy and in somecasesmdashyou always have to follow the moneymdashchange appropriations

NRC also manages studies boards for theuniformed services

Bob Sheldon A distinction between theoreti-cians and practitioners For the field of optimi-zation there are people who do theoreticaloptimization and there are people who solvereal-world problems Sometimes itrsquos viewed asa dichotomy but you seem to embrace both ofthose yoursquore both a practitioner and a theoreti-cian Could you comment on that

Jerry Brown Both are essential You need totry to develop and maintain deep roots and es-pecially be open to new ideas Sometimes ittakes a while to really internalize which of allthese new theoretical results can be put togetherwith other results and have some consequenceOn the other hand having some actual practicewith real-world problems gives you instinctsabout where you need to apply yourself theoret-ically and what will work and what wonrsquot Ihave no particular bias about pure theoreticiansor pure practitioners except to say that I lamentthe fact that pure theoreticians donrsquot at least



have some real experience to help shape theirthinking and their instincts

And pure practitioners can get in trouble ifthey donrsquot understand the underpinnings of thetools theyrsquore using We see embarrassing evi-dence of these extremes all the time So thereis a middle ground I admit Irsquove always writtenpapers with as few theorems as possible Ifyoursquore absolutely forced to write a theorem tojustify what yoursquore doing then maybe thatrsquosa new result If yoursquore writing theorems just topepper a paper with them then yoursquore just fool-ing yourself Most such theorems I see couldhave instead pointed to prior ones

Bob Sheldon Offline we were talking aboutyour experience flying while you were in New-port and I think some people will be interestedif you care to relate that

Jerry Brown Itrsquos a story of how sometimeswhen you get involved in military and Navyregulations funny things happen and therersquosnothing you can do about it I was an officer can-didate at Newport and the Navy and Marineswere quite desperate for aviators At that timeif you could pass an aviation physical and had2020 vision they encouraged you so stronglyyou might say they coerced you into taking avi-ation familiarization training I was put on a busto a local airport introduced to an instructor pi-lot and we took off for our initial familiarizationflight over Narragansett Bay My instructorstarted me with some basic maneuvers Thiswent well so we quickly progressed to stallsdeparture stalls spins hood work unusual atti-tudes and he finally asked me to land Taxiingback to the ramp he declared lsquolsquoYoursquore a ringerYou already know how to fly What are you do-ing herersquorsquo I replied lsquolsquoBecause they told me Ishould be here so here I amrsquorsquo

We both revealed all to OCS We were en-couraged to continue my aviation familiariza-tion so my delighted flight instructor and Ihad a lot of fun with aerobatics and will neitherconfirm nor deny regularly enjoying apple pieand a cup of coffee on Martharsquos Vineyard

Bob Sheldon Anything else that you care torelate

Jerry Brown We are most grateful for sus-taining pure research support from the AirForce Office of Scientific Research and the Of-fice of Naval Research This enables us to para-

chute in to problem situations on short notice(NPS has no mission funding for researchmdashafact that surprises folks seeking our help)

Most of all Irsquom grateful to my loving andtrusting spouse who knows how much I lovethis job She understands and appreciates theconsequences of our work and tolerates thelong hours and those occasions when I haveto disappear without prior warning or laterexplanation

REFERENCESMost of these references can be downloaded

from httpfacultynpsedugbrown

Alderson D G Brown M Carlyle and KWood 2011 lsquolsquoHow to Assess the Value ofCritical Infrastructure A Worst-Case View ofRisk and Its Implications for DefensiveInvestmentrsquorsquo(in review)

Avery W Brown GG Rosenkranz J andWood RK 1992 lsquolsquoOptimization of PurchaseStorage and Transmission Contracts for Nat-ural Gas Utilitiesrsquorsquo Operations Research Vol40 pp 446ndash462

Bausch D Brown GG Hundley D Rapp Sand Rosenthal RE 1991 lsquolsquoMobilizing MarineCorps Officersrsquorsquo Interfaces Vol 21 pp 26ndash38(1989 Koopman Award winner)

Bradley G Brown GG and Graves G 1977lsquolsquoDesign and Implementation of Large-ScalePrimal Transshipment Algorithmsrsquorsquo Manage-ment Science Vol 24 No 1 pp 1ndash34 (1977Lanchester Prize Finalist)

Brown G 2003 lsquolsquoHas IT Obsoleted ORrsquorsquo ple-nary address INFORMS Phoenix AZ May 4

Brown GG 2004 lsquolsquoHow to Write About Opera-tions Researchrsquorsquo PHALANX Vol 37 No 3 p 7

Brown G and Carlyle WM 2008 lsquolsquoOptimiz-ing the US Navyrsquos Combat Logistics ForcersquorsquoNaval Research Logistics Vol 55 pp 800ndash810(Winner 2009 Harold W Kuhn Award)

Brown G and Cox L 2011 lsquolsquoHow ProbabilisticRisk Assessment Can Mislead Terrorism RiskAnalystsrsquorsquo Risk Analysis Vol 31 pp 196ndash204

Brown GG and Dell RF 2007 lsquolsquoFormulatingLinear and Integer Linear Programs ARoguesrsquo Galleryrsquorsquo INFORMS Transactions onEducation Vol 7 No 2 January



Brown G and Rosenthal R 2008 lsquolsquoOptimiza-tion Tradecraft Hard-Won Insights fromReal-World Decision Supportrsquorsquo InterfacesVol 38 pp 356ndash366

Brown GG and Rutemiller H 1971lsquolsquoSequential Stopping Rule for Fixed-SampleAcceptance Testsrsquorsquo Operations Research Vol 19pp 970ndash976

Brown GG and Rutemiller H 1973 lsquolsquoA CostAnalysis of Sampling Inspection Under MIL-STD 105Drsquorsquo Naval Research Logistics QuarterlyVol 20 pp 181ndash199

Brown GG and Rutemiller H 1974 lsquolsquoTablesfor Determining Expected Cost per UnitUnder MIL-STD-105D Single SamplingSchemesrsquorsquo AIIE Transactions Vol 6 pp 135ndash142

Brown GG and Rutemiller H 1975 lsquolsquoAnAnalysis of the Long-Range Operating Char-acteristics of the MIL-STD-105D SamplingScheme and Some Suggested ModificationsrsquorsquoNaval Research Logistics Quarterly Vol 22 pp667ndash679

Brown G and Washburn A 1980 lsquolsquoKhachianrsquosAlgorithm A Tutorialrsquorsquo Technical ReportNPS55-80-008 Naval Postgraduate SchoolMonterey CA February

Brown GG Coulter D and Washburn AR1994 lsquolsquoSortie Optimization and MunitionsPlanningrsquorsquo Military Operations Research Vol 1pp 13-18 httpfacultynpsedugbrowndocsBrownCoulterWashburpdf

Brown G Cox L and Pollock S 2008alsquolsquoWhen is Uncertainty About UncertaintyWorth Characterizingrsquorsquo Interfaces Vol 38 pp465ndash468

Brown GG Dell RF and Farmer R 1996lsquolsquoScheduling Coast Guard District CuttersrsquorsquoInterfaces Vol 26 pp 59ndash72

Brown GG Dell RF and Newman AM2004 lsquolsquoOptimizing Military Capital Plan-ningrsquorsquo Interfaces Vol 34 pp 415ndash425

Brown GG Dell RF and Wood RK 1997lsquolsquoOptimization and Persistencersquorsquo InterfacesVol 27 pp 15ndash37

Brown G Carlyle WM Salmeron J andWood K 2005 lsquolsquoAnalyzing the Vulnerabilityof Critical Infrastructure to Attack and Plan-ning Defensesrsquorsquo in Tutorials in Operations Re-search Emerging Theory Methods and

Applications H Greenberg and J Smith edsInstitute for Operations Research and Man-agement Science Hanover MD

Brown GG Clemence R Teufert W andWood RK 1991 lsquolsquoAn Optimization Modelfor Army Helicopter Fleet ModernizationrsquorsquoInterfaces Vol 21 pp 39ndash52

Brown GG Dell RF Holtz H and NewmanAM 2003 lsquolsquoHow the US Air Force SpaceCommand Optimizes Long-Term Investmentin Space Systemsrsquorsquo Interfaces Vol 33 pp 1ndash14

Brown G G Kline J Rosenthal R andWashburn A R 2007 lsquolsquoSteaming on ConvexHullsrsquorsquo Interfaces Vol 37 pp 342ndash352

Brown G Kline J Thomas A Washburn Aand Wood K 2011 lsquolsquoA Game-TheoreticModel for Defense of an Oceanic BastionAgainst Submarinesrsquorsquo Military Operations Re-search Vol 16 No 4 pp 25ndash40

Brown G Banks D Borio L Parnell G andWilson A 2008b lsquolsquoScientists Urge DHS toImprove Bioterrorism Risk AssessmentrsquorsquoBiosecurity and Bioterrorism Biodefense Strat-egy Practice and Science Vol 6 pp 353ndash356

Brown G Carlyle M Diehl D Kline J andWood K 2005 lsquolsquoA Two-Sided Optimizationfor Theater Ballistic Missile Defensersquorsquo Opera-tions Research Vol 53 pp 263ndash275

Cox LA 2008 lsquolsquoSome Limitations of lsquolsquoRisk frac14Threat x Vulnerability x Consequencersquorsquo forRisk Analysis of Terrorist Attacksrsquorsquo RiskAnalysis Vol 28 No 6 pp 1749ndash1761

de la Cruz C 2011 lsquolsquoDefending the MaritimeTransport of Cargo for the Hawaiian Islands(U)rsquorsquo NPS MS-OR thesis March

Dell RF Brau J Brown GG and WM CarlylelsquolsquoScheduling Our Call-for-Haul Airline forOperation Iraqi Freedomrsquorsquo 74th MilitaryOperations Research Society SymposiumColorado Springs Colorado June 13-15 2006

Horner P 2010 lsquolsquoMeet the (OR) Press Inter-view with Adm Mike Mullen Chairman ofthe Joint Chiefs of Staffrsquorsquo httpwwwinformsorgAbout-INFORMSNews-RoomINFORMS-BlogMeet-the-OR-Press-Interview-with-Adm-Mike-Mullen-Chairman-of-the-Joint-Chiefs-of-Staff (July 23)

Ileto J 2011 lsquolsquoImproving the Resiliency of thePetroleum Supply Chain for the HawaiianIslands (U)rsquorsquo NPS MS-OR thesis March



Logan K 2007 lsquolsquoOptimizing Layered BallisticMissile Defense and Comparing Prioritizedwith Damage-Minimizing Defense Planning(U)rsquorsquo NPS MS-OR thesis September

National Research Council 2008 lsquolsquoDepartmentof Homeland Security Bioterrorism RiskAssessment A Call for Changersquorsquo Commit-tee on Methodological Improvements tothe Department of Homeland SecurityrsquosBiological Agent Risk The National Acad-emies Press Washington DC http

booksnapeduopenbookphprecord_idfrac1412206

Naval Research Logistics 2011 lsquolsquoIn Memory ofRichard E Rosenthal 1950-2008rsquorsquo Vol 58 No3 April

Newman A M Rosenthal R E Salmeron JBrown GG Price W Rowe A FennemoreCF and Taft R L 2011 lsquolsquoOptimizing As-signment of Tomahawk Cruise Missile Mis-sions to Firing Unitsrsquorsquo Naval Research LogisticsVol 58 No 3 pp281ndash295



Speech Movement and the riots and tear gasand so forth And not having a scholarship Ihad to work for a living

Between the demonstrations and workingat night I had had enough of that in about threequarters and transferred I ended up getting twocollege degrees from California State UniversityFullerton and then I went to UCLA for my PhD

Kirk Yost Both your bachelorrsquos and masterrsquosdegrees are from Cal State Fullerton

Jerry Brown They are And are you ready forthis My bachelorrsquos degree is in business market-ing (quantitative methods) my masterrsquos degreeis an MBA (quantitative methods) and my PhDis in management

Bob Sheldon I want to ask a question aboutperception Therersquos often a perception of two-way snobbery between engineering schools andbusiness schools Many folks in the engineeringschools assume that if somebodyrsquos too weak intheir mathematical rigor to survive the engineer-ing school they go to the business school And inthe business schools therersquos a perception that ifa personrsquos too weak in their personality skills tosurvive in the business environment they go tothe engineering school Given that you survivedquite well you excelled in business school andyou are quite rigorous mathematically whatrsquosyour comment on that

Jerry Brown It depends on the business schooland on the engineering school My recollectionis that if you could understand the Black-Scholesequation you were no slouch mathematically Ifyou talk to a mathematical economist or to some-one mathematical in finance and so forth theydonrsquot have much to apologize for in terms ofmathematical rigor You need to establish foun-dations in mathematics and English expositionEquipped with these skills you can do about any-thing you want The engineering and business op-tions you mention just emphasize one or the otherbut cannot (or should not) be attempted with seri-ous deficiencies in either area

Irsquom not a keen fan of undergraduate businessprograms I prefer to see MBA students whohave completed undergraduate studies in someconventional academic area

I donrsquot think therersquos any particular weak-ness on either side here If you choose a good uni-versity a good curriculum and if you seek rigorand follow accomplished scholars yoursquoll find it

Kirk Yost Although all of your degrees are inmanagement or business administration yourearly publications are on statistics Did you thinkyou were going to be a statistician And how didyou eventually turn to optimization

Jerry Brown One of my early mentors wasHerb Rutemiller and yoursquoll see his name as aco-author in some of those papers He was veryinfluential and showed me how to do researchand how to publish He was a very accomplishedstatistician I followed his lead and got exposedto the literature of operations research (OR)But I donrsquot regret the business degrees becauseI can read operating statements and speak thelanguage and thatrsquos been useful in consultingI had worked for some time at North AmericanAviation Beckman Instruments and HughesSatellites on computers and so I was lucky tobe an early computer guy when such were stillrare

I was at that time hired out as a general-pur-pose mathematical and scientific digital pro-grammer This was when there werenrsquot thatmany large-scale machines and before the ad-vent of the current numerical recipes and suchOn my bookshelf here Irsquove still got basic refer-ences like Abramowitz and Stegun with tablesof Bessel functions and the like It was computerexperience that opened up early research to mepaid my bills and also got me working on somedefense problems

I became interested in optimization whileworking with Rutemiller but also while work-ing at UCLA with Art Geoffrion Glenn Gravesand company And theyrsquore the ones who reallypersuaded me to move from descriptive statis-tics to prescriptive optimization

I became besotted with the leverage thatthis sort of modeling gave us and I was fortunateto be involved in some early practical applica-tions at large scale including one at RAND Cor-poration you may remember Kirk that helpedthe Air Force decide what bombs to buy everyyear for two decades (Brown et al 1994)

Kirk Yost Can you outline how you went fromCal State to UCLA and subsequently into theNavy

Jerry Brown When I got my masterrsquos degreeand was accepted into UCLA I was hired at CalState Fullerton as an assistant professor of quan-titative methods



























































































































































































































































































































































































































2 Military Operations Research V16 N4 2011
































































































































































































































































































































INTRODUCTION O Military Operations Research Society …faculty.nps.edu/gbrown/docs/MOR Vol 16...

Documents

Transcript of INTRODUCTION O Military Operations Research Society …faculty.nps.edu/gbrown/docs/MOR Vol 16...