ComputerPhysicsCommunications46 (1987)401—415North-Holland, Amsterdam

MERLIN - A PORTABLE SYSTEM FOR MULTIDIMENSIONAL MINIMIZATION

G.A. EVANGELAKIS, J.P. RIZOS,I.E. LAGARISPhysicsDepartment,UniversityofIoannin~Joannina,Greece

and

I.N. DEMETROPOULOS

Departmentof Chemistry,Universityof Ioannina loannina, Greece

Received5 January1987; in revisedform 5 May 1987

PROGRAM SUMMARY

401

Title of theprogram: MERLIN

Cataloguenumber:AAXW

Programobtainablefrom: CPCProgramLibrary, Queen’sUrn-versity of Belfast, N. Ireland (see application form in thisissue)

Computer:CDC CYBER-171; Installation: Universityof loan-nina, loannina,Greece

Operatingsystem:NOS 2.4.1 630/628

Programminglanguageused: ANSI FORTRAN 77

High speed storagerequired: dependingupon the maximumnumberof variablestheobjectcanhandle(28000wordsfor 30variables,50000words for 150 variables)

Numberofbits in a word: 60

Peripheralsused: terminal, line printer, cardreader,disc

Numberof lines in combinedprogram and lest deck: 4370

Separatedocumentationavailable: manual(54 pages)

Keywords:minimization, datafitting, SIMPLEX, DFP, BFGS,variablemetric,stochasticsearch,line search,numericaldiffer-entiation

NatureofphysicalproblemA lot of problemsin physics, chemistry,appliedmathematicsas well asin engineeringand in other fields,are quite oftenreducedtominimizing afunctionof severalvariables.MERLINis a systemdesignedto minimize a multidimensionalfunction.

Methodof solutionSix algorithms are implemented.Threeof them make useofthe function’s gradientandhenceare suitablefor minimizingdifferentiablefunctions,therest threedo not usederivativesatall and so are applicableto non-differentiablefunctions aswell.The first three arethequasi-Newtonmethodsknownas DFP(1], BFGS [2] andtheconjugategradientmethodof PolakandRibiere [3]. The SIMPLEX method of Nedler and Mead [4],correctedas suggestedby Mike and Chaves[5], a randomsearchand anadhocmethod,which useonly function values,areimplementedaswell.

Restriction of thecomplexityof theproblemCurrentlyMERLIN is dimensionedto handleup to 150 varia-bles.However by redimensiornnga few arraysit can easilybeenhancedorreducedaccordingto user’sneeds,asdescribedintheprovidedmanual.

Typical running timeSince housekeepingoperations are quite fast, running timeheavily dependson the complexity of the objective function.The provided test run, took 7.12 CPU secondson a CDCCYBER-171.

Unusualfeaturesof theproblemApart from beinga minimizationprogram,MERLIN is desig-ned to be easilyfurther developedby the user and evolve indifferent directions.Henceit can serveasa testing groundofdifferent trial algorithmsand can be seenasan optimizationdevelopmentsystem.

References[1] W.C.Davidon,AEC R&D report ANL 5990(1959).

R. Fletcherand M.J.D. Powell, ComputerJ. 6 (1963)163.

OO1O-4655/87/$03.50© ElsevierSciencePublishersB.V.(North-HollandPhysicsPublishingDivision)

402 GA. Evangelakiset al. / MERLIN

[2] R. Fletcher,ComputerJ. 13 (1970) 317.D. Goldfarb,Maths. Comput.24 (1970)23.C.G. Broyden,J. Inst. Maths.Appl. 6 (1970) 76 and222.D.F. Shanno,Maths. Comput.24 (1970)647.

[3] E. Polak,ComputationalMethodsin Optimization:A Uni-

LONG WRITE-UP

1. Introduction

Minimizing a function of many variablesis aproblem common to many fields. A lot of prob-lems in engineering,chemistry, physics etc. arequite often reducedto minimizing a function ofseveral variables. As examples we refer tonuclear-interactionmodelingby potentials [1], tocomputationalfitting of ab initio potentialenergysurfaces[2], to curve fitting (anevery-dayneedinexperimentalwork), andto all kinds of variationalmethodsthat are applied either to classicalor toquantummechanicalproblems.

The difficulties of multidimensionalminimiza-tion are many anddiverse.The function may notbe analyticallyknown, so that derivativeshavetobe estimated numerically, which costs both incomputing time and in accuracy. Problemsalsoarisewhenmany localminima exist, since thereisno algorithm that guaranteesconvergenceto theglobal solution. We do not know of any methodthat dealssatisfactorilywith this problem. Whenphysical limits are imposedon some parameters,the problem becomesconstrainedand thus moredifficult to handle.

MERLIN is a systemconstructedin anattemptto solve the abovementionedproblems,aiminginadditionto be user-friendlyandeasilyportabletodifferent machines.It hasbeenundercontinuOusdevelopmentsince1983 andit hasbeenbenefitedfrom our experiencewith the programMINUIT[3].

2. Generaldescription

The easeof use is achievedby meansof anoperating system residing in SUBROUTINEMERLIN. The userenterscommandsto instructthe programfor the route of action. There is an

fied Approach(AcademicPress,New York, 1971).[4] J.A.Nedlerand R. Mead,ComputerJ. 7 (1965) 308.[5] K. Mika and Th. Chaves, Berichte der KFA JOlich Nr.

1643 (1980).

on-line commanddirectory to remind theuserofthe commandsin case of uncertainty, and anon-line HELP that offers a brief descriptionforeachcommand.

Thecommandscanbeseparatedinto categoriescorrespondingto their action as: Minimization,Information, Administration, Editing and Usercommands. -

Minimization commandsinstruct MERLIN tocall the appropriateminimization routines.

Information commandsgive information aboutMERLIN’s operatingsystem.

Administration commandscan be mode-com-mands,displaycommandsor control commands.

Editing commands refer to MACROS. AMACRO is a packageof commandsthat the usermay createinteractively.To modify a MACRO asimple editor is providedwhich is operatedby theabovementioned editing commands.User com-mandsare dummycommands.They do nothing.Simply transfer the control to a point in theprogramwherea futurecall to an associatedsub-routinemay appearandthenreturn the controltothe commandpromptingpoint. MERLIN recog-nizes them as valid commandsi.e. they are builtinto MERLIN’s operatingarchitecture,in order toeasefurther developmentby the interesteduser.

MERLIN is portablein the sensethat carehasbeen taken to follow closely the FORTRAN 77ANSI standards.Assembly routinesare not usedandopen/closestatementsaremadeoptionalsincemany compilerstreat them in different manner.Thereis only oneexception.The randomnumbergenerator function (used only in one place, insubroutineRANDOM) is chosento be the CDC’sintrinsic function RANFO, that has to be re-placed in order to run on othermachines.

The rest of this paperis organizedin the fol-lowing way.

In section 3, MERLIN’s operatingsystemis

G.A.Evangelakiseta!. / MERLIN 403

presented.Therein, a description of MERLIN-modesand all MERLIN commandsis given.

In section4, the minimization algorithms aresketched.

In section 5, the structureof the usersuppliedroutinesis presented,along with an example.

In section6, some of the Merlin’s specialfea-turesare discussed.

3 MERLIN — operatingsystem

3.1. MERLIN .- files andassociatedunits

MERLIN uses the following twelve units forI/O operations:

31, 32, 33, 34, 35, 36 and 41, 42, 43, 44, 45, 46.

Units 31 and 32 are the input and output filesand when running interactively they should beassigned to the terminalby the appropriatesystemcommand. Unit 45 is a scratch file necessary forpicking recordsof information by the commandPICK describedlateron. Unit 42 is reservedfor aHELP-file. Unit 41 is the unit where the MAC-ROS are written and MERLIN assignesto it thenameMACROF. Units 33, 34, 35, 43 and 44 areassignedthe names: DATA, INIPO, BACKUP,STORE and DISPO. In thesefiles resideoneormore records,each with the following informa-tion:

The following line appearsN times,whereN isthe numberof variablesof the objectivefunction:

index,(name),fix status,value, (left margin),

(right margin).

Quantitiesin parenthesesmay or may not ap-pear.For examplethe usermay assigna nametoeach variable in which case, names will appear,also marginsmay or may not appeardependingupon whether the user wishes to frame or not,someof the variablesin a certainrange.For fixedvariablesan asteriskappearsin betweendots, inthe oppositecaseonly dotsappear.

The last line of the record contains the func-tion’s value,evaluatedat the pointdefinedby thevariable-valueslisted in the record. An example

for a three-variablecaseof such a recordis shownin table1.

The DATA file containsonly onerecord, whilefiles: INIPO, BACKUP, STOREandDISPOmaycontain noneor morethanonerecords.

File BACKUP containsinformation aboutthecurrent point and about previous points and isupdated every time a minimization routineterminatesby appendingto it a record with thecurrentpoint information.

File DATA is rewoundandupdatedevery timeprior to the executionof a minimization routine,so that it always containsinformation about thepreviousandnot aboutthe currentpoint.

File STOREcontainsinformation aboutpointsselected by the userand is updatedby appendingrecords to the end of it every time the commandMEMOis issued.

File DISPO containsinformation aboutpointswhich the user wishes to temporarily reject inorder to try processingthe minimization from adifferent point. It is updated by appendingre-cords to the end of it every time either of thecommands PICK, POINT, INIT or ACCUMareissued.Units 36 and 46 are associatedwith thecommandsREWIND and INIT only when theoptionalOPEN statementsareused.

All the aboveunits are consideredreservedbyMERLIN.

3.2. MERLIN — operatingmodes

Therearesevencategoriesof behaviorthat areglobally followed. Thesearecalled modesof oper-ation. Eachmodehasits options.The modesandthe associatedoptions are shown in table2. Thefirst option-linecontainsthe defaultoptions.

The BACKUP modeoptions,arrangethe levelof theprotectionof the intermediateresults,whichis implementedby updating the file BACKUP.For example the option FULLBACK, corre-

Table 1

1) WO 50.000000000000 50.00 70.002) AS .. s.. 87.000000000000 80.003) RO 6.6147969121794VALUE 9913.4824639511

404 G.A.Evangelakiset a!. / MERLIN

Table 2MERLIN modesandassociatedoptions

BACKUP DERIVA PRINTO USERSO CALLBY PROCES PANEL

FULLBACK NUMER FULLPRINT EXPERT INDEX IAF PANEL-ONLASTBACK ANAL HALFPRINT ROOKIE NAME BATCH PANEL-OFFNOBACK NOPRINT

spondsto updatingthat file withouterasingearlierinformation,while the option LASTBACK, corre-spondsto backing-uponly thelatestresults(earlierpointsare deleted)andtheNOBACK optionoffersno protection at all.

Changingamode option is doneby issuinganappropriatecommand(seenext section).

The DERIVA mode options instruct MERLINhow to calculate the gradient of the objectivefunction. If the NUMER option is chosen, thecalculation proceedsvia a numerical algorithm(describedin the program’s manual),while if theANAL option is chosen,MERLIN calls a user-suppliedroutine.

The PRINTO mode options arrange the levelof MERLIN’s output. The FULLPRINT optioncorresponds to the highest level (i.e. to the maxi-mum outputinformation),while the HALFPRINToption correspondsto an intermediatelevel (partsof theoutputare suppressed)and the NOPRINToption correspondsto totally suppressingthe out-put information.

The USERSO mode options set the usersophisticationlevel. In the EXPERToption, mini-mization proceedsthroughthe minimization com-mands,while in the ROOKIE option thereis onlyone minimization command (the commandAUTO) which invokes an automatic minimizer toease the operation for the inexperienced user.

The CALLBY mode options set the way ofreferencingthe parametersof the objective func-tion. In INDEX option, eachparameteris refer-encedby its index,while in the NAME option byits name. To operatein the NAME option, theuser must have previously assigned parameternames(see commandGODFATHER).

The PROCESmode options set the way oferror handling.

In IAF option MERLIN is tolerant to errors,

issuesinformative messagesand does not abort.This option is very convenient for interactiveusage.

In BATCH option, upon an error in input,executionis terminatedanda diagnosticmessageis issued.This option is to be selectedfor batchprocessing,whereuserinterventionis notpossible.

The PANEL mode options PANEL-ON,PANEL-OFF activate or deactivate the Panel-prompting(a way to input information describedlater).

Inspection of the current mode-options, isfacilitatedby the commandMODEDIS.

3.3. MERLIN — operatingcommands

The description of the MERLIN commands,reveals the capabilities of the programand clari-fies the possibilitiesofferedto the user.

MERLIN’s prompt for commandentryis:

tA/\/\/\ MERLIN IS AT YOUR COM-MAND!!!ADMINISTRATIONCOMMANDS:Mode commands:FULLBACK, LASTBACK, NOBACKThesecommandsset the BACKUP mode optionsin an obvious correspondence.ANAL, NUMERThesecommandsset the DERIVA modeoptions.FULLPRINT, HALFPRINT, NOPRINTThese commands set the PRINTO mode options.EXPERT, ROOKIEThese commands set the USERSOmode options.INDEX, NAMEThesecommandsset the CALLBY modeoptions.IAF, BATCHThesecommandsset the PROCESmode options.PANELOFF, PANELONThese commands set the PANEL modeoptions.

G.A.Eoangelakiseta!. / MERLIN 405

Display commands:MODEDISDisplayscurrentmodeoptions.SHORTDISDisplays a record of information for the currentpoint and in addition the valuesof two counters.The first counterinforms the user of the totalnumber of evaluations of the objective functionsincethe beginningof the run, the second,issuesthenumberof calls to the objectivefunction sincethelast time the RESETcommandwasused.GRADDISIf in NUMERoption, the user is prompt to enteran error-bound, required for the derivativecalcu-lation. The valuesof the first partial derivativesare thendisplayedalong with their error codes.Azero error code indicates satisfactory derivativecalculation. (Within the requiredaccuracy).If inANAL option, no further prompting takes placeandno error codesare displayed.GRAPHPermitstheuserto obtain a one-dimensionalroughgraph of the objectivefunction, versusa specifiedvariable, in a specifiedrange.A minor minimiza-tion is performed.Promptingis done via a panel.GRADCHECKChecks the values of the user-supplied derivativesagainstthosecalculatednumericallyby MERLIN.This commandworks only if the ANAL option isselected,if not, a reminder to the user is issued.The useris promptto enteranerror-boundfor thenumericalcalculationof the gradient.CATALOGLists the namesof MERLIN files that the usercan manipulate.An asteriskbelowthe file’s nameindicates that the file is being used, otherwise theindication ‘EMPTY’ appears instead.

Control commands:STOPTerminatesexecutionof theprogram.RETURNTerminatesMERLIN minimization and transfersthe control to the main program.If oneusesthemain program provided here, (Program MAS-TER) the STOP and RETURN commandsareequivalent.However the usermay call MERLINfrom his own main program, work with it, and

upon reaching at a satisfactory point to terminatethe minimization sessionandtransferthe controlback to his main programfor further processing.QUITPromptsthe user to entera valuefor MERLIN’sreturn flag output parameterIQUIT and returnscontrol to the main program. Through thisreturn-flag, additional control is offered to theuser’s main program.GODFATHERAssigns names to variables. This command whenissued for the first time, invokes prompting andthe user is expectedto enter a name for thevariable whose index is displayed. Prompting stopswhen all variablesare assignednamesor when acarriage return is enteredimmediately after theprompt.If thiscommandis usedmorethanonce,MERLINassumesthat the userneedsonly to modifyoneormore namesand prompting stops after the firstentry which may be a string of the form:indexi,namei,indexj,namej,indexk,namek,...,in-dexq,nameq and can be 79 characters long atmost.MERLIN does no checks for duplicate names sothe user should be careful while assigning them.MARGINInitiates prompting for assigningleft and rightmarginsto variables.By that is understoodthat avariable must be less or equal to its right andgreateror equalto its left margin.Thefirst promptasks for left margins the secondprompt asks forright margins. The entry to both prompts may bea string of the form:indexi,margini,indexj,marginj,...,indexk,marginkand can be 79 characters long at most.If the NAMEoption is selected for the CALLBYmode, indices are replacedby names.DEMARGINRemoves margins from specified variables. It firstprompts for removal of left margins and consecu-tively for removalof right margins.The inputstrings requiredare of the form:indexi,index2,index3 indexkIf in INDEX option andnamel,name2,name3,.. ., naniekif in NAMEoption of the CALLBY mode.FIX

406 G.A. EvangelakisetaL / MERLIN

Fixesspecifiedvariablesto their currentvalues.If option INDEX is selected for the CALLBYmode, the user is prompt to enter the indices ofthe variables to the fixed; otherwiseif optionNAMEis selected, the user should enterthenamesof the variablesinstead.The required input is as in the DEMARGINcommand.LOOSELooses previously fixed variables. Prompting issame as for the FIX command.LOOSALLLooses all fixed variables.No prompting.STEPALLCalculates search steps for all non-fixed variables.It invokes a step-size procedurethat uses thegradient of the objective function. It also performsa minor minimization. No prompting.ADJUSTCalculatessearchstepsfor all non-fixedvariables.It invokesa step-sizeprocedurethat does not needthe gradient components. Hence it is preferablewhen the gradient is either not provided by theuser or when its calculation is time consuming. Italso performs a minor minimization. Noprompting.STEPInitiates prompting for assigning values to thesearch steps for each variable. The input stringshould be of the form:indexi,stepi,indexj,stepj,...,indexk,stepk

(INDEX option)namei,stepi,namej,stepj,...,namek,stepk

(NAME option)POINTInitiates prompting for assigningvalues to thevariables. The entry is as in STEP command.Promptingstopsif eithera singlecarriagereturnisenteredon promptor a zerois enteredin placeofan index (or name).The previouspoint informa-tion is appendedin file DISPO.REWINDRewinds specified unit. Unit specification isprompt. If OPENstatements are used, a file-nameis prompt associated with unit 36.HIDEOUTAssigns MERLIN’s output to specified unit. Unitspecification is prompt. If OPENstatements are

used,connectsthe unit to the file:HIDE(unitnumber).For example,unit 16 is con-nectedto file: HIDE16.REVEALReassignsMERLIN’s outputto the defaultunit.RESETResets the partial call-counter to zero.MEMOAppends the current point information to fileSTORE.INITThis commandpermits the user to initialize thevariables(or to resetthem)with values storedinsome unit. This is done through a free-formatREAD statement,so no specific format is re-quired. It promptsthe userfor the unit number.IfOPEN statementsare used,a filenameis promptassociatedwith unit 46.ACCUMThis commandaccumulatesat random points inthe N-dimensionalspacethat result to an objec-tive function’s value less than a targetvalue. Itpromptsthe userfor the numberof desiredpoints,the maximumnumberof calls to be spentandthetarget value.To usethis commandall non-fixedvariablesmustbe framed, and values for all the variables mustexist (i.e. this commandwill not work if initiallyno valuesare assignedin all variables).The accu-mulatedpoints are storedin MERLIN’s file IN-IPO.Note that if all calls are spent and the desirednumber of points is not reached,no action istakenby MERLIN and the control is transferedto the commandpromptingpoint.INSPECTInitiates inspection of any of the files: DATA,BACKUP,DISPO,STORE,INIPO. The user isprompt to enterone of thesefilenames. The in-spectionis done recordby record of information.To interrupt inspection(in order to subsequentlyusethe PICK commandto selecta newpoint) anycharacterbesidesa spaceor a $ sign is required.PICKPicks the last point inspectedand makes it thecurrentpoint. The previouspoint is appendedtofile DISPO.PICK works only if issuedimmediately after an

G.A.Eoange!akiseta!. / MERLIN 407

INSPECT command.DISCARDDiscards any one of the files: DISPO,STORE,BACKUP.By this, it is understoodthat the specified file isrewoundand overwritten with the current pointinformation. File specificationis prompt.

INFORMATIONCOMMANDS:LISTDisplays all MERLIN commands.HELPInitiates a HELP session.INTRODUCEInforms the userof the extensions(if any) to thestandardversion of MERLIN that are imple-mentedby otherusers.EDITING COMMANDS:MACROInitiates MACROconstruction. It prompts theuser for a MACRO-nameand consecutive forMERLIN commands.A MACRO— name must always start with aperiod and should not be longer than ten char-acters.To enda MACRO constructionsessionenterthecommandCLEAR.CLEARInstructs MERLIN to terminate MACRO-con-struction session. It is an otherwiseinert com-mand.MEDITInitiates editing of the file MACROF, where inthe various MACROs reside, by invoking theMEDIT editor.The editor’s commandsare describedin the user’smanual (deactivated if the BATCH-option hasbeen selected).

MINIMIZATION COMMANDS:All minimization commands are using panel-prompting, if the option PANEL-ON is selected,andno promptingat all otherwise.The underlying algorithms are describedin thesectionthat follows.SIMPLEXInvokes the SIMPLEX method of minimization,basedon refs. [6,7].ROLL

Invokesan adhocbut robustno-derivativemethodof minimization.RANDOMInvokes a randomsearchno-derivativeroutine.BFGSInvokesa routine that usesthe BFGS formula toupdatethe inverseHessianmatrix [4].DFPInvokes a routine that usesthe DFP formula toupdatethe inverseHessianmatrix [4].CONGRAInvokesaroutine that usesthe conjugategradientmethodof Polakand Ribiere[5].The restof the commandswhich appearwhen aLIST commandis issuedare dummy commands.Thepurposeof their existenceis to easethedevel-opment a usermay wish to pursue.CommandsMIN1O—MIN2O are meant to be minimizationcommandsand updating of the files BACKUPand DATA is being performed,while commandsUSCOM13-USCOM2Oare meant to be usedasutility commands.

4. Minimization algorithms

a. The following algorithm is invoked by issuingthe commandSIMPLEX:1) Starting from the initial point (F0) calculate

initial simplex vertices P,, I = 1, N and thecorrespondingfunction values: y1 = f(F,), I =

0, N.2) Determine y10~= min(y,, i = 0, N}, y~,=

max{ y1, 1 = 0, N } and calculate (PC) thecentroid of all F1 excluding P~.Set F* = (1 + a)Fc— ctP~and calculatey~= ~( p *)

3) If y” <yi0~ then form ~** = (1.+ ..y)F* —

ypC calculate ~ y(p**) and go to (4);elsego to (5).

4) If y** <y* then replaceP,,~by ~** and goto (6).

5) If y * > y1 for all y, except yh~5hgo to (7).6) ReplaceP~,by P * andgo to (9).7) If y * <y~ replaceF~ by P~ Form P * *

= (1.— $)FC + $P~ and calculate y * *

y(p**).

408 G.A. EvangelakisetaL / MERLIN

8) If ~ >Yh~ thenreplaceF~.by(P1+P10~)/2.,i=0, N; else replace P~,byp**.

9) Check if convergence criterion is satisfied.If so return y10~, F10~as the optimum valueandpoint; elsego to (2).

Initially the simplex vertices are takenas theminima in eachof the N directionsthrough thestarting point F0. If both margins exist for allnon-fixedvariables,thereis an option to selectthesimplex vertices at random. Convergenceis as-sumedwhen:

N+1

where �, is apresettolerance.The valuesfor a, $, y are 1, 1/2, 2 as sug-

gestedin ref. [6].

b. The following algorithm is followed when thecommandRANDOMis issued.

Let the currently best point be XC = (X~,.., X~)and let f~=f(Xc) be the correspond-

ing value of the objective function. A box isconstructedin the hyperspaceof n dimensions,centeredaround the current point XC. This isfacilitatedby picking for eachvariable,a stepsize5, > 0 and the box is definedby the n intervals(X,’~—S1,X,~+S~).

A trial point Xt = (Xi, Xi,..., X,’) is sampled

at randomfrom insidethe box and is acceptedorrejectedaccordingto whetherj = f( X~)is loweror not than f~.If f(Xt) <f~the trial is successful;elseit is said to be a failure.

In an attempt to gain in efficiencywe includean optionalline searchin the directionof progressand an optional volume excluding techniqueforreshapingthe box after every failure, which isdescribed next.

Supposethat the box is definedby the set ofthe n intervals(l~~r,) and supposethat the sam-pled trial point Xt results to a failure. After thefailure the box is redefinedby the set of thefollowing n-intervals:

(ii, xt) if X<X~

(/‘, r’)= (x’ ,.) if x~>x1’

(I,, r,) if XC_.Xt

When the above option is not selected,thereshaping is done by merely multiplying the boxsides with a reducing factor, whenevera presetnumber of consecutive failures occurs. This iscalled a failure cycle. One can instruct the routineto returncontrol to the calling programif apresetnumberof consecutivefailure cyclesis reached.

c. The following algorithm is the basis of theprocedureinvoked by the commandROLL.

For eachvariable X1 thereexistsan associated

step S~.For a problem where the objective function

dependson n variables the proceduredescribedbelowis repeatedn times,as i takeson the valuesl,2,3,...,n.Let XC = (X,~,X~,...,X~)be the currentpoint

and let f~f(X~

1) Pick a trial point:

= Xf for al/f * I and

x’t=x~+s’.2) Calculate1+ =f(Xt).3) If f+’szf~set X~=Xt,f~=f÷and S

aS~proceedfrom step1) for thenext valueof

4) If f.~.>~f~pick anothertrial point as:

= ~ for all f # 1 and

Xt~XCS

5) Calculatef =f(Xt).6) If f<zf~ set X~=Xt, f~=f_and S~

— aSproceedfrom step 1) for the nextvalueof

7) if f_ ~f~calculate an appropriate stepby:

~1= —~(f÷—f_)/(f÷+f_—2f~)s,.

8) Proceedfrom step1) for thenext valueof

In the above, a, is an enhancementfactorchosenby the user.

If after looping over all variables thereis no

G.A.EvangelakisetaL / MERLIN 409

progress,a line searchis performedin the direc-tion S=(S1, S2,...,S~).

The aboveprocedureis repeateduntil a presetnumber of calls to the objective function is re-ached,whereuponcontrol is transferedto thecal-ling program.

The routineterminatesalsowhenapresetnum-berof consecutivefailuresis reached.Weconsideras a failure the casewhereeitherlooping over allvariablesor afterhavingperformedthe line search,the relative progressmade (i.e. [fi,~~aI — lanaI1/[finitial]) is lessthanapresetsmallnumber.

d. To describethe generalalgorithm followed byall the so-called quasi-Newtonmethodswe needto definea few quantities.Following Fletcher[4],let f(x) be the objective function with x =

(x1, x2,...,x~),letg be its gradientand G be itsHessianmatrix, i.e.:

3f(x) a2f(x)

g, = ~ and G,3 = ax, ax3

and let ~ = [G’J1~.

Sinceall thesemethodsare iterative,wedenotethe kth iterationrelatedquantitieswith a super-script in parentheses.

For instancex(k) is the vector x at the k thiteration, H(k) is the inverseHessianat the k thiterationetc.

Let 8(k) = — x(k) and ~(k) = g(k±l) —

g(k) Then the algorithmis as:1) Calculatethedirection s(k) = _H(k)g(k)

2) Perform a line-searchalong s~and ob-tain so 1(k+1) = +

3) Update H(k) to obtain H~””~, calculateg(k+l) go to 1.

Initially onetakesH~’~= I, theunit matrix.The updating, is the operationthat distinguishesbetweenmethods.For the DFP methodthe updatingformula is:

~.ô. H.yyH.H(~~~+~)= H. + ~ — ip p q qj

‘~ 8 H ‘

mim Im mnlnwhile for the BFGSit is given by:

“H 8.8.H(k+l)=H + 1+ m mnn ‘IU If 87

— ~ + H,~y~836v.yU

Where in both caseswe used the summationconventionover repeatedindices andwe droppedthe superscriptk on the quantitieson the righthandside.

e. The conjugategradientmethodsdo not usetheHessianmatrix, theyuse only the gradientof theobjectivefunction.Theycreatedirectionsof searchconjugate to each other. Among the many al-gorithms of that type the one due to Polak andRibierehasdisplayeda staggeringimprovementinperformancewhen solving problemswith a largenumberof variables.

A sketchof the algorithmfollows:At the k th iteration:

1) Calculate: S~’~’~= _g(k+l) + /3~s~’~,where:p(k) = (g(k+~)— g(k)) .g(k+l),/g(k) .g(k)

2) Perform a line-searchalong the direction~ obtainingso thenew point, calculatethe newgradientandgo to 1.

Initially we take the steepestdescentdirection:5(1)= _g(l)•

5. User-suppliedroutines

A) SUBROUTINEGRANAL(N,X,VAL,GRAD)INPUT: N,X,VALOUTPUT: GRADX is a realarraydimensionedas X(N).X(N) containsthe valuesof the variables.VAL the valueof the objectivefunction.GRAD is a real array dimensioned as

GRAD(N).GRAD(N) upon return, should contain the

componentsof the gradient.

This subroutineis optional. MERLIN calcu-lates the gradient numerically. If however, thepartial derivativesare analytically known or theycan be computedin some otherconvenientway,the user should take advantageof it. The abovesubroutine,whenprovided,is meantto serve thispurpose.

410 GA. EvangelakisetaL / MERLIN

B) FUNCTION FUNMIN(X,N)INPUT: N,XX is an array, dimensionedas X(N) and con-tains the values of the variables.N is the di-mensionalityof the problem.This function subprogramis the implementa-

tion of the objective function i.e., upon returnFUNMIN(X,N) shouldbe equal to the objectivefunctionevaluatedat the input point X(N):

As an example consideras objective functionthe following:

x2, x3)=(x1—3)2+5x~(x

3—x1)4

+10x~(10O— x1x3)

2

with partial derivatives:

af/ax, = 2(x1 — 3) — 2Ox~(x,— x,)’

—20x~(100— x1x,),

4al/ax2 = 10x2(x, — x1)

af/ax, = 20x~(x3— x1)’ + 20x,(100— x,x,)2

—20x1x~(100— x,x,).

The two routinesappearingin table 3, imple-ment the objective function and the gradientcalculation.

Table3

As one can see from this example, the VALinput-argumentto the GRANAL subroutinemayor may not be used.Its presenceservesthe pur-pose of saving additional calls to the objectivefunction, if in the calculationof the gradientthefunction’svalueis needed.

6. Specialfeatures

Merlin provides a friendly andconvenienten-vironment to work with, and at the sametime,powerful optimization routines. Our attitude,basedon the fact that theredoesnotexist a singlealgorithmto be usedin all casesas a panacea,isthat the user should be able to try easily thevarious different methods, in order to find theonesmostsuitableto his problem.Very importantis also the capability to storepointsconsideredtobe potentially useful and pick points that havepreviouslybeenstored.

In caseswherethereexist a lot of local minimainsidea known region, onemay be ableto reachthem by starting from different initial points. Soaccumulatingvarious initial points at randominthe specified region that correspondto objectivefucntion’s values lower than a user-set upperbound,canbe quitehelpful.

FUNCTION FUNMIN(X,N)DIMENSION X(N)X1=X(1)X2 = X(2)X3=X(3)FUNMIN= (X1—3) s * 2+5 * X2 * * 2* (X3—X1) * * 4+10 * X3 * * 2 * (100—X1 * X3) * * 2RETURNEND

SUBROUTINEGRANAL(N,X,VAL,GRAD)DIMENSION X(N),GRAD(N)Xl = X(1)X2 = X(2)X3 = X(3)GRAD(1)= 2 * (Xl —3)—20 * X2 * * 2 * (X3 — Xl) * * 3—20 * X3 * * 3 * (100—Xl * X3)GRAD(2)=10 * X2 *(X3—X1)* * 4GRAD(3)= 20 * X2 * * 2 * (X3—X1) * * 3+20 * X3 *(1()0—X1 * X3) * * 2—20 * X3 * * 2

> * Xl * (100—X1 * X3)RETURNEND

G.A.EvangelakiselaL / MERLIN 411

Creation of command packages(Macros) isquite convenientand important especiallywhenonedealsoftenwith thesametype of functions.Ina macroonecan storeasuccessfulstrategyfor thetype of problems he usually faces and use it overand over. A collection of such macros is thereforeusefulsincethey acceleratethe processof minimi-zation.

The input operationsare handledin a naturalway which wecall the“one-line-input”. Themaxi-mum length of the input recordis 79 characters.

Whenonly variablespecificationis required(asfor examplein the FIX command)oneenterstheindices (or names)of the associatedvariables,separatedby blanks or commas.

When assigningvaluesto variables (as for ex-ample in the STEP command), the input lineconsistsof pairs, containing first the index (orname)of the variable,and then thecorrespondingvalue.

For the minimization routines (as well as forthe graph-routine) the input is assisted by panels.The panelsshow a list of variables(that are inputto the subprogram to be called) along with their

Table4

current values, a short explanation of their roleand an indicative menu. These variables do nothave names but they are associated with indicesthat are displayed on the panel. To change theirvalues, one follows the philosophy described aboveof the one-line-input. Representative panels aresown in table 4.

A full description of all Merlin’s special fea-turescanbe found in the user’smanual.

Merlin has been successfullyused in severalcalculations. As examples we refer to X-ray andionic conductivity datafits to thetheoreticalmod-els, study of nuclear interactions via potentialmodeling, Compton profile calculations,solutionof linear systemswhenothermethodssuffer fromnumericalinstabilities,studyof chemicalreactionsandreactionpathfitting, variationalcalculationofthe ground state for simple systemsetc. We listcollectively under reference[8], severalworks inwhich MERLIN was usedto solve the relevantoptimizationproblemsthat came-up.

In the nearfuturewe intent to presenta specialversionof this programappropriateto CDC-mac-hinesthat will takeadvantageof the NOS operat-

RANDOM-PANEL

INDEX DESCRIPTION VALUE MENU1) NUMBEROF CALLS 1000 ANY INTEGER2) ACTIVATE/DEACTIVATE

VOLUME EXCLUSION0 (0/1)

3) STEPFACTOR 0.70 ANY REAL IN (0,1)4) CYCLE-SIZE 30 ANY INTEGER5) NUMBEROF CONSECUTIVE 5 ANY INTEGER

CYCLE FAILURES ALLOWED6) LINE-SEARCHPERIOD 3 ANY INTEGER7) PRINTOUTSELECTION 1 0/1/28) CANCEL-BUTr0N 1 (0/1)

ENTERCHANGES?1300030.4420565

SIMPLEX-PANEL

INDEX DESCRIPTION VALUE MENU1) INITIALIZATION SCHEME

(SYSTEMATIC/RANDOM)1 (1/2)

2) INITIALIZATION TOLERANCE 0.0 ANY REAL3) INIT. CALLS/VARIABLE 100 ANY INTEGER4) SIMPLEXTOLERANCE 0.0 ANY REAL5) SIMPLEX CALLS 1000 ANY INTEGER6) PRINTOUTSELECTION 0 (0/1/2)7) CANCEL-BUrFON 1 (0/1)

ENTERCHANGES?2 0.001 3 50 5 500

412 G.A.Evange!akiset aL / MERLIN

ing systemthroughnon-standardFORTRANandassemblylanguageroutines[9]. This will be quiteconvenientsince it will permit issuing of NOScommandsfamiliar to the CYBER users,will con-tain additional file manipulationcommands,pri-ority controletc.

A MERLIN control languageis currently un-der intensivedevelopment[10], which will enablethe user to program MERLIN according to hisparticularneeds.

Finally, we report that following the installa-tion instructionsgiven in the user’smanual,wewere able to run this programon APPLE’s 512kMacintosh, using the Microsoft FORTRAN’77compiler.

Acknowledgements

We would like to acknowledge the contribu-tions made in the course of the developmentofthis work by severalcolleaguesand friends. Spe-cial thanksare dueto Dr. I. Demetriou,V. Vout-sadakisand D. Papageorgioufor illuminating dis-cussions,Dr. T. Bakas,Dr. C. Kotsis,D. Katsanos,C. Hasapis,T. LiakopoulosandC. Hatzigeorgioufor useful suggestionsafter running innumeroustimesandreadingthe severalpreliminaryversionsof the manuscript, the computercenter of theUniversity of Ioanninafor generousprovision ofcomputer time on a CDC-CYBER 171 and toprofessorsN.G. Alexandropoulosand C. Sakare-los for constantsupport and encouragement.Wealso thank Th. Fresta,for carryingout efficiently

the difficult task of typing the various differentversionsof the manuscript.

References

[1] T. HamadaandI.D. Johnston,Nucl. Phys.34 (1962)382.R.V. Reid, Ann. Phys.50 (1968) 411.M. Laccombe,B. Loiseau,J.M. Richard,R. Vinh Mau, J.Cote, P. Pires and R. deTourreil, Phys.Rev. C21 (1980)861.I.E. Lagaris andV.R. Pandhanpande,Nucl. Phys. A359(1981)331.R.B. Wiringa, R.A. Smith andT.L. Ainsworth,Phys.Rev.C29 (1984) 1207.

[2] N. Sathyamurthy,Comput.Phys.Rep.3 (1985)1.S. Bell andJ.S. Crigliton, Chem. Phys.80 (1984)2465.

[3] F. James and M. Roos, Comput. Phys. Commun. 10(1975)343.

[4] R. Fletcher, Practical Methods of Optimization (JohnWiley, New York).R. FletcherandM.J.D. Powell,ComputerJ. 6 (1963)163.

[5] E. Polak, ComputationalMethods in Optimization: AUnified Approach(AcademicPress,NewYork, 1971).

[6] J.A.Nedlerand R. Mead, ComputerJ. 7 (1965)308.[7] K. Mika and Th. Chaves,Berichte der KFA JUlich No.

1643 (1980).[8] N.I. Papanicolau,N.C. Bacalisand D.A. Papaconstanto-

poulis, Z. Phys.B 65 (1987)453.G. PantisandI.E. Lagaris,Z. Phys.A 321 (1985) 149.IN. Demetropoulos,S.C. Plaskasovitisand I.E. Lagaris,Abstracts, 2nd Intern. Symp. on Kinetics in AnalyticalChemistry,P111-18(1986).C. Kotsis, DoctoralDissertation,Universityof loannina(1986).

[9] D.G. Papageorgiou,C.S. Hassapisand I.E. Lagaris, inpreparation.

[10] C.S. Hassapis,D.G. Papageorgiouand I.E. Lagaris, inpreparation.

G.A.EvangelakisetaL / MERLIN 413

INPUT DECK

3POINT

1 30 2 30 3 33.660

1 ROLL2 501600203 .54 SIMPLEX5 520006 .5

1415161?161920212223242526

ENTER * OF VARIABLES

MERLIN — 1.0 *n ST~C~LERSIONG.A.ELflIOELAKIS,J.P.RIZOS, I .E.LAGRRIS

PIP/S iCS DEP%ITPENTI .N.DEPETROPOLLOS

DEPARTMENTOF CHEMISTRY1986UNIVERSITY OF IOAFIIINA. OCTOBER

IOANNINA — 0 R E E C E

ANOTHER MINIMIZATION SESSION STARTS.THERE IS A HELP FACILITY ASSOCIATED WITH AHELP—FILE ASSIGNED TO UNIT 42MAKE SURE THIS FILE IS PRESENT TO PROUIDE ON—LIME INFORMATION

ESTIMATED MACHIME~SACCURACY: 1.E—14

WARMINGINITIALIZE VARIABLES

PUI NTPOINT

GODFATHERVARIABLE 1VARIABLE 2VARIABLE 3

I) .

2)3)

xVZ

/V~/\/VV\ MERLINMACRO** BITER MACRO- NAtE **** MACROEDITING STARTS **** ENTER DESIRED COMMflC **

** ENTER DESIRED COMMAND**** MACROEDITING COMPLETE**

IS AT YOUR COItSV

600FATHER

V

MACRO.5SHORTOISCLEAR.9

OUTPUT

7 ANAL8 8F659 1200060

10 OFP11 1 2000 6 012 .513 STOP

MERLIN ISATYOURCOI1~ID ii

MERLIN IS AT YOURCOttflC ii!

414 G.A. EvangelakisetaL / MERLIN

/\/\/\/\/\/\ MERLIN IS AT YOUR COMMAND if!SHORTDISTOTAL MU. OF CALLS = 1MU. OF CALLS SINCE LAST RESET I

I) X 30.0000000000002) V 30.0000000000003) Z 33.8$0000000000

VALUE...... 9640575114.3915

5*5 END OF MACROPROCESSINGas

/\/\/\/\/\/\ MERLIN IS AT YOURCOMMANDI!!ROLL

ROLL — PANEL

VALUE300100E-02

30E+O 1

3

INDEX DESCRIPTION1) NUMBER OF CALLS2) TOLE~’ICE3) STEP FACTOR4) FAILURES ALLOWED5) PRiNTOUT SELECTION5) WALL—PARAMETER7) CANCEL—BUTTON

ENTER CIWIGESNUMBEROF ROLL

/\/\/\/‘¼/\/~ MERLINSHORTDISTOTAL Mi. OF CALLS — 802lii. OF CALLS SINCE LAST RESET 802

U X 37.9502407070302) V —.56843418860808E—133) Z 2.6351484637018

VALUE 1221 .5207432240

as END OF MACRO PROCESSINGas

MERLIN iS AT YOURCOIIIAND if!

SIMPLEX — PANEL

INDEX DESCRIPTION UALUE1) INITIALIZATION SCHEME

(SVSTEPIAT I C/RANDOM)2) INITIALIZATION TOLERANCE 100E—013) INIT. CALLS/VARIABLE 1004) SIMPLEX TOLERANCE 0.5) SIMPLEX CALLS 1006) PRINTOUT SELECTION U7) CANCEL—SUTTON 1

BITER CFRIGESNUMBEROF SIMPLEX CALLS: 2118

MENUANY INTEGERANY REAL IN (0,1)ANYREAL’ IANY INTEGER0/1/2ANY INTEGERC 0/1 )

MENU(1/2)

ANY REALANY INTEGERANY REALANY I NTEGER

(0/1/2)C 0/1 )

CALLS:

IS AT YOUR

801

iii

Si MPLEX

GA. EvangelakisetaL / MERLIN

/\/\/\/‘¼/~¼/\ MERLIN IS AT YULE COMMANDI!!SHORTDISTOTAL Mi. OF CALLS = 2920NV. OF CALLS SINCE LAST RESET 2920

1) X 3.87166844040212) V —.56843418860808E-133) Z 25.828400351577

VALUE .76644352226356

as END OF MACRO PROCESSING as

/V\/\/\/\/\ MERLIN IS AT YOUR COMMAND I!!ANAL

INDEX DESCRIPTIONI) NUMBER OF CALLS2) TOLERANCE3) ERROR—BOUt)4) USE/RECALCULATE GRADIENT5) USE/RESET HESSIAN6) PRINTOUT SELECTION7) WALL—PARAMETER8) CANCEL—SUTTON

ENTER C*ENGESNUMBER OF DFP CALLS: 1188

/\/\/\/\/\/\ MERLIN IS AT YOUR COtIIWIO ~‘!SHORTDI S

TOTAL MU. OF CALLS — 6109MU. OF CALLS SINCE LAST RESET 6109

I) X 3.00000000001322) V 21351172413793E—143) Z 33.333333333187

VALUE . 19321 191098585E—21

as END OF MACRU PROCESSINGas

/\/\/‘c/\/\/\ MERLIN IS AT VOLE COtTH’I) U!

415

BFGS/VV\/\/\/\ MERLIN IS AT VOLE cottac w

SFGS-PAFEL

INDEX DESCRIPTION VALUE1) NUMBER OF CALLS 3002) TOLERANCE .IOOE—023) ERROR—SOUND .IOE—014) USE/RECALCULATE GRADIENT 05) USE/RESET HESSIAN 06) PRINTOUT SELECTION I7) WALL—PARAMETER 38) CANCEL—SUTTON I

MEltANY INTEGERANY REAL IN (0,1)ANY REAL 1(1/0)(1/0)0/1/2ANY I NTEGERC 0/1 )

ENTER C1ENGESNLEEROF BFGS CALLS: 2001

OFP/V\/\/\/\/\ MERLIN IS AT YOUR COMMAND if!

OFP-PRtEL

VALUE MENU300 ANY INTEGER

.100E—02 ANY REAL IN (0,1).IOOE—01 ANY REAL IN (0,1)

0 (1/0)0 (1/0)1 0/1/23 ANY INTEGERI (0/I)