New features of Mosel in Release 7 - fico. · PDF filec 2013 Fair Issac Corporation....

The material in this presentation is the property of Fair Issac Corporation, is provided for the recipient only, andshall not be used, reproduced, or disclosed without Fair Isaac Corporation’s express consent.

c©2013 Fair Issac Corporation.

New features of Mosel in Release 7.5

Y. Colombani, S.HeipckeXpress Team, FICO

http://www.fico.com/xpress


Xpress-Mosel


Xpress-Mosel

» A high-level modeling language combinedwith standard functionality of programminglanguages» implementation of models and solution algorithms in

a single environment

» Open, modular architecture» extensions to the language without any need for

modifications to the core system

» Compiled language» platform-independent compiled models for

distribution to protect intellectual property


...and also

» Mosel modules» solvers: mmxprs, mmquad, mmxslp, mmnl, kalis» data handling: mmetc, mmodbc, mmoci» model handling, utilities: mmjobs, mmsystem» graphics, GUI: mmive, mmxad, mminsight

» IVE: visual development environment(Windows)

» Library interfaces for embedding models intoapplications (C, Java, C#, VB)

» Tools: debugger, profiler


Mosel modules

QP

MIPMIQP

QCQPMI

IVE

XAD

SLP

MISLP

Kalis

LP

Mosel

Extensions

MIPQCQP

Userextension

ODBC,OCI, XML,

CSV,...Insight

Utilities

Modeling platform

GUI

Data interfaces

Solvers

System,mmjobs


Highlights ofMosel development

» Launch in November 2001» Replacement of mp-model (Dash’s modeling

language since 1985)

» Mosel 1.2 (September 2002)» Open access to Native Interface

» Mosel 1.4 (April 2004)» Generalized file handling, I/O drivers; 64bit

» Mosel 1.6 (April 2005)» Parallelism / multiple models; debugger and profiler



» Mosel 2.0 (April 2007)» Data structures list and record ; definition of user

types; packages (Mosel libraries written with Mosel)

» Mosel 2.2 (October 2007)» Enhancements to initializations functionality

(evaluation of )

» Mosel 2.4 (July 2008)» Module mmnl; creation of arrays "on the fly" (array )



» Mosel 3.0 (July 2009)» Handling of multiple problems within a single model;

mmxprs: indicator constraints; kalis: automatic linearrelaxations

» Mosel 3.2 (November 2010)» Framework for distributed computing; handling of

matrices with more than 231 coefficients

» Mosel 3.4 (April 2012)» Mosel remote launcher (XPRD)


New features of Mosel in Release 7.5

» Secure connections via SSH» mmsheet: Spreadsheet drivers» mmxml: XML handling» Miscellaneous


Secure connections via SSH

» The xprmsrv server now accepts secureconnections through an SSH tunnel (inaddition to standard connections)

» mmjobs and XPRD define a new driver xssh(secure version of xsrv) to connect securely toan xprmsrv server without any additionalsoftware or settings

» Optionally, a client can authenticate the serveron connection


Secure connections via SSH

» Documentation:» Chapter mmjobs in the Mosel Language Reference» XPRD: Mosel remote invocation library reference

manual


mmsheet: Spreadsheet drivers

» The excel driver is now part of mmsheet» usage: mmsheet.excel:myfile.xls

» New I/O drivers:» xls — XLS files» xlsx — XLSX and XLSM files» csv — CSV format files

NB: The xls and xlsx drivers do not depend onExcel (including under Windows)

» Documentation: Chapter mmsheet in theMosel Language Reference


mmsheet: Spreadsheet drivers

» xls and xlsx have the same format and optionsas the excel driverinitializations from "mmsheet.xls:mydata.xls"

[A,B,C] as "MyRange" ! Named rangeD1 as "[Sheet1$B3:E4]" ! Cell referenceD2 as "[Sheet1$R3C2:R4C5]" ! Row-column notationA2 as "MyRange(#1,#4)" ! Selecting columns

end-initializations

» csv defines some additional formattingoptions and cannot use named rangesinitializations from "mmsheet.csv:dsep=,;fsep=;;mydata.csv"

D1 as "[B3:E4]" ! Cell referenceD2 as "[R3C2:R4C5]" ! Row-column notationA2 as "[B3:E4](#1,#4)" ! Selecting columns

end-initializations


Comparison of spreadsheet drivers

excel xsl/xslx csv

File type physical file physical file extended file

Platforms Windows Windows, Linux, Mac all Xpress platforms

Requirements Excel+open inter-active session

none, can beused remotely

none, can beused remotely

File creation no yes yes

Output writingmechanism

on-screen displaywithout savingif applicationrunning, other-wise data savedinto file

data savedinto file

data savedinto file

Named ranges yes yes no

Multiple sheets yes yes no

VB macros yes yes no


mmxml: XML handling

» The module mmxml provides an XML interfacefor the Mosel language.» new type: xmldoc represents an XML document» each node/element in the document is identified by a

node number (an integer) that is attached to thedocument (i.e. a node number cannot be shared bydifferent documents)

» Documentation: Chapter mmxml in the MoselLanguage Reference


Example: Reading XML format data

» Data file folio.xml :

<portfolio ><share name="treasury" ret ="5" dev ="0.1" country ="Canada"

region ="NA" risk ="low" /><share name="hardware" ret ="17" dev ="19" country ="USA"

region ="NA" risk ="high" />...<share name="electronics" ret ="21" dev ="16" country ="Japan"

region ="Asia" risk ="high" /></portfolio>


Example: Reading XML format data

declarationsSHARES: set of string ! Set of sharesRISK: set of string ! Set of high-risk values among sharesRET: array (SHARES) of real ! Estimated return in investment

AllData: xmldoc ! XML documentNodeList: list of integer ! List of XML nodes

end-declarations

! Reading data from an XML fileload(AllData, DATAFILE)

getnodes(AllData, "portfolio/share" , NodeList)RISK:= union (l in NodeList | getattr(AllData,l, "risk" )= "high" )

{getstrattr(AllData,l, "name" )}forall (l in NodeList)

RET(getstrattr(AllData,l, "name" )):= getintattr(AllData, l, "ret" )


Example: Outputting data in XMLformat

declarationsSHARES: set of string ! Set of sharesfrac: array (SHARES) of mpvar ! Fraction of capital used per share

ResData: xmldoc ! XML documentShare,Root,Sol: integer ! XML nodes

end-declarations

! Create solution representation in XML formatRoot:=addnode(ResData, 0, XML_ELT, "result" ) ! Create root nodeSol:= addnode(ResData, Root, XML_ELT, "solution" ) ! Add "solution" nodeforall (s in SHARES) do

Share:=addnode(ResData, Sol, XML_ELT, "share" ) ! Add node to "solution"setattr(ResData, Share, "name" , s) ! ...with attr. "name"setvalue(ResData, Share, frac(s).sol) ! ...and solution value

end-do

save(ResData, "result.xml" ) ! Save solution to XML format filesave(ResData, Sol, "" ) ! Display XML format solution on screen


Example: Outputting data in XMLformat

» Generated output file result.xml :

<?xml version ="1.0" encoding ="ISO-8859-1" standalone ="yes" ?>

<result ><solution >

<share name="treasury" >0.3 </share><share name="hardware" >0</share>...<share name="electronics" >0</share>

</solution></result>


Example: XML paths

declarationsDB: xmldocEmployees, AllEmployees, Names: list of integer

end-declarations

! Check a property of a text node (start from document root)getnodes(DB, "persList/employee/language[position()=3]/.." , Employees)forall (p in Employees) save(DB, p, "" )

! Check a property of a text node (start path from a node)getnodes(DB, "persList/employee" , AllEmployees)forall (n in AllEmployees)

getnodes(DB, n, "./name[starts-with(string(),’J’)]/.." , Employees)

! Check existence of an attributegetnodes(DB, "persList/employee[@parttime]" , Employees)writeln( "Number of part-time workers: " , Employees.size)

! Check a specific attribute valuewriteln( "Employee with id=T345: " ,

getvalue(DB, getnode(DB, "persList/employee[@id=’T345’]/name" ))


And also

» mmxprs: addmipsol + solnotify callback» see example model

mosel/WhitePapers/MoselPar/jobshopas.mos

declarationsheursol: dynamic array ( set of mpvar ) of real

end-declarations

public procedure solnotify(id: string , status: integer )writeln( "Optimiser loaded solution " ,id, " status=" ,status)

end-procedure

addmipsol(id,heursol)setcallback(XPRS_CB_SOLNOTIFY, "solnotify" )

setparam( "XPRS_HEURSEARCHROOTSELECT", 31) ! Configure MIP heuristicssetparam( "XPRS_HEURSEARCHTREESELECT", 19) ! to use provided solutions


And also

» New sizing options for mem and shmem I/Odrivers (including incremental resizing)» mem: see Mosel Libraries Reference» shmem: see Chapter mmjobs in the Mosel Language

Reference

⇒ addresses efficiency problems on Windowsplatforms


Tips and Tricks


Tip 1: Adding a file to a compressedTAR archive

declarationslsf: list of textorigfname: text

end-declarations

makedir(gettmpdir+ "/tartemp" ) ! Create temporary directoryif getfstat(archivename)<> 0 then ! Untar if archive exists

untar( "zlib.gzip:" +archivename, gettmpdir+ "/tartemp" )end-if

! Copy file to temporary directoryorigfname:=pathsplit(SYS_FNAME,filetoadd)fcopy(filetoadd, ":" + gettmpdir+ "/tartemp/" +origfname)

! Rebuild the archivefindfiles(SYS_RECURS, lsf, gettmpdir+ "/tartemp" , "*" )newtar( 0, "zlib.gzip:" +archivename, gettmpdir+ "/tartemp" , lsf)

removefiles(SYS_RECURS, gettmpdir+ "/tartemp" , "*" )removedir(gettmpdir+ "/tartemp" ) ! Delete temporary files


Tip 2: Inserting a time stamp into afile name

public procedure addtimestamp(filestochange: set of string )declarations

origdir, origfname, origextn, timestmp: textend-declarations

! Create a time stamp - redefining datetime format as to avoid use! of ’:’ to prevent interpretation as I/O driver

usedfmt:=getparam( "datetimefmt" ) ! Save present formatsetparam( "datetimefmt" , "%0d%0m%0yT%0H.%0M.%0S")timestmp:= text(datetime(SYS_NOW))setparam( "datetimefmt" , usedfmt) ! Reset to previous format

! Rename all files to include the time stamp in their nameforall (f in filestochange) do

origdir:=pathsplit(SYS_DIR,f)origfname:=pathsplit(SYS_FNAME,f)origextn:=pathsplit(SYS_EXTN,origfname,origfname)fmove(f, origdir + "/" + origfname + "-" +timestmp + "." +origextn)

end-doend-procedure


Tip 4: Working with packages

» Package = Mosel library written in the Mosellanguage (as opposed to Mosel modules thatare implemented in C)» similar structure as models, keyword model is

replaced by package» included with the uses statement» compile in the same way as Mosel models, place BIM

on DSO search path» package name = name of the BIM file (package.bim)» definition of new subroutines, constants, and types

» Alternative to packages: include insertscontents into a model to be compiled with themodel


Tip 4: Working with packages

» File toolbox.mos (compile to toolbox.bim )package toolbox

uses "mmsystem"version 12.08 . 30

public procedure addtimestamp(filestochange: set of string )...end-procedure

! Overloaded version with different argumentspublic procedure addtimestamp(filetochange: string )

addtimestamp({filetochange})end-procedure

end-package

» Use package in a Mosel model:uses "toolbox"


Tip 21: Using assert to validateinput data

» The assert statement serves for checkinginput data (including model parameters) forcorrect types and values» by default, assert is only executed for models

compiled in debug mode (flages ’g’ or ’G’), thisbehaviour is changed by specifying optionkeepassert

» the default error code returned by ’assert’ is 8, analternative value can be specified as the optionalthird argument


Tip 21: Using assert to validateinput data

options keepassert ! Always apply ’assert’

parametersA=10DATAFILE="mydata.dat"

end-parameters

assert(A in 1.. 20, "Wrong parameter value" )(! Same as:

if A not in 1..20 thenwriteln("Wrong parameter value")exit(8)

end-if!)

! If file not found, return exit code 5assert(getfstat(DATAFILE)= 0, "Data file not found" , 5)


Tip 16: Dates and times

» mmsystem defines types date , time , anddatetime for handling date and time types» conversion to/from numerical values, e.g., for use as

indices (getasnumber )

» Data connector modules mmsheet, mmodbcand mmoci support these types for readingand writing data» representation of date and time information within

databases may not be compatible with Mosel’sdefault format⇒ adapt format settings


Tip 16: Dates and times

declarationsd: integerT: timeDates: array ( 1.. 5) of date

end-declarations

writeln(datetime(SYS_NOW)) ! Datetime returned by system

setparam( "timefmt" , "%0H:%0M:%0S") ! Set new time formatsetparam( "datefmt" , "%y-%0m-%0d" ) ! Set new date format

! Convert date created from string to number (JDN)d:= getasnumber(date( "2012-12-25" )))

initializations from "mmodbc.odbc:datetime.mdb"T as "Time1"Dates as "noindex;Dates"

end-initializations


Tip 17: Calculating the calendarweek number

» Function getweek below returns the calendarweek count for a given date.» implementation: getasnumber calculates the Julian

day number and getweekday returns the week daynumber for a date

» usage: wnum:= getweek(date(2012,2,29))

function getweek(d:date): integerfirstday:=date(getyear(d-getweekday(d)+ 4), 1, 3)returned:= (getasnumber(d) - getasnumber(firstday) +

getweekday(firstday+ 1)+ 5) div 7end-function


Tip 12: Indicator constraints

» Indicator constraint = global entity thatassociates a binary variable b with a linearconstraint C» models an implication:

’if b = 1 then C’, in symbols: b→ C, or’if b = 0 then C’, in symbols: b̄→ C(the constraint C is active only if the condition is true)

» ⇒ Use indicator constraints for theformulation of logic expressions in the placeof big-M constructs


Tip 12: Indicator constraints inMosel

» Need a binary variable (type mpvar ) and alinear inequality constraint (type linctr )

» Specify the type of the implication (1 forb→ C and -1 for b̄→ C)

» The subroutine indicator returns a newconstraint of type logctr that can be used inthe composition of other logic expressions


Tip 12: Indicator constraints inMosel

uses "mmxprs"

declarationsC: linctrL: logctrx1, x2, b: mpvar

end-declarations

b is_binary ! Variable for indicator constraints

C:= x2<= 5

L:= indicator( 1, b, x1+x2>= 12) ! b=1 -> x1+x2>=12indicator( -1 , b, C) ! b=0 -> x2<=5C:= 0 ! Delete auxiliary constraint definition


Tip 13: Logic constructs

» Package advmod defines type logctr fordefining and working with logic constraints inMIP models

» Implementation of these constraints is basedon indicator constraints

» Build logic constraints with linear constraintsusing the operations and , or , xor , implies ,and not

» Must include the package advmod instead ofthe Optimizer library mmxprs


Tip 13: Logic constructs

uses "advmod"

declarationsx: array ( 1.. 3) of mpvar

end-declarations

implies(x( 1)>= 10, x( 1)+x( 2)>= 12 and not x( 2)<= 5)

declarationsp: array (R) of mpvar

end-declarations

forall (i in R) p(i) is_binary

! Choose at least one of projects 1,2,3 (option A)! or at least two of projects 2,4,5,6 (option B)

p( 1) + p( 2) + p( 3) >= 1 or p( 2) + p( 4) + p( 5) + p( 6) >= 2

! Choose either option A or option B, but not bothxor(p( 1) + p( 2) + p( 3) >= 1, p( 2) + p( 4) + p( 5) + p( 6) >= 2)


Tip 14: Deployment via an Excel VBmacro

» Generate VB code using the IVE deploymentwizzard» menu Deploy� Deploy ..., select Visual Basic, then

confirm with Next» copy the displayed VB code into the clipboard

» Generate the BIM file» Create a macro-enabled spreadsheet (.xlsm)



» Select Excel menu Developer� Insert» select the button object and position it using the

mouse» assign a new macro to the button and paste the VB

code from clipboard into the macro

» Using Excel menu Developer� Visual Basic�File� Import File... add xprm.bas from theinclude subdirectory of Xpress to the project

» Select the button with right mouse key to editits text



» If you do not see the Developer menu:

» Excel 2007: click on the round button top leftthen select Excel Options; under the Popular section,check the option Show Developer tab

» Excel 2010: on the File tab, choose Options, thenchoose Customize Ribbon and in the list of Main Tabs,select the Developer check box:



» Output redirection to spreadsheet:» Define a cell range Output» Edit the macro (after XPRMinit):

Call XPRMsetdefstream( 0, XPRM_F_WRITE, XPRM_IO_CB(AddressOf OutputCB))...Public Function OutputCB(ByVal model As Long, ByVal info As Long,

ByVal msg As String, ByVal size As Long) As Long’ Process windows messages so Excel GUI respondsDoEvents

’ Strip the extra newline character and print to sheetRange( "Output" ).Cells( 1, 1) = Mid(msg, 1, Len (msg) - 1)

End Function

The material in this presentation is the property of Fair Issac Corporation, is provided for the recipient only, andshall not be used, reproduced, or disclosed without Fair Isaac Corporation’s express consent.


www.fico.com/xpress

New features of Mosel in Release 7 - fico. · PDF filec 2013 Fair Issac Corporation....

Documents

Transcript of New features of Mosel in Release 7 - fico. · PDF filec 2013 Fair Issac Corporation....