A1 - Introduction to Python - Gurobi · •Gurobi Interactive Shell is a Python shell with Gurobi...

Introduction to Python

Quick reminder about questions

Copyright 2017, Gurobi Optimization, Inc.

Please ask them!

A few words about Gurobi

Gurobi algorithms

• Continuous optimization• LP: primal and dual simplex, sifting, parallel barrier, concurrent• QP: primal and dual simplex, parallel barrier• QCP: parallel barrier

• Discrete optimization (MIP, MIQP, MIQCP)• Parallel branch-and-bound• Relaxations solved with continuous algorithms• Cutting planes and heuristics

• General• Automatic presolve• Identifying irreducible inconsistent subsystems (IIS)• Automated parameter tuning tool• …


Interfacing with Gurobi

• Command-line tool

• Full-featured interactive shell

• Matrix-oriented APIs• C, MATLAB, R

• Object-oriented APIs• C++, Java, .NET, Python

• Modeling systems• Commercial: AMPL, Frontline Solvers, …• Free: JuliaOpt, PuLP, Pyomo, SolverStudio, …


Getting started with Python

Python in a nutshell

• Python is a powerful programming language

• Very easy to learn• Runs on every major operating system• Suitable for beginners and professional developers• Clean, simple and compact syntax• Open Source community: Tons of ready-made libraries and frameworks

• Python is also a great choice for mathematical modeling

• Many popular packages for scientific computing• Jupyter Notebooks: Interactive graphical environment• Language syntax looks similar to mathematical notation

Copyright 2017, Gurobi Optimization, Inc. 7

Why Python?

• Gurobi provides a variety of programming interfaces• Python• Java• C++• .NET (C#, Visual Basic, …)• MATLAB,• R• ...

• Gurobi is committed to all of these interfaces

• This presentation focuses on Python since it is easy to learn and use


Python installation

• The Gurobi Optimizer installation includes a small Python 2.7 distribution with a basic set of packages.

• Gurobi Optimizer works with multiple Python distributions (including http://www.python.org).

• In this presentation, we use Anaconda Python

• Download from http://www.gurobi.com/downloads/get-anaconda• Windows, Mac, Linux; 32/64-bit• All basic packages in the default installation• Easy-to-use package management (conda)• Gurobi provides/maintains Anaconda packages• Python 2.7 (recommended) and 3.5 (latest)


Basic concepts

• Python is an “interpreted language”• No need for a manual compiling step before running a program.• Performance optimization is done in the background (byte-code compilation/execution).

• Two basic ways to use Python• Put your code in a text file (e.g. example.py) and run the command “python example.py”.• Start the Python Interactive Shell (command “python”) and enter commands line by line.

• In this presentation we will also use Jupyter Notebook• A rich graphical interactive environment for Python


Jupyter Notebook

• The Jupyter Notebook is a web application that allows you to create and share documents that contain live code, equations, visualizations and explanatory text.

• Free software with an active open source community

• Born out of the IPython Project in 2014• Already included in Anaconda Python

• Simply change to your working directory andrun jupyter notebook on the command-line

• Learn more at http://jupyter.org/


Integrated development environments (IDEs)

• Many Python IDEs available: https://wiki.python.org/moin/IntegratedDevelopmentEnvironments• Easy to install Gurobi in other Python distributions

• Spyder is distributed with Anaconda• Scientific Python Development EnviRonment• Familiar for users of MATLAB, Visual Studio, etc.• Built-in editor, file explorer, documentation browser• Graphical interface for debugging (ipdb)• IPython console for interactive code execution


Connecting Gurobi and Python


Connecting Python and Gurobi

• Basic Python interpreter: gurobi.bat / gurobi.sh• Included and configured when you install Gurobi Optimizer

• Anaconda Python – automated installationconda config --add channels http://conda.anaconda.org/gurobiconda install gurobi

• Manual installation for other Python distributions• Change to GUROBI_HOME directory• Run python setup.py install

• For details, see the Gurobi Quick Start Guide: http://www.gurobi.com/documentation/


The Gurobi Python API

• Gurobi Interactive Shell is a Python shell with Gurobi extensions• Gurobi distribution includes Python interpreter and basic modules• Nothing extra to buy or install

• Don't need to be a Python expert to start modeling in Python

• Powerful environment for interacting with existing models• Model file formats: .mps, .rew, .lp, .rlp, .ilp• All file formats support built-in compression: .zip, .gz, .bz2, .7z


Demonstration: Solve model in Jupyter Notebook

• Goals:• Make yourself familiar with Jupyter Notebook• Solve a model file in MPS format

• Example file: afiro.mps (included in the examples directory of the Gurobi installation)

• Steps:• Import the gurobipy module• Create model object from model file• Solve model to optimality• Print solution values


Log file (default: gurobi.log)


Gurobi 7.0.2 (mac64, Python) logging started Thu Feb 2 11:57:06 2017

Read MPS format model from file afiro.mpsReading time = 0.00 secondsAFIRO: 27 rows, 32 columns, 83 nonzerosOptimize a model with 27 rows, 32 columns and 83 nonzerosCoefficient statistics: Matrix range [1e-01, 2e+00] Objective range [3e-01, 1e+01] Bounds range [0e+00, 0e+00] RHS range [4e+01, 5e+02]

Presolve removed 18 rows and 20 columnsPresolve time: 0.01sPresolved: 9 rows, 12 columns, 32 nonzeros

Iteration Objective Primal Inf. Dual Inf. Time 0 -4.8565680e+02 1.363638e+02 0.000000e+00 0s 3 -4.6475314e+02 0.000000e+00 0.000000e+00 0s

Solved in 3 iterations and 0.01 secondsOptimal objective -4.647531429e+02

Log file


Variable X-------------------------

X01 80X02 25.5X03 54.5X04 84.8X06 18.2143X14 18.2143X16 19.3071X22 500X23 475.92X24 24.08X26 215X36 339.943X37 383.943

Interactive shell environment

• Interactive shell provides access to all Gurobi Optimizer functionality through Python API

• read() is a global function which returns an object of type Model

• Model.optimize() and Model.printAttr() are methods (functions on the Model object)

• We can choose any name for the Model object variable:

model = read('afiro.mps') m = read('afiro.mps')model.optimize() or m.optimize()model.printAttr('X') m.printAttr('X')


Basic Gurobi concepts• Parameters control the operation of the Gurobi solvers. They

must be modified before the optimization begins.http://www.gurobi.com/documentation/current/parameters.html

• Attributes are the primary mechanism for querying and modifying properties of a Gurobi model.

http://www.gurobi.com/documentation/current/refman/attributes.html

Attributes can be associated with • the model as a whole (e.g. the objective function value)• variables (e.g. lower bounds)• constraints (e.g. the right-hand side)• SOSs (e.g. IIS membership)

• Environments are the containers for models and global parameter settings. When a model is created inside an environment, it creates a local copy of all global parametersettings.

• The Python API automatically provides a default environment.


Environment Global Parameter Settings

Local Parameter SettingsModel 2

Local Parameter SettingsModel 1

Model Attributes

Variables

Constraints

Special Ordered Sets

Attributes

Attributes

Attributes

Parameters

Attributes

• Multiple ways to access attributes• Methods: getAttr() and setAttr()on Model, Constr, or Var objects

• Model.getAttr()also can retrieve attributes for multiple variables or constraints• Directly: m.ModelName

• Examples:• print(m.getAttr('ObjVal'))• print(m.ObjVal)

• m.ModelName = 'My Model'• m.setAttr(m.ModelName, 'My Model')


Python API fundamentals #1

• Important global functions• setParam() set (global) parameters for new models in this session• read() create model object from a model file

• Primary objects• Model the model• Var a variable• Constr a constraint

• Parameter/attribute examples:• setParam('TimeLimit', 60) Set global parameter• model.TimeLimit = 60 Set local parameter• variable.VarName = 'TotalCosts' Set variable attribute• print(constraint.RHS) Set constraint attribute

• Other important methods:• Model.setObjective(expression) Set objective function• Model.optimize() Start optimization• Model.reset() Reset model state


Python API fundamentals #2

• Model summary information methods• Model.printAttr() display attribute values in bulk, without needing a loop• Model.printStats() display sizes, coefficient statistics, etc.• Model.printQuality() display info about solution quality

• Model introspection methods• Model.getVars() get variables as a list of Var objects• Model.getConstrs() get constraints as a list of Constr objects• Model.getRow() get LHS expression for a given constraint• Model.getCol() get list of constraints for a given variable

• Python control statements• for i in <some list>:

<do something for each i here>• if <condition>:

<do something if condition is true here>

• Must indent body of each statement (at least one space or tab)


Getting help

• Use the help() function with no arguments to get general help

• You can get help on specific Gurobi objects• help(Model) Gurobi model object (or help(m) if m is a model)• help(Var) Gurobi variable object (or help(v) if v is a variable)• help(Constr) Gurobi constraint object (or help(c) if c is a constraint)• help(GRB.attr) Gurobi attributes

• You can get help on specific functions or methods• Ex: help(read)• Ex: help(Model.printAttr)

• You can get help on any parameter• Ex: paramHelp('Method')


Demonstration – model introspection

• Launch interactive shell• Read coins model from examples/data subdirectory of Gurobi installation:

model = read('<installdir>/examples/data/coins.lp')

• Print model statistics and solve it:model.printStats()model.optimize()

• Display solution:for v in <get a list of variables in the model>:

if v.<solution value> != 0:print(v.<variable name>, v.<solution value>)

• Notes/Hints:• Don't forget to indent for control structures and enter blank line to end the for loop• Use help(Model) to replace <get a list of variables in the model> with correct method• Use help(Var) to replace <variable name> and <solution value> with correct attributes


Console


gurobi> model = read('coins.lp')gurobi> model.printStats()

Statistics for model Unnamed :Linear constraint matrix : 4 Constrs, 9 Vars, 16 NZsVariable types : 4 Continuous, 5 IntegerMatrix coefficient range : [ 0.06, 7.2 ]Objective coefficient range : [ 0.01, 1 ]Variable bound range : [ 50, 1000 ]RHS coefficient range : [ 0, 0 ]

Console


gurobi> model.optimize()...Found heuristic solution: objective -0Presolve removed 1 rows and 5 columnsPresolve time: 0.01sPresolved: 3 rows, 4 columns, 9 nonzerosVariable types: 0 continuous, 4 integer (0 binary)

Root relaxation: objective 1.134615e+02, 4 iterations, 0.00 seconds

Nodes | Current Node | Objective Bounds | WorkExpl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time

0 0 113.46154 0 1 -0.00000 113.46154 - - 0sH 0 0 113.4500000 113.46154 0.01% - 0s

Explored 0 nodes (4 simplex iterations) in 0.03 secondsThread count was 4 (of 4 available processors)

Optimal solution found (tolerance 1.00e-04)Best objective 1.134500000000e+02, best bound 1.134500000000e+02, gap 0.0%

Console


gurobi> for v in model.getVars():....... if v.x != 0:....... print v.varname, v.x....... Dimes 2.0Quarters 53.0Dollars 100.0Cu 999.8Ni 46.9Zi 50.0Mn 30.0

Deciphering errors

• Errors are raised when something goes wrong• Trace specifies where error occurred• Last line specifies error type and may provide additional details

• Ex: IndentationError raised when block spacing is incorrect (must indent before if)

gurobi> for v in model.getVars():....... if v.x != 0:

File "<stdin>", line 2if v.x != 0:^

IndentationError: expected an indented block

• <stdin> corresponds to interactive shell


Deciphering errors

• Ex: AttributeError raised when attribute is incorrect (should be VarName)

gurobi> for v in model.getVars():

....... if v.X != 0:

....... print v.Name, v.X

.......

Traceback (most recent call last):

File "<stdin>", line 3, in <module>

File "var.pxi", line 76, in gurobipy.Var.__getattr__ (../../src/python/gurobipy.c:11798)

File "var.pxi", line 121, in gurobipy.Var.getAttr (../../src/python/gurobipy.c:12310)

AttributeError: 'gurobipy.Var' object has no attribute 'Name'

• Don't worry about trace from within gurobipy


Additional model methods

• Attribute and parameter management• Model.getAttr() get attribute values• Model.setAttr() set attribute values• Model.setParam() set parameter values for a model• Model.getParamInfo() get current parameter values

• Importing and exporting• Model.read() import data from a file (ex: .prm)• Model.write() export model data to a file (ex: .sol)• Data type is determined by file extension

• Copying and exporting• Model.copy() create model copy, ex: n = m.copy()• Model.relax() create model copy with integer variables relaxed• Model.fixed() create model copy with integer variables fixed to values in the current solution


Additional model methods

• Algorithms• Model.optimize() solve a model• Model.reset() discard any solution information (next solve starts from scratch)• Model.presolve() create new model containing presolved version of original model• Model.computeIIS() compute IIS for an infeasible model• Model.feasRelax() perform feasibility relaxation on a model by introducing penalty variables

and penalty objective


Editing and running Python programs

• Python is not just an interactive shell

• Python programs are typically stored in text files with .py extension• Any text editor can be used to edit them (ex: notepad)• Most programming editors have Python mode with syntax highlighting, etc.• Comments are preceded with '#' character

• Most functionality in Python is provided by modules• To use a module in a program, include an import statement• Ex: import math

• To use the Gurobi interface in a program, import the gurobipy module:from gurobipy import *

• Use command line to run program with Python interpreter distributed with Gurobi


program.py


# import gurobipy modulefrom gurobipy import *

# create empty modelm = Model()

# change model name attributem.ModelName = 'My First Model'

# process the pending changem.update()

# export model to filem.write('emptymodel.mps')m.write('emptymodel.lp')

# print to consoleprint('This program creates an empty model named:', m.Modelname)print('Model exported to emptymodel.mps and emptymodel.lp')

Run a Python program: Console output


$ gurobi.sh program.pyThis program creates an empty model named: My ModelModel exported to emptymodel.mps and emptymodel.lp

Useful packages

• Common questions:• Can Gurobi read data from … ?• Can Gurobi output the solution as …?• Can Gurobi be used through a web service?• Can Gurobi solve this model using an approach that … ?• Can Gurobi… ?

• The answer:• Yes, most of the time since Gurobi stands on the shoulders of giants

• Python includes a large standard library • Many more add-on libraries and frameworks are available for download

• The Python API is a full interface that includes all Gurobi features• You can use Python to build powerful, complete Gurobi applications


Pandas

• Fast and efficient DataFrame and Series objects• Read/write CSV, HDF5, Excel, SQL, plain text• Missing data handling• Slicing, fancy indexing, subsetting• Merges, joins• Split-apply-combine operations (groupby)

http://pandas.pydata.org


Bokeh

• Interactive visualization for web browsers• High performance with streaming data and big data• In-browser interactivity on otherwise static pages• Bindings for Python, Scala, Julia, R

http://bokeh.pydata.org


Use our Gurobi online resources

• The Gurobi Reference Manual is freely available at

http://www.gurobi.com/documentation/current/refman/index.html

It contains detailed descriptions of our algorithms and APIs.

• We provide API examples for all major programming languages as well asinteractive examples at

http://www.gurobi.com/resources/examples/example-models-overview


Gurobi documentation

• Quick Start Guides• One for each supported OS

• Example Tour• 22 examples• Examples for all APIs

• Reference Manual• Detailed information on APIs, parameters, attributes, …

• Available online at http://www.gurobi.com/documentation

• Docs are installed in the docs subdirectory• Sample code is installed in the examples subdirectory


A1 - Introduction to Python - Gurobi · •Gurobi Interactive Shell is a Python shell with Gurobi...

Documents

Transcript of A1 - Introduction to Python - Gurobi · •Gurobi Interactive Shell is a Python shell with Gurobi...