A Guide to Compiling and Running Matlab Compiled Code on ... · A Guide to Compiling and Running...

A Guide to Compiling and Running Matlab Compiled Code on the CBI cheetah cluster:

Matlab sample code( testcompiling.m) :function [ average, rc ] = testcompiling( rows, cols)%Testcompiling:% This function just creates a 2D matrix of random numbers and % returns the mean. This will be used to illustrate the % process of compiling a Matlab code and running it within% the CBI cluster( cheetah.cbi.utsa.edu ) using the % Sun Grid Engine.% Copyright CBI Lab/UTSA. 2011. All Rights Reserved.%% version 1.0.0: [email protected]%rc = 0; % return codeaverage= 0; % return valueif ( rows > 100 ) rc = -1; return;end if ( cols > 100 ) rc = -2; return;end x = rand(100,100);average = mean(mean(x));

Compile codeFirst step is to go to Desktop, Then Deployment Tool

Copyright 2011. CBI/UTSA. All Rights Reserved. Document version 1.0

Next, go to create a new deployment project. Most of time, you will want to create a standalone application. This will create an executable that can run together with the Matlab Compiler Runtime.

Next, go to Main File, Add main file


After the file is added, click on the build button. This will pop up the Build window.

If the build is successful, you should see a “Build Finished” message with no error messages.


Next, you need to create a Package.Do not include the MCR if you are planning to run on the CBI cheetah cluster since it is already installed. If you include it, the build package will be over 200+ MB. You may want to include it however, if you plan to deploy the executable to other users since they would need the same exact version of the MCR that matches the compiler version.

Next, you would click on the Package button.

You should end up with the following structure:


At this point, the process of creating the executable should be complete.

How to run on the CBI cheetah cluster:

You can now login to cheetah.cbi.utsa.edu and then use qlogin to get to one of the nodes in interactive mode.

ssh -Y <username>@cheetah.cbi.utsa.eduqlogin

Once you are logged in, if you change to the directory used to compile the project:

Two main directories are created by the Matlab Compiling procedure automatically:The distrib directory contains the executable file and auto- generated shell script that is used to call the executable. The shell script is needed because it sets up the dynamically linked library paths that the executable needs to run. Mainly the MCR runtime library path.

./compilingexample/testcompiling/src


./compilingexample/testcompiling/distrib, This will be the directory you will need to referencewithin the Sun Grid Engine job file.

Running locally for an initial test:cd ./distrib/./run_testcompiling.sh /share/apps/matlab2011a argument1 argument2 .. argumentN

In this case, argument1 maps to the number of rows, and argument2 maps to the number of columns of our 2D matrix.

[nelson.ramirez@compute-6-2 distrib]$ ./run_testcompiling.sh /share/apps/matlab2011a 50 50------------------------------------------Setting up environment variables---LD_LIBRARY_PATH is .:/share/apps/matlab2011a/runtime/glnxa64:/share/apps/matlab2011a/bin/glnxa64:/share/apps/matlab2011a/sys/os/glnxa64:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/server:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/client:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64

Two of the most important issues when using the compiled version versus the command line version are the following:

1) Converting input parameters from string to numeric types in compiled version. Arguments are passed into the executable as string types, if the executable is expecting numeric types, then these need to be converted to numeric types using the str2num function.

2) Returning results. There are only 2 ways to return results from the compiled Matlab executable. By printing the output or by saving it within a file. In this example we will send it to the standard output and use file output redirection to store the results in an output file.

To handle these issues the following code changes need to be made:

function [ average, rc ] = testcompiling( rows, cols)%Testcompiling:% This function just creates a 2D matrix of random numbers and % returns the mean. This will be used to illustrate the % process of compiling a Matlab code and running it within% the CBI cluster( cheetah.cbi.utsa.edu ) using the % Sun Grid Engine.% Copyright CBI Lab/UTSA. 2011. All Rights Reserved.%% version 1.0.0: [email protected]%rc = 0; % return codeaverage= 0; % return value if (isdeployed)


rows = str2num(rows) cols = str2num(cols)end if ( rows > 100 ) rc = -1; average % needed in compiled version rc % needed in compiled version return;end if ( cols > 100 ) rc = -2; average % needed in compiled version rc % needed in compiled version return;end x = rand(100,100);average = mean(mean(x)); average % needed in compiled versionrc % needed in compiled version end

Rebuild the project:

Repackage the project:


Now, when you re-run the executable, this time you will see output on the command window.

This output can be re-directed to an output file also.

[nelson.ramirez@compute-6-2 distrib]$ ./run_testcompiling.sh /share/apps/matlab2011a 50 50------------------------------------------Setting up environment variables---LD_LIBRARY_PATH is .:/share/apps/matlab2011a/runtime/glnxa64:/share/apps/matlab2011a/bin/glnxa64:/share/apps/matlab2011a/sys/os/glnxa64:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/server:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/client:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64

rows =

50

cols =

50

average =

0.4996

rc =


0

[nelson.ramirez@compute-6-2 distrib]$

If you redirect the output to a specific

./run_testcompiling.sh /share/apps/matlab2011a 50 50 > testcompilingresults.txt[nelson.ramirez@compute-6-2 distrib]$ [nelson.ramirez@compute-6-2 distrib]$ lsreadme.txt run_testcompiling.sh testcompiling testcompilingresults.txt

vi testcompilingresults.txt

This results file can then be processed to extract the output values as needed. If a specific format is required, saving the results directly to a file from within the Matlab code provides more flexibility.

Running the executableOnce you have developed your code and debugged the compilation process, to perform the actual runs of the program, you should submit your jobs to the cluster via the Sun Grid Engine job interface ( qsub ).


The pages that follow detail taking the executable generated in the previous steps and creating job scripts for grid submission:

Running as a batch job with the Sun Grid EngineThe original sge job script example is from http://cbi.utsa.edu/sge_tutorial

Below is a sample Sun Grid Engine job script modified for running with the Matlab compiled executable:

compiledmatlabtest.job

#!/bin/bash#$ -N compiledtestmatlab#$ -j y#$ -o compiledtestmatlab.log#$ -cwd./run_testcompiling.sh /share/apps/matlab2011a 50 50exit 0

Issues:There is 1 key issue when running the compiled version of the code from within the the sge job script. The shell script that is auto-generated by the Matlab compile and build tool wraps the call to the actual Matlab executable, as well a requiring the first argument to the shell script to be the path of the MCR. Therefore, to add the -nodisplay -nodesktop -nojvm -r functionname options from the original job script in the sun grid engine tutorial in the link above, we need to modify the auto-generated Matlab shell script as follows:

run_testcompiling.sh#!/bin/sh# script for execution of deployed applications## Sets up the MCR environment for the current $ARCH and executes # the specified command.#exe_name=$0 -nodisplay -nodesktop -nojvm -r testcompilingexe_dir=`dirname "$0"`echo "------------------------------------------"if [ "x$1" = "x" ]; then echo Usage: echo $0 \<deployedMCRroot\> argselse echo Setting up environment variables MCRROOT="$1" echo --- LD_LIBRARY_PATH=.:${MCRROOT}/runtime/glnxa64 ;


http://cbi.utsa.edu/sge_tutorial

LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/bin/glnxa64 ; LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/os/glnxa64;

MCRJRE=${MCRROOT}/sys/java/jre/glnxa64/jre/lib/amd64 ;LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRJRE}/native_threads ; LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRJRE}/server ;LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRJRE}/client ;LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRJRE} ;

XAPPLRESDIR=${MCRROOT}/X11/app-defaults ; export LD_LIBRARY_PATH; export XAPPLRESDIR; echo LD_LIBRARY_PATH is ${LD_LIBRARY_PATH}; shift 1 args= while [ $# -gt 0 ]; do token=`echo "$1" | sed 's/ /\\\\ /g'` # Add blackslash before each blank args="${args} ${token}" shift done eval "${exe_dir}"/testcompiling $argsfiexit

We just need to provide the command line arguments directly within the Matlab executable wrapper script:exe_name=$0

Needs to be changed to:

exe_name=$0 -nodisplay -nodesktop -nojvm -r testcompiling

Using QSUB to submit the job:

[nelson.ramirez@compute-2-4 distrib]$ qsub compiledmatlabtest.job Your job 40181 ("compiledtestmatlab") has been submitted[nelson.ramirez@compute-2-4 distrib]$ qstatjob-ID prior name user state submit/start at queue slots ja-task-ID ----------------------------------------------------------------------------------------------------------------- 37206 0.50500 QLOGIN nelson.ramir r 03/22/2012 15:54:06 [email protected] 1 40181 0.00000 compiledte nelson.ramir qw 04/02/2012 13:23:03 1 [nelson.ramirez@compute-2-4 distrib]$

After the job finishes, you should see a log file within the current working directory:

[nelson.ramirez@compute-2-4 distrib]$ lscompiledmatlabtest.job readme.txt testcompiling


compiledtestmatlab.log run_testcompiling.sh[nelson.ramirez@compute-2-4 distrib]$

The compiledtestmatlab.log file should contain the results:Setting up environment variables---LD_LIBRARY_PATH is .:/share/apps/matlab2011a/runtime/glnxa64:/share/apps/matlab2011a/bin/glnxa64:/share/apps/matlab2011a/sys/os/glnxa64:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/native_threads:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/server:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64/client:/share/apps/matlab2011a/sys/java/jre/glnxa64/jre/lib/amd64Warning: No display specified. You will not be able to display graphics on the screen.

rows =

50

cols =

50

average =

0.4996

rc =

0

Summary:

Moving your code to a compiled version using the Matlab Compiler Runtime is great because it will allow you to run many more distributed jobs without using up Matlab licenses.

There is a setup cost to bring in the MCR dynamically linked libraries into memory, but this should be a minimal cost especially for long-running batch Matlab workloads.

Questions/Comments: www.cbi.utsa.edu


http://www.cbi.utsa.edu/

A Guide to Compiling and Running Matlab Compiled Code on ... · A Guide to Compiling and Running...

Documents

Transcript of A Guide to Compiling and Running Matlab Compiled Code on ... · A Guide to Compiling and Running...