Why is Matlab so slow?

IAP 2009 Scott Gorlin gorlins@mit.edu

http://stellar.mit.edu/S/project/advanced-matlab/

Everyone Gets Frustrated

Why does it take forever to do x? Other applications analyze my data faster

Everyone Gets Frustrated

Why only 50%?!?!

The Curse of Compilation

Computers dont speak English English commands (m-files) must either be interpreted or compiled

The Curse of Compilation

Compiled
Fast languages (i.e. C++) are compiled to machine code, which turns them into executables Slow creation time, fast execution time

The Curse of Compilation

Compiled
Static memory declare all variables, types
Cant change memory types, add/remove variables Separate functions for int, double, etc!

The Curse of Compilation

Interpreted
Every command immediately translated, executed at run-time No up-front time, much slower execution time
NB: ~ 500 vs 10 s per command

Allows dynamic memory addressing

The Curse of Compilations

Interpreted
Examples of dynamic benefit:
Dont need to declare/init variables Can interactively run code (console, guis) Immediately update code (anonymous functions) Work with dynamic data types (structures, cells)

Drawback
Takes longer to interpret commands than to execute them!

Optimization

Vectorize
Get rid of for loops
fib = [1 1 2 3 5 8 13 21 34 ...]; s = sum(fib);

instead of
for i = 1:length(fib) s = s+fib(i); end

Optimization

Vectorize...
Can get overly complex! Eg,

Interpreters

JIT
Just-In-Time compilation Matlab immediately compiles parts of functions when they are called, before running Small up-front cost (ms) first time a function is called Much faster execution for some commands
Esp. loops

Optimization

What no one knows:

JIT makes vectorizing obsolete!
(sometimes)

Optimization

For 10-1000x speedup, write JIT-compliant code! On Stellar: accel_matlab.pdf describes details of this Shaded: Accelerated data types (as of v6.5)

Optimization

Cell vs double arrays

60 fold acceleration! W/o JIT, is ~3

Optimization

Here, the for loop is made JIT compliant by using scalar indexing instead of a vector In fact, here the JIT code is faster than the vectorized code! W/o JIT, 1st ratio is 1

Optimization

JIT Takehome:
Interpreting is powerful but SLOW Write code which can be compiled
Give up some flexibility for a lot of speed!

To test code compliance:

>> feature accel off

[execute function again], note time difference

Optimization

JIT Cheat Sheet

All code in a loop must be compliant for loop to be compiled Supported data types
Ie int/double/char arrays, not cells/structs

Built-in (non m-code) Matlab commands For loops: scalar indexes if/switch/while: scalar expressions 3 or fewer dimensions per array

Optimization

JIT No-nos (version/platform dependent)

4+ dimensional arrays Function calls Interpreted commands/dynamic allocation Publishing, evaluating w/ R-click, console commands In some versions, using i, j, e without defining them first Calling function w/ different data types (forces recompile)

Optimization

JIT (Un)Fortunate Truth:

It is VERY POORLY documented
See original spec (online), then read 1-liners in the release notes for every version of Matlab since 6.5

But, JIT improves in every version

Ie console accelerated in 7.6

Doesn't work as well on OSX, Linux, but getting better Vectorize if possible, when unsure

Optimization

Local variables copied when passed to functions Not entirely true (see links under Materials)
Copy-On-Write Pass-In-Place

Optimization

Copy-On-Write
Variables not really copied until they are changed Passing or renaming a variable does not increase memory if the variable is unchanged!

Optimization

In-Place Operations
Function must return the same variable it's passed, both in definition and when called

Everyone Gets Frustrated

Why only 50%?!?!

Multithreading

A computer can only do 1 thing at once Start Windows, everything freezes

Multithreading

Multithreading illusion of concurrent processes OS rapidly switches between threads and allows each to work for a brief period of time

Multithreading

Side note:

Windows is a poor scheduler, meaning threads do not always begin when they should This leads to timing jitters, or random delays in Matlab up to several hundred ms Can be helped by realtime priority: Matlab thread takes precedence over other threads

Task Manager -> Processes -> MATLAB.exe R-click and set priority to Realtime Can be done programatically through a Java/C function, or download Psychtoolbox for an implementation On a single core, will FREEZE windows until process completes!

Multithreading

Parallel Programming
2 or more threads to simultaneously do something On a single-core computer gives simultaneous execution
I.e., one thread handles GUI, other does background calculations

On a dual-core computer, ideally, doubles performance

Multithreading

Matlab is Single-Threaded
Means you can only do 1 thing at once On newer, dual-core computers this is not optimal

Multithreading

Matlab is Single-Threaded
New versions have multi-threading features This is a misnomer only a multithreaded BLAS
Faster matrix operations (sometimes), but that's it

Multithreading

Matlab is Single-Threaded
Benefits
Dont worry about Thread Safety On a Dual Core, setting Realtime Priority lets Matlab dominate one core, and the rest of the computer runs on the other!

We can hack multithreading, some of the time

Multithreading

Simplest hack timer function

This actually creates a Java object, and executes a callback function with variable delay, interval, etc However, not a true multithread will not execute while another process is dominating! (at least through v7.4)
Therefore mostly useful in GUI/console applications, etc NB: Brief testing in v7.6 indicates it may be asynchronous now! Not recommended to build true parallel applications with this method...

Multithreading

True parallel programming

Must either start a 2nd Matlab session,
Two Matlab sessions can run in parallel via COM, Java, or shared memory spaces Actually a client/server interface one session will dispatch jobs to the other

Write a C/Java applet for native threading, Complicated but powerful Or, use the distributed computing toolbox $$, not provided with free MIT student bundle

Automation Server

Simplest parallel Matlab example See External Interfaces/COM Automation Server for documentation Starts 2nd Matlab session via COM/ActiveX (Windows only, sadly) COM protocol may be too slow for you

Automation Server

Create server with:

h = actxserver('matlab.application'); fields(h), methods(h) show ways to control new Matlab server. Important are:
Execute(h, command) Feval(h, fcnName, numout, arg1, arg2, ) PutFullMatrix, GetFullMatrix, etc

Can even hide window, but make sure to later close it programatically!
h.Visible = 0; ... h.Quit();

Automation Server

Benefits
Native Matlab interface, easy to use, get/send data directly through Matlab

Automation Server

BUT:
By default, still single threaded!! Calling Execute and Feval starts function in server workspace, but client Matlab pauses and waits for completion! Makes sense for Feval(waits for a return value) but not for Execute Must find a way to return control to client while server computes!

Automation Server

Remember timer()?
Execute or call a function which starts a timer in the new workspace! This will allow client to return, and server will process timer thread immediately after Main problem: still a SLOW protocol
Data transfer not too fast Min ~3ms simply to invoke function in server and return May or may not be fast enough for your application

Parallel Matlab

Better, more complicated

Multithread in Java/C-mex code Open 2nd Matlab, but communicate via a different protocol
Sockets Java RMI between the JVMs

Shared memory through C-mex

Actually quite simple code posted online in MatlabRMI.zip Also enables the second Matlab session to run anywhere on your network, for a truly distributed program May run with Matlab Component Runtime (untested) I have no idea how to do this!

Distributed computing toolbox

MPI

Must purchase, I have no experience with so cannot recommend either way Seems good for parallel array operations, batching identical tasks across worker pool but not designed for concurrent programming

Optimization

Take home 4 steps for faster programs

JIT compliant code Memory management
Copy-On-Write Pass-In-Place

Write a parallel program Write a Java/C object

Friday

Object Oriented Programming