Function parfor: How and When

The easiest (but not necessarily the best) way to make use of the Parallel Computing Toolbox is without doubt the parfor loop. This is due to it's implementation simplicity!
A parfor loop has the same syntax as a for loop. An example follows:

As you can see this loop is similar to the familiar for loop. The difference between them is that the parfor loop doesn't run in a serial order, which means that each lab will take care of a unique set of statements, associated with the variable n. Imagine we have two labs and the loop above. Then Matlab will order the lab 1 to take care of the statements when n equals 1,3,6,7,10 while the lab 2 will run the remaining statements in no particular order.

A few problems arise from the way parfor works. Whenever we have a dependency between iterations we can't use this function. Example:

If you tried to use parfor instead of for in the example above you would receive an error. Why ? Because the iterations are dependent of the iterations before. To calculate S(6) you need to know the value of S(5) and S(4), but because of the way parfor loop work there is no guaranties that the statements for n=6 won't be run before the statements for n=4. And even if it was run in a increasing order we would still have a problem. Each lab has a different memory that is used to store the variables they come across. This means that when we run a parfor loop each lab will store the values for the statements they run, and won't share them with the remaining labs until the loop is complete. Therefore, even if the loop was ran in an increasing order the lab assigned to n=6 might not have the value of S(5) or S(4) stored.

When trying to improve our code by substituting parallel commands, in this case parfor, we always have to make sure there isn't any kind of dependence between the iterations. If there is we should just leave that section of the code and move on to another one that might be easier to improve.

Other things you must be cautious about when dealing with parfor:
- You can't have nested parfor loops.
- You can't have the command break or return inside a parfor loop.
- You can't use the commands evalc, eval, evalin and assignin.

There are more errors that can arise from the use of parfor. I've simply listed some of the more common.

In my next post I will show a simple program and the respective run time with for and parfor to give you an idea of the advantages of parallelizing your code!