avoiding data races with intel mkl

Background

If you do numerically intensive computing, you might want to make it faster. One way to do this is to switch out the backend implementation of linear algebra routines on your computer. At a low level, most linear algebra on your computers happens by making calls via BLAS/LAPACK APIs, and there are several libraries that offer differing implementations of the underlying methods. Some of these libraries are faster than others; many are faster than the default implementation.

I use Intel MKL. On Ubuntu, you can install Intel MKL via the shell with:

sudo apt install intel-mkl

You can check your current BLAS/LAPACK implementation in R using utils::sessionInfo():

si <- utils::sessionInfo()
si$LAPACK

[1] "/usr/lib/x86_64-linux-gnu/libmkl_rt.so"

The problem

On computers with multiple cores, Intel MKL can induce data races during (sufficiently large) matrix multiplies, depending on the threading model. Using the default threading model, I can reliably induce these data races like this:

# this function should return 249.7852
can_induce_data_race <- function() {
  X <- matrix(1:500 / 500, 50, 10)
  Y <- matrix(1:1000 / 1000, 10, 100)

  norm(X %*% Y)
}

can_induce_data_race()

[1] 249.7852

can_induce_data_race()

[1] 249.7852

can_induce_data_race()

[1] 249.7852

See discussion here, here and here if you’re curious about underlying causes.

The solution

To avoid this problem, set the environmental variable MKL_THREADING_LAYER=GNU. You can check the current value of the MKL_THREADING_LAYER environmental variable via

Sys.getenv("MKL_THREADING_LAYER")

[1] "GNU"

When MKL_THREADING_LAYER is not set, it defaults to INTEL. To set MKL_THREADING_LAYER=GNU universally, you can update /etc/environment.

echo "MKL_THREADING_LAYER=GNU" >> /etc/environment

Alternatively, tell MKL to operate without parallelism by setting MKL_NUM_THREADS=1. Note that you will need to restart R for these changes to take effect. More generally, environmental variables are typically read once, at program startup, so you cannot use Sys.setenv() to set MKL_THREADING_LAYER at runtime. Similarly, you cannot change the number of threads MKL uses are runtime via environmental variables (see documentation).

If you want to test the effect of different environmental variables without leaving R, you can use callr::r(), as below:

callr::r(can_induce_data_race, env = c(MKL_THREADING_LAYER = "INTEL"))

[1] 2823.996

callr::r(can_induce_data_race, env = c(MKL_THREADING_LAYER = "GNU"))

[1] 249.7852

If you would like to change set environmental variables locally (for example, if you want targets to manage parallelism across tasks, while running each individual task sequentially), use an .Renviron at the top level of your R project. You can do this via:

usethis::edit_r_environ(scope = "project")