Valgrind memcheck with R: A quick and dirty guide
This may or may not help someone. It is a quick and dirty guide written by a first time valgrind memcheck user. I wished for something similar, so here it is.
Suppose, like me, you submit your new R package, version 1.0.0, to CRAN and it’s accepted. You rejoice happily and move on to bigger things, like building a fantastic update. After a few months you have 8000 downloads, your cool new update is done, and you submit version 1.0.1. Being the amazing professional that you are, you expect your work will slide breezily through into CRAN.
You get an email from CRAN saying everything is great, except for one little issue: there were some errors detected in your previous version (1.0.0) during some kind of secondary checking procedure, reported by this thing called valgrind, which apparently detects “memory management and threading bugs”, but you have no idea what that is. Your spirits are still high. You’re sure this is no big deal…
I hope that this quick guide, from my similar experience, can alleviate some of the pain that may be to come. That way, you can get your R package on CRAN and spend the weekend doing the things you love, like reading stats journals.
Step 0: Get Linux (skip if you have Linux).
It’s time to get Linux, if you don’t already have it. It really makes life easier. Besides, I have no idea how to do this without Linux. For a newbie, Linux Mint is the best distribution (in my humble opinion).
Note: You may run into a number of errors to do with package dependencies if you’re using a fresh installation of Linux. The easiest solution is to wait until you get an error and then copy some part of the error into a search engine and find a forum where someone has outlined what dependencies you need to install.
Note: I suggest using git to copy your package files from your remote repository (e.g., a GitHub repo) to your Linux machine.
Step 1: Install valgrind.
Something like this should work
sudo apt-get install valgrindStep 2: Run R CMD Check.
You need to specify the directory “./check” with check_dir so that you can retrieve mypkg-Ex.R from the results.
rcmdcheck::rcmdcheck( check_dir = ”./check” )Step 3: Run valgrind on R CMD Check examples.
Open your terminal in the package directory. In Ubuntu you can do this as follows: open the package directory in the file explorer, press CTRL-L to highlight the address bar, followed by CTRL+C to copy the address, then open terminal and type cd followed by a space, then press CTRL+SHIFT+V to paste the address. Now, run (replacing mypkg with the name of your package):
R -d valgrind --vanilla < “./check/mypkg.Rcheck/mypkg-Ex.R”The script mypkg-Ex.R will be executed with valgrind running in the background. Valgrind is looking for memory issues. It will run through all the examples in mypkg-Ex.R and every time valgrind detects an issue it will print some (helpful?) information, such as:
==27138== Conditional jump or move depends on uninitialised value(s)
==27138== at 0x55F4CEC: __wcsnlen_avx2 (strlen-avx2.S:103)
==27138== by 0x5522EC1: wcsrtombs (wcsrtombs.c:104)
==27138== by 0x54A8B20: wcstombs (wcstombs.c:34)
==27138== by 0x4EDACC2: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F3EE4A: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E1D7: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F5066D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4569D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E1D7: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F5066D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E3A1: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4EE5176: ??? (in /usr/lib/R/lib/libR.so)
==27138==Or maybe…
==27138== Invalid read of size 32
==27138== at 0x55F4C91: __wcsnlen_avx2 (strlen-avx2.S:62)
==27138== by 0x5522EC1: wcsrtombs (wcsrtombs.c:104)
==27138== by 0x54A8B20: wcstombs (wcstombs.c:34)
==27138== by 0x4EDACC2: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F3EE4A: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E1D7: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F5066D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4569D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E1D7: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F5066D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E3A1: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4EE5176: ??? (in /usr/lib/R/lib/libR.so)
==27138== Address 0x1fb9d5d0 is 0 bytes inside a block of size 12 alloc'd
==27138== at 0x4C31B25: calloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27138== by 0x4F7B1B0: R_chk_calloc (in /usr/lib/R/lib/libR.so)
==27138== by 0x4EDAC3D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F3EE4A: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E1D7: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F5066D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4569D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E1D7: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F5066D: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F4E3A1: Rf_eval (in /usr/lib/R/lib/libR.so)
==27138== by 0x4EE5176: ??? (in /usr/lib/R/lib/libR.so)
==27138== by 0x4F3EE4A: ??? (in /usr/lib/R/lib/libR.so)
==27138==Or even…
==27138== Invalid read of size 1
==27138== at 0x4C33DA3: strcmp (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27138== by 0x243B1683: _XimUnRegisterIMInstantiateCallback (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x24398D32: XUnregisterIMInstantiateCallback (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x243B1566: _XimRegisterIMInstantiateCallback (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x24398CBA: XRegisterIMInstantiateCallback (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x23C8BDFB: TkpOpenDisplay (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFA769: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFB0AB: TkCreateMainWindow (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C06CC6: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C04C7A: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFD6C4: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x235E4D5B: ??? (in /usr/lib/R/library/tcltk/libs/tcltk.so)
==27138== Address 0x1f4f12d0 is 0 bytes inside a block of size 9 free'd
==27138== at 0x4C30D3B: free (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27138== by 0x243A7D8C: XSetLocaleModifiers (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x23C8C536: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C8C4BB: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x243B1566: _XimRegisterIMInstantiateCallback (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x24398CBA: XRegisterIMInstantiateCallback (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x23C8BDFB: TkpOpenDisplay (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFA769: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFB0AB: TkCreateMainWindow (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C06CC6: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C04C7A: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFD6C4: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== Block was alloc'd at
==27138== at 0x4C2FB0F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==27138== by 0x243A7984: _XlcDefaultMapModifiers (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x243A7D75: XSetLocaleModifiers (in /usr/lib/x86_64-linux-gnu/libX11.so.6.3.0)
==27138== by 0x23C8C536: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C8BDE3: TkpOpenDisplay (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFA769: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFB0AB: TkCreateMainWindow (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C06CC6: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23C04C7A: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x23BFD6C4: ??? (in /usr/lib/x86_64-linux-gnu/libtk8.6.so)
==27138== by 0x235E4D5B: ??? (in /usr/lib/R/library/tcltk/libs/tcltk.so)
==27138== by 0x4F12633: ??? (in /usr/lib/R/lib/libR.so)These are fun to look at :). The last one is a bit messy, below I’ve replaced some parts with “…” to make the structure more apparent:
==27138== Invalid read of size 1
==27138== at 0x4C33DA3: strcmp (in...)
==27138== by 0x243B1683: _XimUnRegisterIMInst...
==27138== by 0x24398D32: XUnregisterIMInstant...
==27138== by 0x243B1566: _XimRegisterIMInstant...
==27138== by 0x24398CBA: XRegisterIMInstantiat...
==27138== by 0x23C8BDFB: TkpOpenDisplay (in...)
==27138== by 0x23BFA769: ??? (in...)
==27138== by 0x23BFB0AB: TkCreateMainWindow (in...)
==27138== by 0x23C06CC6: ??? (in...)
==27138== by 0x23C04C7A: ??? (in...)
==27138== by 0x23BFD6C4: ??? (in...)
==27138== by 0x235E4D5B: ??? (in...)
==27138== Address .... is 0 bytes inside a block of size 9 free'd
==27138== at 0x4C30D3B: free (in...)
==27138== by 0x243A7D8C: XSetLocaleModifiers (in...)
==27138== by 0x23C8C536: ??? (in...)
==27138== by 0x23C8C4BB: ??? (in...)
==27138== by 0x243B1566: _XimRegisterIMInstant...)
==27138== by 0x24398CBA: XRegisterIMInstant...)
==27138== by 0x23C8BDFB: TkpOpenDisplay (in...)
==27138== by 0x23BFA769: ??? (in...)
==27138== by 0x23BFB0AB: TkCreateMainWindow (in...)
==27138== by 0x23C06CC6: ??? (in...)
==27138== by 0x23C04C7A: ??? (in...)
==27138== by 0x23BFD6C4: ??? (in...)
==27138== Block was alloc'd at
==27138== at 0x4C2FB0F: malloc (in...)
==27138== by 0x243A7984: _XlcDefaultMapModifiers (in...)
==27138== by 0x243A7D75: XSetLocaleModifiers (in...)
==27138== by 0x23C8C536: ??? (in...)
==27138== by 0x23C8BDE3: TkpOpenDisplay (in...)
==27138== by 0x23BFA769: ??? (in...)
==27138== by 0x23BFB0AB: TkCreateMainWindow (in...)
==27138== by 0x23C06CC6: ??? (in...)
==27138== by 0x23C04C7A: ??? (in...)
==27138== by 0x23BFD6C4: ??? (in...)
==27138== by 0x235E4D5B: ??? (in...)
==27138== by 0x4F12633: ??? (in...)Valgrind memcheck reports two kinds of bugs: memory leaks and memory error. Read more here if you like.
Now bring out the big guns:
R -d "valgrind --tool=memcheck --leak-check=full --track-origins=yes --show-leak-kinds=definite" --vanilla < “./check/mypkg.Rcheck/mypkg-Ex.R”Step 4: Start the hunt
Create a new empty R script ./check/mypkg.Rcheck/debug-Ex.R . Now, open up the script ./check/mypkg.Rcheck/mypkg-Ex.R and look for the first appearance of cleanEx() . Copy everything above the first appearance of cleanEx() into your new script. Then go to the end of mypkg-Ex.R and copy the last few lines below the word “FOOTER” and paste it to the end of debug-Ex.R . Now your debug-Ex.R should look something like this (where I added # symbols to visually separate the header from the footer):
pkgname <- "mypkg"
source(file.path(R.home("share"), "R", "examples-header.R"))
options(warn = 1)
library('mypkg')base::assign(".oldSearch", base::search(), pos = 'CheckExEnv')
#####################################################################################################################################
### * <FOOTER>
###
options(digits = 7L)
base::cat("Time elapsed: ", proc.time() - base::get("ptime", pos = 'CheckExEnv'),"\n")
grDevices::dev.off()
###
### Local variables: ***
### mode: outline-minor ***
### outline-regexp: "\\(> \\)?### [*]+" ***
### End: ***
quit('no')
Results
The function causing the error was minDist. It looked like this:
double minDist_internal1(Rcpp::DataFrame cmbdf, NumericVector point) {int n = cmbdf.nrow();
NumericVector x = cmbdf["x"];
NumericVector y = cmbdf["y"];
NumericVector z = cmbdf["z"];
double px = point[1];
double py = point[2];
double pz = point[3];// Find the minimum distance
double mindot = -1;
for ( int i = 0; i < n; i++ )
{
// Find d(xi, xj)
double dot = x[i]*px + y[i]*py + z[i]*pz;//if ( dot > mindot ) mindot = dot;
if ( true ) mindot = dot;
}return acos(std::min<double>(std::max<double>( mindot, -1),1));
}
Turns out I was indexing starting at 1 instead of 0.
double px = point[0];
double py = point[1];
double pz = point[2];Also, if I just used round brackets then this never would have happened.
double px = point(0);
double py = point(1);
double pz = point(2);That’s because Rcpp will throw an index out of bounds exception when round brackets are used, but square brackets will happily go ahead with out-of-bounds access.
References
Section 4.3 of Writing R Extensions (https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Using-valgrind)
