PGI Compiler Documentation

C++ 11 issues

One group had an issue running against the C++ 11 standard with the default PGI. Chester's versions of C++ were built against an older version of GNU so you must rebuild against a newer version. For PGI 15.7, you can do that with the command below (the command for other versions of PGI should be similar, but the version number in the PGI path will differ):

/opt/pgi/15.7.0/linux86-64/15.7/bin/makelocalrc /opt/pgi/15.7.0/linux86-64/15.7/bin -gcc /opt/gcc/4.8.2/bin/gcc -gpp /opt/gcc/4.8.2/bin/g++ -g77 /opt/gcc/4.8.2/bin/gfortran -x -o >> ~/.mypgcpprc

Default version on Chester/Titan

15.7

Using Dave Norton's 15.9 version

Source modules from Norton's dir setenv MODULEPATH ${MODULEPATH}:~norton/.modules Type module switch pgi/15.9.lustre

Using CUDA Unified Memory (aka managed memory)

To enable use of CUDA Unified Memory, use Dave Norton's 15.9 compiler then add "-ta=tesla:managed" to your compilation. For details about CUDA Unified Memory see: http://www.pgroup.com/lit/articles/insider/v6n2a4.htm

Combining pgi & gnu compilers

pgcpp has a different name-mangling scheme than g++, however pgc++ can link g++-compiled object files. See https://www.pgroup.com/support/interlan.htm#pgiCpp2gpp

Creating a Call Graph with PGI

Compile the whole program with the flag "-Mipa=cg" and view the call graph using the "pgicg" utility. C++ can be a bit difficult to read due to name managing and the depth of calls.

Running out of memory?

We found a test case where a routine containing OpenACC grabbed a lot of device memory, but did not return it, leading to an error later on where the device ran out of memory in another library. The PGI runtime uses a memory pool to avoid the cost of allocating and freeing memory with the device. To force it to release the memory in its pool, call the acc_clear_freelists() function. NOTE: This will add additional overheads.