Tools Installation and Setup (For R)

Wiki ▸ Tools Installation and Setup (For R)

Prerequisites
- Environment variables
Pre-installed tools and Configuration
- Checks
Packages
- Configuration
- Useful packages
  - Quarto
  - TinyTex
  - Plotly
  - renv
  - ROracle
  - Hugo
- Other packages
- Installation from source
Customization
NEST

Prerequisites

Developing on your computer

Your computer has likely been activated with the default windows group policy applied. This should be changed to the Windows group policy "SP GG AppLocker ER Dev PC" if you are planning to program in R or Python. Please see that Rainer Koelle agrees with the elevation of group policy to enable you to program. If so please request the following to the IT support team either through mail:

Dear IT Service Desk,

Given my work requirements the currently applied group policy (default) to my provided laptop is inadequate and I cannot fulfill my business requirements. I would like to request to change my group policy to SP GG AppLocker ER Dev PC (similarly as has been done for Quinten Goens, see ticket RITM0160163).

The business reason: As a (data) analyst at DECMA/AIU/OPS I will be developing code in R, Python and SQL which perform the following purposes: Data processing, data cleaning, data visualizations, reporting, … To do this I will need to have a developer profile as I will need to be able to run these codes and use the necessary libraries. Please make a ticket for this to enable this.

Kind regards,

Your name

Environment Variables

Find the panel where to edit the environment variable Set or append to them as follows:

Set your proxy password in environment variable HTTP_PROXY_PASSWORD
- Note: if you use special characters in your password (#, $, @, !, etc.), you may have to percent-encode it. To percent-encode your password, replace the special character by the correspondent code. A full list of percent-encoded special characters can be found here. For example, Password123! becomes Password123%21 ; #Password123 becomes %23Password123 ; Password@123 becomes Password%40123 ; etc.
If you do not have special characters in your password (#, $, @, !, etc.): (Eventually create and) Set the following environment variables in order for R and unix tools to pick proxy settings up:
- HTTP_PROXY to http://inputyourusername:%HTTP_PROXY_PASSWORD%@pac.eurocontrol.int:9512
- HTTPS_PROXY (Note: http://) to http://inputyourusername:%HTTP_PROXY_PASSWORD%@pac.eurocontrol.int:9512
If you do have special characters in your password (#, $, @, !, etc.): (Eventually create and) Set the following environment variables in order for R and unix tools to pick proxy settings up (remember to input your percent-encoded password directly into the http_proxy and into https_proxy:
- HTTP_PROXY to http://inputyourusername:inputyourpercentencodedpasswordhere@pac.eurocontrol.int:9512
- HTTPS_PROXY (Note: http://) to http://inputyourusername:inputyourpercentencodedpasswordhere@pac.eurocontrol.int:9512
Set R_LIBS_USER to
- C:\Users\<user>\dev\R\win-library\<R major version>.x (eventually create the relevant folder: 3.x for R-3.x.y, 4.x for R-4.x.y)
(if you want to use R from Git Bash terminal) add C:\Program Files\R\R-4.x.y\bin to your Path environment variable (fille the correct x and y).
Add the Rtools compilers/tool-chain to your Path environment variable, i.e. C:\Rtools42\x86_64-w64-mingw32.static.posix\bin (NOTE: make sure to use the correct XY in RtoolsXY)
UNSET/DELETE the BINPREF user environment variable (if ever you had it defined). NOTE: there is a temporary hack, see next section.

The following is needed only if you are going to compile ROracle, otherwise ask Enrico for a pre-compiled zip.

Set OCI_LIB64 to the result of shortPathName("C:/Program Files/instantclient_12_2") with one single \, i.e. C:\PROGRA~1\INSTAN~1

Pre-installed tools and Configuration

IT Help Desk should have already pushed the following tools on your machine, if not request them via the IT Portal / IT Request / Software Installation:

Once installed set OCI_LIB64 environment variable to the result of shortPathName("C:/Program Files/instantclient_12_2") with one single \, i.e. C:\PROGRA~1\INSTAN~1

Also you should have the right AppBlocker configuration, i.e. no issues when running R packages installed on C\Users\%USERNAME%\dev\R\win-library\M.x.y (i.e. M=4, x=2, y=1). See tests from Useful Packages.

Hacks

For Rtools, until solved by IT Help Desk, the system environment variable BINPREF needs to be unset to have package installation from source working. You can do it by adding the following line in your .Rprofile

Sys.unsetenv("BINPREF")

You can edit your personal .Rprofile from RStudio issuing usethis::edit_r_profile() (of course install the usethis package if not present.)

Checks

make sure Rtools is installed by IT and can be found by R/RStudio, execute in the following R lines (eventually install pkgbuild):

> install.packages("pkgbuild")
> pkgbuild::find_rtools(debug = TRUE)
Found in Rtools 4.2 installation folder
[1] TRUE
> pkgbuild::has_rtools(TRUE)
Found in Rtools 4.2 installation folder
[1] TRUE
> pkgbuild::check_rtools(TRUE)
Found in Rtools 4.2 installation folder
[1] TRUE
> pkgbuild::rtools_path()
[1] "C:\\Rtools42/usr/bin"
> pkgbuild::check_build_tools(TRUE)
Found in Rtools 4.2 installation folder
Your system is ready to build packages!
> pkgbuild::check_compiler(TRUE)
Trying to compile a simple C file
Running "C:/PROGRA~1/R/R-42~1.1/bin/x64/Rcmd.exe" SHLIB foo.c
gcc  -I"C:/PROGRA~1/R/R-42~1.1/include" -DNDEBUG     -I"C:/Rtools42/x86_64-w64-mingw32.static.posix/include"     -O2 -Wall  -std=gnu99 -mfpmath=sse -msse2 -mstackrealign  -c foo.c -o foo.o
gcc -shared -s -static-libgcc -o foo.dll tmp.def foo.o -LC:/Rtools42/x86_64-w64-mingw32.static.posix/lib/x64 -LC:/Rtools42/x86_64-w64-mingw32.static.posix/lib -LC:/PROGRA~1/R/R-42~1.1/bin/x64 -lR
 
[1] TRUE
> Sys.getenv("PATH")
[1] "C:\\Rtools42/x86_64-w64-mingw32.static.posix/bin;C:\\Rtools42/usr/bin;\\usr\\bin;C:\\Program Files\\R\\R-4.2.1\\bin\\x64;C:\\Progra~1\\ODAC\\client\\product\\12.2.0\\client_1;C:\\Progra~1\\ODAC\\client\\product\\12.2.0\\client_1\\bin;C:\\Program Files\\instantclient_12_2;C:\\Rtools42\\usr\\bin\\;C:\\Oracle12c\\product\\12.1.0\\dbhome_1\\bin;C:\\Program Files\\Python39\\Scripts\\;C:\\Program Files\\Python39\\;C:\\PROGRA~2\\Oracle\\product\\12.1.0\\client_1\\bin;C:\\Program Files (x86)\\Common Files\\Oracle\\Java\\javapath;C:\\WINDOWS\\system32;C:\\WINDOWS;C:\\WINDOWS\\System32\\Wbem;C:\\WINDOWS\\System32\\WindowsPowerShell\\v1.0\\;C:\\WINDOWS\\System32\\OpenSSH\\;C:\\Program Files (x86)\\Pulse Secure\\VC142.CRT\\X64\\;C:\\Program Files (x86)\\Pulse Secure\\VC142.CRT\\X86\\;C:\\Program Files (x86)\\Common Files\\Pulse Secure\\TNC Client Plugin\\;C:\\Program Files (x86)\\Google\\Cloud SDK\\google-cloud-sdk\\bin;C:\\Program Files\\Git\\cmd;C:\\Program Files\\TortoiseSVN\\bin;C:\\Program Files\\Quarto\\bin;C:\\Program Files (x86)\\Webex\\Plugins;C:\\Program Files\\PuTTY\\;C:\\Program Files\\Microsoft VS Code\\bin;C:\\Users\\spi\\dev;C:\\Users\\spi\\dev\\bin;C:\\Users\\spi\\dev\\quarto-1.2.269\\bin;C:\\PROGRA~1\\INSTAN~1;C:\\Program Files\\R\\R-4.2.1\\bin;C:\\Program Files\\Anaconda3\\Scripts;C:\\Users\\spi\\AppData\\Local\\Microsoft\\WindowsApps;C:\\Users\\spi\\dev\\qpdf-11.1.1-mingw64\\bin;C:\\Users\\spi\\dev\\TinyTeX\\bin\\win32;C:\\Rtools42\\x86_64-w64-mingw32.static.posix\\bin;"
> Sys.getenv('BINPREF')
[1] ""
> sessionInfo()
R version 4.2.1 (2022-06-23 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 19044)

Matrix products: default

locale:
[1] LC_COLLATE=English_United Kingdom.utf8  LC_CTYPE=English_United Kingdom.utf8   
[3] LC_MONETARY=English_United Kingdom.utf8 LC_NUMERIC=C                           
[5] LC_TIME=English_United Kingdom.utf8    

attached base packages:
[1] graphics  grDevices utils     datasets  stats     methods   base     

loaded via a namespace (and not attached):
 [1] Rcpp_1.0.9        compiler_4.2.1    later_1.3.0       urlchecker_1.0.1 
 [5] prettyunits_1.1.1 profvis_0.3.7     remotes_2.4.2     tools_4.2.1      
 [9] digest_0.6.29     pkgbuild_1.3.1    pkgload_1.3.1     memoise_2.0.1    
[13] lifecycle_1.0.3   rlang_1.0.6       reprex_2.0.2      shiny_1.7.3      
[17] cli_3.4.1         rstudioapi_0.14   fastmap_1.1.0     withr_2.5.0      
[21] stringr_1.4.1     fs_1.5.2          htmlwidgets_1.5.4 devtools_2.4.5   
[25] glue_1.6.2        R6_2.5.1          processx_3.7.0    sessioninfo_1.2.2
[29] callr_3.7.2       purrr_0.3.5       magrittr_2.0.3    ps_1.7.1         
[33] promises_1.2.0.1  ellipsis_0.3.2    htmltools_0.5.3   usethis_2.1.6    
[37] mime_0.12         xtable_1.8-4      httpuv_1.6.6      stringi_1.7.8    
[41] miniUI_0.1.1.1    cachem_1.0.6      crayon_1.5.2

If this returns an error message, make sure IT has whitelisted the dev folder C:\Users\<username>\dev.

As per instruction in the Rtools page (for MS Windows), with IT installation we should have Rtools in the PATH and BIN_PREF correctly setup, so the following should work:

install.packages("jsonlite", type = "source")

Packages

Configuration

MANDATORY: install {usethis} and run

usethis::use_blank_slate(scope = c("user", "project"))

Useful packages

Quarto

Quarto comes included with RStudio but sometimes you want to use a newer version. Then you can

download LATEST (not PRE) release quarto cli zip from https://github.com/quarto-dev/quarto-cli/releases, for example quarto-1.2.269-win.zip.
Unzip in C:\Users<userid>\dev\quarto-1.2.269
Add C:\Users<userid>\dev\quarto-1.2.269\bin to Path environment variable

A nice way to use Quarto is to

open a Git Bash terminal
go to the folder where you have your quarto document (in bash you use the cd [change directory] command)
```
$ cd /c/Users/spi/dev/repos/my_folder/
```
preview your project
```
$ quarto preview
```
a browser tab will open on the HTML rendered document
now when you change you qmd files in RStudio you browser will refresh automatically

Useful packages and extensions and links:

The `tinytex` and TeX/LaTeX

Install the tinytex R package, from R/Rstudio console execute:
```
> install.packages("tinytex")
```
Install the TeXLive as prebuilt by Yihui Xie (we cannot use tinytex::install_tinytex() because it will end up in C:\Users\<username>\AppData\Roaming from which we do not have execution rights):
- download the binary build zip file
- unzip it in your dev folder, i.e. C:\Users\<username>\dev
- add the path above, ie.e C:\Users\<username>\dev\TinyTeX\bin\win32 to your PATH environment variable
- close and restart RStudio
- verify
```
> tinytex::tinytex_root()
[1] "C:\\Users\\username\\dev\\TinyTeX"
```
- test it, create an RMarkdown document from one of the templates. Compile and test different outputs, i.e. HTML and PDF.
  1. create a new temp folder in the R library (C:\Users\<user id>\dev\R\win-library\4.0\temp)
  2. create a new environment variable TMPDIR, set to C:\Users\<user id>\dev\R\win-library\4.0\temp
  3. set the correct temp folder in R (use 3.5, 3.6, 4.0 see above installation) and pass it to install.packages:
```
> configure.vars="TMPDIR=C:/Users/<user id>/dev/R/win-library/4.0/temp"
```

The `tidyverse` ecosystems

The tidyverse package includes a collection of R packages, among them

dplyr, for data manipulation
ggplot2, for data visualisation
readr, for data import
purrr, for functional programming
stringr, for strings manipulation
lubridate, for date/times/periods manipulation
... and more as described here.

Install all of them via

> install.packages("tidyverse")

`renv`

The renv package Let the cache location point to C:\Users\<username>\dev\renv via the environment variable RENV_PATHS_ROOT.

Also if you are using renv in a project you need to host it in a folder under your C:\users\<username>\dev, for example C:\users\<username>\dev\repos\my_project.

Using plotly in R

Exporting plots to image in plotly (for R) is not quite so easy. To do so you need to install Anaconda via the ServiceNow IT portal in EUROCONTROL. Once this is installed successfully you can use the following code to inititialize a conda environment with the right packages:

library(reticulate)

# Initialize a conda environment to use Kaleido
reticulate::conda_install('r-reticulate', 'python-kaleido')
reticulate::conda_install('r-reticulate', 'plotly', channel = 'plotly')

If this works successfully you can now create any graphs and save them using the save_image function from the plotly package. Note that, you need to use the plotly developer branch which has been patched by Quinten. You can install this patched branch as follows:

devtools::install_github("quinten-goens/plotly.R@fix/kaleido-export-bug")

Once this is installed you can output a graph to an image as follows:

# Import libraries
library(plotly)
library(reticulate)

# Create example plot
fig <- plot_ly(midwest, x = ~percollege, color = ~state, type = "box")

# Simulate a conda environment to use Kaleido
reticulate::use_condaenv('r-reticulate')

# Save the figure
save_image(fig,file="image.png", height=500, width=700)

If you have any issues.. Make sure your Anaconda version is up to date and that you have initialized the r-reticulate environment as described above with the plotly and kaleido-python packages.

ROracle

In order to be able to access our Oracle databases from R, you need to install the ROracle package.

The easiest way to install ROracle is to copy the ROracle folder in G:\HQ\dgof-pru\Project\Staff folders\spi\R-4.x (or \\sky.corp.eurocontrol.int\DFSRoot\Public\Haren\db_connectivity if you do not have access to G:\...) to where your R_LIBS_USER points to. (NOTE: this is a pre-compiled version for R-4.x)

If you insist to compile from source, follow on.

It needs source code compilation, and here are the steps to install it:

ROracle

From within RStudio

Check whether you development environment is ok:

> system('g++ -v')
Using built-in specs.
COLLECT_GCC=C:\PROGRA~1\Rtools\mingw_64\bin\G__~1.EXE
COLLECT_LTO_WRAPPER=C:/PROGRA~1/Rtools/mingw_64/bin/../libexec/gcc/x86_64-w64-mingw32/4.9.3/lto-wrapper.exe
Target: x86_64-w64-mingw32
Configured with: ../../../src/gcc-4.9.3/configure --host=x86_64-w64-mingw32 --build=x86_64-w64-mingw32 --target=x86_64-w64-mingw32 --prefix=/mingw64 --with-sysroot=/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64 --with-gxx-include-dir=/mingw64/x86_64-w64-mingw32/include/c++ --enable-static --disable-shared --disable-multilib --enable-languages=c,c++,fortran,lto --enable-libstdcxx-time=yes --enable-threads=posix --enable-libgomp --enable-libatomic --enable-lto --enable-graphite --enable-checking=release --enable-fully-dynamic-string --enable-version-specific-runtime-libs --disable-isl-version-check --disable-cloog-version-check --disable-libstdcxx-pch --disable-libstdcxx-debug --enable-bootstrap --disable-rpath --disable-win32-registry --disable-nls --disable-werror --disable-symvers --with-gnu-as --with-gnu-ld --with-arch=nocona --with-tune=core2 --with-libiconv --with-system-zlib --with-gmp=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static --with-mpfr=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static --with-mpc=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static --with-isl=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static --with-cloog=/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static --enable-cloog-backend=isl --with-pkgversion='x86_64-posix-seh, Built by MinGW-W64 project' --with-bugurl=http://sourceforge.net/projects/mingw-w64 CFLAGS='-O2 -pipe -I/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64/opt/include -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/include -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/include' CXXFLAGS='-O2 -pipe -I/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64/opt/include -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/include -I/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/include' CPPFLAGS= LDFLAGS='-pipe -L/home/Jeroen/mingw-gcc-4.9.3/x86_64-493-posix-seh-rt_v3-s/mingw64/opt/lib -L/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-zlib-static/lib -L/home/Jeroen/mingw-gcc-4.9.3/prerequisites/x86_64-w64-mingw32-static/lib '
Thread model: posix
gcc version 4.9.3 (x86_64-posix-seh, Built by MinGW-W64 project)

> system('where make')
C:\Program Files\Rtools\bin\make.exe

Install the devtools package:

> install.packages(c("devtools", "pkgbuild"))
> library(devtools)
> library(pkgbuild)
> find_rtools()
[1] TRUE

Install the ROracle package:

> install.packages("ROracle")

and test it:

> library(ROracle)
> drv <- dbDriver("Oracle")
> con <- dbConnect(drv, "prutest", "youknowthepassword", dbname="porape5")
> rs <- dbSendQuery(con, "select sysdate from dual")
> fetch(rs)
              SYSDATE
1 2016-11-04 12:58:25

and disconnect at the end:

> dbDisconnect(con)

Hugo

You need to install hugo to work with the portal.

> options(blogdown.hugo.dir = 'C:\\Users\\<user>\\dev\\R\\win-library\\R-4.2.1')
> blogdown::install_hugo("0.92.1")

Other packages

If you develop an R package you need the following tools (to be checked with IT Service Desk):

Ghostscript, qpdf and relevant ENV vars R_GSCMD, R_QPDF. See this link

Installation from source

When installing a package from source, it's possible that the installation fails when both 32 and 64-bit versions are intended to be installed. To overcome this, add type = "source", INSTALL_opts = "--no-multiarch" (e.g. install.packages("stars", type = "source", INSTALL_opts = "--no-multiarch").

Customization

Put the following in your .Rprofile (you can edit it via usethis::edit_r_profile())

.First <- function() {
  options(
    repos = c(CRAN = "https://cran.rstudio.com/"),
    keep.source = TRUE,
    keep.source.pkgs = TRUE,
    browserNLdisabled = TRUE,
    deparse.max.lines = 2,
    devtools.install.args = "--no-multiarch",
    # prompt
    prompt = "> ",
    # `continuee` prompt, a space is good for easy copy & paste
    continue = " "
    )
}

You can also copy the files from the dotfiles repo in your HOME directory in order to customize your command line experience (they are just about few Unix aliases).

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Tools Installation and Setup (For R)

Prerequisites

Developing on your computer

Environment Variables

Pre-installed tools and Configuration

Hacks

Checks

Packages

Configuration

Useful packages

Quarto

The `tinytex` and TeX/LaTeX

The `tidyverse` ecosystems

`renv`

Using plotly in R

ROracle

Hugo

Other packages

Installation from source

Customization

NEST

Setup

Workflows

Clone this wiki locally

Tools Installation and Setup (For R)

Prerequisites

Developing on your computer

Environment Variables

Pre-installed tools and Configuration

Hacks

Checks

Packages

Configuration

Useful packages

Quarto

The tinytex and TeX/LaTeX

The tidyverse ecosystems

renv

Using plotly in R

ROracle

Hugo

Other packages

Installation from source

Customization

NEST

Setup

Workflows

Clone this wiki locally

The `tinytex` and TeX/LaTeX

The `tidyverse` ecosystems

`renv`