diff --git a/.nojekyll b/.nojekyll index 8c45611..63a86de 100644 --- a/.nojekyll +++ b/.nojekyll @@ -1 +1 @@ -8a56bfb1 \ No newline at end of file +c88b014a \ No newline at end of file diff --git a/preserve_computing-env.html b/preserve_computing-env.html index 2d83e97..31e7a70 100644 --- a/preserve_computing-env.html +++ b/preserve_computing-env.html @@ -20,6 +20,40 @@ margin: 0 0.8em 0.2em -1em; /* quarto-specific, see https://github.com/quarto-dev/quarto-cli/issues/4556 */ vertical-align: middle; } +/* CSS for syntax highlighting */ +pre > code.sourceCode { white-space: pre; position: relative; } +pre > code.sourceCode > span { line-height: 1.25; } +pre > code.sourceCode > span:empty { height: 1.2em; } +.sourceCode { overflow: visible; } +code.sourceCode > span { color: inherit; text-decoration: inherit; } +div.sourceCode { margin: 1em 0; } +pre.sourceCode { margin: 0; } +@media screen { +div.sourceCode { overflow: auto; } +} +@media print { +pre > code.sourceCode { white-space: pre-wrap; } +pre > code.sourceCode > span { display: inline-block; text-indent: -5em; padding-left: 5em; } +} +pre.numberSource code + { counter-reset: source-line 0; } +pre.numberSource code > span + { position: relative; left: -4em; counter-increment: source-line; } +pre.numberSource code > span > a:first-child::before + { content: counter(source-line); + position: relative; left: -1em; text-align: right; vertical-align: baseline; + border: none; display: inline-block; + -webkit-touch-callout: none; -webkit-user-select: none; + -khtml-user-select: none; -moz-user-select: none; + -ms-user-select: none; user-select: none; + padding: 0 4px; width: 4em; + } +pre.numberSource { margin-left: 3em; padding-left: 4px; } +div.sourceCode + { } +@media screen { +pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; } +} @@ -405,9 +439,153 @@

Preserve your computing environment

pip freeze > requirements.txt will capture all the python modules installed in your current environment

Virtual environments

-

You can even go a step further and help others to recreate the same computing environment that you used independently of what versions you have installed on your machine: https://rstudio.github.io/renv/articles/renv.html

+

You can even go a step further and help others to recreate the same computing environment that you used independently of what versions you have installed on your machine.

+

Level up!

+
+

R

+

Here is a short introduction to renv, an R package that creates virtual environment to encapsulate your R work. source: https://rstudio.github.io/renv/articles/renv.html

+

We are going to add renv to our shorebird data cleaning project. Make sure you have the renv package installed:

+
install.packages("renv")
+

You may also choose Tools>Project Options>Environments and check “use renv for this project”

+

Or at the R console:

+
renv::init( )
+

Let’s check the new files that we have…

+ +

Let’s say we reworked our script and suppose we would need the naniar R package to deal with the NAs. Let’s update or virtual environment:

+
renv::install("naniar")
+

Add some R code using this package, for example

+
library(naniar)
+
+snowsurvey_csv %>% 
+  miss_var_summary()
+

Alright, now that the installation is completed and we added this package to our code, we can save, and take a snapshot:

+
renv::snapshot()
+

This action will update the lock file, and we will see naniar and all its dependencies included.

+

Let’s check. If updated, you should be good to go. If your attempts to update packages introduced new problems, you may run renv::restore() to revert to the previous state as encoded in the lockfile.

+

Use .libPaths() to confirm where package installations are located!

+

Want to know more? here is a good resource to get started: https://rstudio.github.io/renv/articles/renv.html

+
+
+

Python

+

virtualenv is a tool to create Python virtual environments. In a nutsheel here are the steps to follow:

+

Create a new folder, your project folder. I will use EDS213. Make sure you set the path to this folder.

+ +

To create the environment: (second venv is the name of the environment)

+
python3 -m venv venv213
+ +

Time to activate it:

+
source venv213/bin/activate
+

You can tell it is activated because it shows (venv213) in the prompt.

+

Let’s check which packages are there with a new pip list

+

Nothing, right? Only setup tools, and pip). Nothing to worry about, it should be this way! Let’s proceed.

+

Install libraries:

+

We will be installing two packages for this exercise.

+

First:

+
pip install numpy
+

And then:

+
pip install pandas
+

This should take a little longer!

+

Another pip list

+

Alright, the packages and dependencies installed are right there!

+

Export and allow future replication of the environment:

+

Let’s save the packages and dependencies we have after the installs.

+
pip freeze
+

That should be stored in a `requirements.txt file

+

So let’s get it redirected to the required file:

+
pip freeze > requirements.txt
+

Question: This file won’t leave inside the venv folder, but rather in the project root folder any idea why?

+

Well, you only need that file to reproduce the environment. And the venv should be should throw away and be able to rebuild easily! So, do not include any project file in that folder and treat that as disposable after the pip freeze

+

To double-check if all is good, we can run the following command:

+
cat requirements.txt
+

This file should be included in your repository to let others reinstall your packages and dependencies as needed.

+

Deactivate

+

If you are done with that, you should deactivate that environment by typing:

+
deactivate
+

Then, you will see you no longer have the environment we created in our prompt.

+

If you are willing to delete the environment altogether, you should delete the directory for the virtual environment

+

Remove folder:

+
rm -rf venv213/
+

Reusing the Requirements

+

Create a new project folder to reuse the requirements

+
mkdir my-project
+

Create a virtual env for it

+
python3 -m venv my-project/venv
+

Activate it

+
source my-project/venv/bin/activate
+

Install required packages

+
pip install -r requirements.txt
+ +
+
+

If you are using Conda

+

Checking what is in the system:

+

conda list

+

To create the environment:

+
conda create --name env213
+

proceed ([y]/n)? y

+

Time to activate it

+

conda activate env213 only work on conda 4.6 and later versions. For Conda versions prior to 4.6, run:

+ +

You can tell it is activated because it shows (venv213) in the prompt.

+

Let’s check which packages are there with a new

+
conda list env213
+

Empty, right? Nothing to worry about, it should be this way! Let’s proceed.

+

Install packages:

+

We will be installing two packages for this exercise.

+

First:

+
conda install numpy
+

proceed ([y]/n)? y

+

And then, one more package:

+
conda install Pandas
+

proceed ([y]/n)? y

+

Now check which packages are in the specific environment we are working on:

+
conda list
+

Alright, the packages and dependencies installed are right there!

+

Export and allow future replication of the environment:

+

Let’s save the packages and dependencies we have after the installs.

+
conda list --export
+

That should be stored in a environments.yml file

+

So let’s get it redirected to the required file:

+
conda list -e > environment.yml
+

This file should be included in your research compendium to let others reinstall your packages and dependencies as needed.

+

Deactivate it:

+

If you are done with that, you should deactivate that environment by typing:

+
conda deactivate
+

Note: only works on conda 4.6 and later versions. For conda versions before 4.6, run:

+

Windows: deactivate or Linux and macOS: source deactivate

+

Then, you will see you no longer have the environment we created in our prompt.

+

If you are willing to delete the environment altogether, you should delete the directory for the virtual environment

+

Back to base we can create a new environment based on the .yml packages and dependencies by running (this is noted on top of the yml file):

+
conda create --name my-env --file environment.yml
+

proceed ([y]/n)? y

+

Activate it:

+
conda activate my-env (or see above if conda version \< 4.6)
+

proceed ([y]/n)? Y

+

Check if packages are there:

+
conda list
+

Remember to deactivate it when done:

+
conda deactivate (or see above if conda version \< 4.6)
+

Check all your environments conda env list

+

More info: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html

+
diff --git a/search.json b/search.json index 43d6f3c..bf7c4a2 100644 --- a/search.json +++ b/search.json @@ -576,7 +576,7 @@ "href": "preserve_computing-env.html", "title": "Preserve your computing environment", "section": "", - "text": "Your analysis was done with specific versions of the program used (e.g. R 4.4.3) but also of all the packages involved, as well as the specifications of the Operating System (OS) that was used. The good use is that there are tools to let you systematically capture this information.\nIn R:\nsessionInfo() or devtools::session_info() are great ways to capture this information. You should save it into a session_info.txt file to include in your GitHub repository\nIn Python:\npip freeze > requirements.txt will capture all the python modules installed in your current environment\n\nVirtual environments\nYou can even go a step further and help others to recreate the same computing environment that you used independently of what versions you have installed on your machine: https://rstudio.github.io/renv/articles/renv.html", + "text": "Your analysis was done with specific versions of the program used (e.g. R 4.4.3) but also of all the packages involved, as well as the specifications of the Operating System (OS) that was used. The good use is that there are tools to let you systematically capture this information.\nIn R:\nsessionInfo() or devtools::session_info() are great ways to capture this information. You should save it into a session_info.txt file to include in your GitHub repository\nIn Python:\npip freeze > requirements.txt will capture all the python modules installed in your current environment\n\nVirtual environments\nYou can even go a step further and help others to recreate the same computing environment that you used independently of what versions you have installed on your machine.\nLevel up!\n\nR\nHere is a short introduction to renv, an R package that creates virtual environment to encapsulate your R work. \nWe are going to add renv to our shorebird data cleaning project. Make sure you have the renv package installed:\ninstall.packages(\"renv\")\nYou may also choose Tools>Project Options>Environments and check “use renv for this project”\nOr at the R console:\nrenv::init( )\nLet’s check the new files that we have…\n\nLook at the .gitignore\nLook at the renv.lock file\n\nLet’s say we reworked our script and suppose we would need the naniar R package to deal with the NAs. Let’s update or virtual environment:\nrenv::install(\"naniar\")\nAdd some R code using this package, for example\nlibrary(naniar)\n\nsnowsurvey_csv %>% \n miss_var_summary()\nAlright, now that the installation is completed and we added this package to our code, we can save, and take a snapshot:\nrenv::snapshot()\nThis action will update the lock file, and we will see naniar and all its dependencies included.\nLet’s check. If updated, you should be good to go. If your attempts to update packages introduced new problems, you may run renv::restore() to revert to the previous state as encoded in the lockfile.\nUse .libPaths() to confirm where package installations are located!\nWant to know more? here is a good resource to get started: https://rstudio.github.io/renv/articles/renv.html\n\n\nPython\nvirtualenv is a tool to create Python virtual environments. In a nutsheel here are the steps to follow:\nCreate a new folder, your project folder. I will use EDS213. Make sure you set the path to this folder.\n\nNote: The environment will be created in the current version of Python that you are running (in Conda we can specify the version we want).\n\nTo create the environment: (second venv is the name of the environment)\npython3 -m venv venv213\n\nThe -m flag makes sure you are creating a pip that is tied to the active Python executable\n\nTime to activate it:\nsource venv213/bin/activate\nYou can tell it is activated because it shows (venv213) in the prompt.\nLet’s check which packages are there with a new pip list\nNothing, right? Only setup tools, and pip). Nothing to worry about, it should be this way! Let’s proceed.\nInstall libraries:\nWe will be installing two packages for this exercise.\nFirst:\npip install numpy\nAnd then:\npip install pandas\nThis should take a little longer!\nAnother pip list\nAlright, the packages and dependencies installed are right there!\nExport and allow future replication of the environment:\nLet’s save the packages and dependencies we have after the installs.\npip freeze\nThat should be stored in a `requirements.txt file\nSo let’s get it redirected to the required file:\npip freeze > requirements.txt\nQuestion: This file won’t leave inside the venv folder, but rather in the project root folder any idea why?\nWell, you only need that file to reproduce the environment. And the venv should be should throw away and be able to rebuild easily! So, do not include any project file in that folder and treat that as disposable after the pip freeze\nTo double-check if all is good, we can run the following command:\ncat requirements.txt\nThis file should be included in your repository to let others reinstall your packages and dependencies as needed.\nDeactivate\nIf you are done with that, you should deactivate that environment by typing:\ndeactivate\nThen, you will see you no longer have the environment we created in our prompt.\nIf you are willing to delete the environment altogether, you should delete the directory for the virtual environment\nRemove folder:\nrm -rf venv213/\nReusing the Requirements\nCreate a new project folder to reuse the requirements\nmkdir my-project\nCreate a virtual env for it\npython3 -m venv my-project/venv\nActivate it\nsource my-project/venv/bin/activate\nInstall required packages\npip install -r requirements.txt\n\nAttention! Never include project files in the venv folder.\nDo not commit your venv file to the environment itself to source control (git ignore)\nYou may install more packages and update the requirements.txt with the pip freeze command\nYou should commit /share only your requirements.txt file. That is all that others and your future self need to recreate the environment.\nThe environment should be something you should throw away and be able to rebuild easily.\nMake sure to deactivate when done using it.\n\n\n\nIf you are using Conda\nChecking what is in the system:\nconda list\nTo create the environment:\nconda create --name env213\nproceed ([y]/n)? y\nTime to activate it\nconda activate env213 only work on conda 4.6 and later versions. For Conda versions prior to 4.6, run:\n\nWindows: activate or Linux and macOS: source activate\n\nYou can tell it is activated because it shows (venv213) in the prompt.\nLet’s check which packages are there with a new\nconda list env213\nEmpty, right? Nothing to worry about, it should be this way! Let’s proceed.\nInstall packages:\nWe will be installing two packages for this exercise.\nFirst:\nconda install numpy\nproceed ([y]/n)? y\nAnd then, one more package:\nconda install Pandas\nproceed ([y]/n)? y\nNow check which packages are in the specific environment we are working on:\nconda list\nAlright, the packages and dependencies installed are right there!\nExport and allow future replication of the environment:\nLet’s save the packages and dependencies we have after the installs.\nconda list --export\nThat should be stored in a environments.yml file\nSo let’s get it redirected to the required file:\nconda list -e > environment.yml\nThis file should be included in your research compendium to let others reinstall your packages and dependencies as needed.\nDeactivate it:\nIf you are done with that, you should deactivate that environment by typing:\nconda deactivate\nNote: only works on conda 4.6 and later versions. For conda versions before 4.6, run:\nWindows: deactivate or Linux and macOS: source deactivate\nThen, you will see you no longer have the environment we created in our prompt.\nIf you are willing to delete the environment altogether, you should delete the directory for the virtual environment\nBack to base we can create a new environment based on the .yml packages and dependencies by running (this is noted on top of the yml file):\nconda create --name my-env --file environment.yml\nproceed ([y]/n)? y\nActivate it:\nconda activate my-env (or see above if conda version \\< 4.6)\nproceed ([y]/n)? Y\nCheck if packages are there:\nconda list\nRemember to deactivate it when done:\nconda deactivate (or see above if conda version \\< 4.6)\nCheck all your environments conda env list\nMore info: https://conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html", "crumbs": [ "Preserving things", "Preserve your computing environment" diff --git a/sitemap.xml b/sitemap.xml index f9bf1d2..2f26108 100644 --- a/sitemap.xml +++ b/sitemap.xml @@ -2,122 +2,122 @@ https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_self.html - 2024-06-27T22:57:33.056Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/github_org.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/intro_cli.html - 2024-06-27T22:57:33.052Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/about.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_readme.html - 2024-06-27T22:57:33.056Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/github_teams.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/coding.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_data-fair.html - 2024-06-27T22:57:33.052Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/index.html - 2024-06-27T22:57:33.052Z + 2024-06-27T23:51:09.684Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/github_intro.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/data_mgmt.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/git_cli.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_prompts.html - 2024-06-27T22:57:33.056Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/git_further_readings.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/collab_exchange.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_computing-env.html - 2024-06-27T22:57:33.052Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_data.html - 2024-06-27T22:57:33.056Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/01-handson_github_website.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/03-handson_github_workflows.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/git_conflicts.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/collab_pairprog.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_code.html - 2024-06-27T22:57:33.052Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/datamgmt_prompts.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/git_rstudio.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/datamgmt_manage.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/datamgmt_plan.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/github_template.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/preserve_license.html - 2024-06-27T22:57:33.056Z + 2024-06-27T23:51:09.688Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/02-handson_github_rstudio.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z https://UCSB-Library-Research-Data-Services.github.io/reproducible-lab/github_workflows.html - 2024-06-27T22:57:32.928Z + 2024-06-27T23:51:09.560Z