Add devcontainer for development purposes #7102

thanodnl · 2023-08-03T13:41:52Z

This change adds a devcontainer configuration to the Citus project. This devcontainer allows for quick generation of isolated development environments, either local on the machine of a developer or in a cloud, like github codepaces.

The devcontainer is updated automatically by github actions when its configuration changes.

For more detailed instructions on how to quickstart the development in a container see CONTRIBUTING.md

FOR THE REVIEWERS: make sure that you open this branch when cloning the devcontainer. This requires not pasting in the github url, but instead go through the wizard step by step. eg. first selecting github, then selecting the repository, lastly selecting the branch.

codecov · 2023-09-05T13:53:36Z

Codecov Report

Merging #7102 (0623c64) into main (6d8725e) will decrease coverage by 0.01%.
The diff coverage is n/a.

❗ Current head 0623c64 differs from pull request most recent head 214bed0. Consider uploading reports for the commit 214bed0 to get more accurate results

@@            Coverage Diff             @@
##             main    #7102      +/-   ##
==========================================
- Coverage   93.23%   93.22%   -0.01%     
==========================================
  Files         275      275              
  Lines       59485    59487       +2     
==========================================
+ Hits        55458    55459       +1     
- Misses       4027     4028       +1

.devcontainer/devcontainer.json

.devcontainer/Dockerfile

.devcontainer/.vscode/generate_c_cpp_properties-json.py

.devcontainer/.vscode/Pipfile

.devcontainer/.gdbinit

aykut-bozkurt · 2023-09-07T11:37:16Z

@aytekinar please feel free to review

aykut-bozkurt

Thanks for great initiative to make it easy to contribute to project!

.devcontainer/.vscode/launch.json

src/test/regress/Pipfile

aykut-bozkurt · 2023-09-07T18:08:03Z

Adding ssh keys from host to container could also help us to run git from terminal. Otherwise, only option is to use UI login and operations.

thanodnl · 2023-09-08T12:44:56Z

Adding ssh keys from host to container could also help us to run git from terminal. Otherwise, only option is to use UI login and operations.

You will want to have a look at https://code.visualstudio.com/remote/advancedcontainers/sharing-git-credentials#_using-ssh-keys on how to do this. This needs to be done outside of the devcontainer. With ssh-agent running vscode will automatically forward the keys of your machine to the container.

.devcontainer/Dockerfile

JelteF · 2023-10-09T11:50:43Z

CONTRIBUTING.md

+
+To quickly start we suggest splitting your terminal once to have two shells. The left one in the `/workspaces/citus`, the second one changed to `/data`. The left terminal will be used to interact with the project, the right one with a testing cluster.
+
+To get citus installed from source we run `make install -sj8` in the first terminal, adjust `j8` to suite to the number of cores you have available. Once installed you can start a Citus cluster in the second terminal via `citus_dev make citus`. The cluster will run in the background, and can be interacted with via `citus_dev`. To get an overview of the available commands.


Probably nicer to add export MAKEFLAGS="-j $(nproc)" to bashrc/zshrc

Or probably even better. Adding it to our make file: https://stackoverflow.com/a/60413199/2570866

I have added it to .bashrc for now, and also use the same ENV for building postgres, to allow for higher parallelism on bigger boxes.

We can later evaluate if adding it to our makefile makes sense, since that's a change outside of the devcontainer scope. I tried to keep this change as isolated from the rest of the codebase as possible to make it easy for rubber stamping.

Also some minor cleanup of last items in the Dockerfile, like the timezone etc.

which will build when merged

This change adds a devcontainer configuration to the Citus project. This devcontainer allows for quick generation of isolated development environments, either local on the machine of a developer or in a cloud, like github codepaces. The devcontainer is updated automatically by github actions when its configuration changes. For more detailed instructions on how to quickstart the development in a container see CONTRIBUTING.md

thanodnl force-pushed the dev/devcontainer branch from bd1d48d to e2652d9 Compare August 18, 2023 12:13

thanodnl marked this pull request as ready for review August 18, 2023 14:21

thanodnl force-pushed the dev/devcontainer branch 4 times, most recently from c47ce66 to ca64edb Compare August 29, 2023 12:49

thanodnl force-pushed the dev/devcontainer branch from ca64edb to 5bdf946 Compare September 4, 2023 18:14

thanodnl force-pushed the dev/devcontainer branch from 4f0f212 to 099f272 Compare September 5, 2023 14:05

thanodnl commented Sep 5, 2023

View reviewed changes

aykut-bozkurt reviewed Sep 7, 2023

View reviewed changes

.devcontainer/.vscode/launch.json Show resolved Hide resolved

src/test/regress/Pipfile Outdated Show resolved Hide resolved

thanodnl force-pushed the dev/devcontainer branch 2 times, most recently from 042cd2b to 1d2dc5d Compare September 21, 2023 10:46

zhjwpku reviewed Sep 25, 2023

View reviewed changes

.devcontainer/Dockerfile Outdated Show resolved Hide resolved

thanodnl force-pushed the dev/devcontainer branch 4 times, most recently from f9f08c8 to 04523ff Compare September 27, 2023 13:46

thanodnl force-pushed the dev/devcontainer branch from 2e7524e to 1170f01 Compare October 5, 2023 11:33

JelteF approved these changes Oct 9, 2023

View reviewed changes

JelteF reviewed Oct 9, 2023

View reviewed changes

thanodnl added 6 commits October 9, 2023 13:51

initial setup for devcontianer

93174e6

make debugger work

c094bce

collapse all package installs

e590c94

add vim for git commit messages

8ab77c7

use prebuilt image

626fb1f

fix line endings

d12e324

thanodnl added 22 commits October 9, 2023 13:51

cancel concurrent builds of devcontainers for the same branch

b21c57c

document what the concurency does in devcontainer builds

807a4d6

use pipenv script for citus_dev

a6cb3f8

devcontainer still uses a pipenv-shebang

6780dcc

bump container

603cdf6

bump postgres minor versions

497807a

use stable release of pgenv

b44ccb2

fix python imports

298a55b

fix flake8 linter issues

4cfee75

use pg16rc1

a361618

use mainline for citus tools, now with citus_dev-pipenv

fbadd28

various tweaks to devcontainer

039df87

undo --link --chown copies

2883a29

add tree command

d74a61d

use released pg16.0

6e6c138

remove redundant package

fb8917d

dont interrupt gdb on SIGUSR1

1994c75

add citus_dev bash completions

6e14466

only seem to need SYS_PTRACE, if not we can revert later

ae1028e

persist bash history

b59afbb

Add python3.9 for the regress Pipfile

54379d5

Also some minor cleanup of last items in the Dockerfile, like the timezone etc.

use python3.9 for vscode script

c072915

thanodnl force-pushed the dev/devcontainer branch from 1170f01 to c072915 Compare October 9, 2023 11:51

thanodnl added 3 commits October 9, 2023 12:07

globally configure make to use the available cores

b76c873

parallelism for postgres builds

0623c64

Use tagges container from the main branch

214bed0

which will build when merged

thanodnl enabled auto-merge (squash) October 9, 2023 13:27

thanodnl merged commit 76fdfa3 into main Oct 9, 2023
31 checks passed

thanodnl deleted the dev/devcontainer branch October 9, 2023 13:37

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add devcontainer for development purposes #7102

Add devcontainer for development purposes #7102

thanodnl commented Aug 3, 2023 •

edited

Loading

codecov bot commented Sep 5, 2023 •

edited

Loading

aykut-bozkurt commented Sep 7, 2023

aykut-bozkurt left a comment

aykut-bozkurt commented Sep 7, 2023

thanodnl commented Sep 8, 2023 •

edited

Loading

JelteF Oct 9, 2023 •

edited

Loading

JelteF Oct 9, 2023 •

edited

Loading

thanodnl Oct 9, 2023


		To quickly start we suggest splitting your terminal once to have two shells. The left one in the `/workspaces/citus`, the second one changed to `/data`. The left terminal will be used to interact with the project, the right one with a testing cluster.

		To get citus installed from source we run `make install -sj8` in the first terminal, adjust `j8` to suite to the number of cores you have available. Once installed you can start a Citus cluster in the second terminal via `citus_dev make citus`. The cluster will run in the background, and can be interacted with via `citus_dev`. To get an overview of the available commands.

Add devcontainer for development purposes #7102

Add devcontainer for development purposes #7102

Conversation

thanodnl commented Aug 3, 2023 • edited Loading

codecov bot commented Sep 5, 2023 • edited Loading

Codecov Report

aykut-bozkurt commented Sep 7, 2023

aykut-bozkurt left a comment

Choose a reason for hiding this comment

aykut-bozkurt commented Sep 7, 2023

thanodnl commented Sep 8, 2023 • edited Loading

JelteF Oct 9, 2023 • edited Loading

Choose a reason for hiding this comment

JelteF Oct 9, 2023 • edited Loading

Choose a reason for hiding this comment

thanodnl Oct 9, 2023

Choose a reason for hiding this comment

thanodnl commented Aug 3, 2023 •

edited

Loading

codecov bot commented Sep 5, 2023 •

edited

Loading

thanodnl commented Sep 8, 2023 •

edited

Loading

JelteF Oct 9, 2023 •

edited

Loading

JelteF Oct 9, 2023 •

edited

Loading