Skip to content

Latest commit

 

History

History
283 lines (209 loc) · 8.71 KB

CONTRIBUTING.md

File metadata and controls

283 lines (209 loc) · 8.71 KB

Contributing

This bot is a Python based Discord bot built on top of the discord.py library.

It uses poetry to manage dependencies. To install development dependencies use: poetry install. This will allow you to run PyTest and the included scripts.

You can install poetry with the script install-poetry.py:

curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/install-poetry.py | python -

Setting up environment

The easiest thing to do is to set up a virtual python environment specifically for spellbot.

python -m venv .venv

Then you can install poetry into this environment with:

.venv/bin/pip install --upgrade pip
.venv/bin/pip install poetry

Now configure poetry to use in the virtual environment we already created:

.venv/bin/poetry config virtualenvs.create false
.venv/bin/poetry env use .venv/bin/python

To make this easier I've also included a scripts/setup.sh to do this automatically.

Installing the application

Install in development mode via poetry:

source .venv/bin/activate
poetry install

Note: On some systems such as most Linux ones you may also need to install python3-venv using your system's package manager as poetry depends on it.

Launch Dependencies

SpellBot requires a database to run properly. The connection string for the database you want to use must be stored in the environment variable DATABASE_URL.

You can start a local database via Docker by running:

docker run -i --rm -p 5432:5432 -e POSTGRES_HOST_AUTH_METHOD=trust postgres:15

Using this locally will allow you to use the default value for DATABASE_URL without having to manually set it to anything.

Running the application

Make sure that you have set up your environmental variables.

If you wish, you can use a .env file. Copy the .env.example file to get started.

When your environmental variables are set, run:

source .venv/bin/activate
poetry run spellbot --help

This will list some useful flags you can provide to run SpellBot. To get started developing, run:

source .venv/bin/activate
poetry run spellbot --dev

This will start SpellBot and reload it whenever the source code changes.

Running tests

source .venv/bin/activate
poetry run pytest --cov --cov-report=html
open coverage/index.html

Formatting and linting

Codebase consistency is maintained by ruff.

source .venv/bin/activate
poetry run pytest -k codebase

Interactive Shell

An interactive shell using IPyhton can be started by running:

poetry run python shell.py

From this shell you will be able to interact with the database using SpellBot models and code. For example:

$ poetry run python shell.py

In [1]: DatabaseSession.query(User).all()
Out[1]: []

Release process

There's two methods for doing a release. You can use a script to handle everything for you automatically, or you can basically do every step in that script manually. Both methods are described below but I recommend the script.

Scripted

To do a release automatically there is a *NIX script available in the scripts directory to help. To use it you will need to have non-interactive poetry publish enabled by running:

source .venv/bin/activate
poetry config pypi-token.pypi "YOUR-PYPI-TOKEN-GOES-HERE"

If you don't have one, you can create your PyPI token for this command by going to the PyPI settings for spellbot and clicking on the Create a token for spellbot button there. Of course you will have to be a collaborator for this project on PyPI to be able to do this. Contact spellbot@lexicalunit.com to be added to the project.

Once you have that set up, you can release a new version by running:

scripts/publish.sh <major | minor | patch>

You must select either major, minor, or patch as the release kind. Please follow semver for guidance on what kind of release to make. But basically:

  • Major: Breaking changes.
  • Minor: New features.
  • Patch: Bug fixes.

Manually

To release a new version of spellbot, use poetry:

source .venv/bin/activate
poetry version [major|minor|patch]
poetry run pytest # verify that all tests pass
# edit the CHANGELOG.md file to promote all unlreased changes to the new version
poetry build
git commit -am "Release vM.N.P"
poetry publish
git tag 'vM.N.P'
git push --tags origin main

Note: The reason you should run pytest after running the poetry version command is to ensure that all test still pass after the version is updated.

You can get the M.N.P version numbers from pyproject.toml after you've run the poetry version command. On a *NIX shell you could also get it automatically like so:

grep "^version" < pyproject.toml | cut -d= -f2 | sed 's/"//g;s/ //g;s/^/v/;'

When you use the poetry publish command you will be prompted for your PyPI credentials.

After publishing you can view the package at its pypi.org project page to see that everything looks good.

Database migrations

We use alembic for database migrations. It can detect changes you've made compared to an existing database and generate migration scripts necessary to apply and reverse those changes. First, make the changes to the data models. Alembic can detect differences between an existing database and changes made to the models. To autogenerate migration scripts that will bring the database inline with the changes you've made to the models, run:

poetry run scripts/create_db_revision.py \
    "<your-sqlalchemy-database-url>" \
    "<Some description of your changes>"

Note: An example database url: postgresql://postgres@localhost:5432/postgres

This will create a revision script in the src/spellbot/versions/versions directory with a name like REVISIONID_some_description_of_your_changes.py. You may have to edit this script manually to ensure that it is correct as the autogenerate facility of alembic revision is not perfect.

Downgrading

Another migration script is scripts/downgrade.py which allows you to pass a revision string to downgrade to. For example, to undo the last migration you could run something like:

poetry run scripts/downgrade.py postgresql://postgres@localhost:5432/postgres "-1"

Metrics

SpellBot is set up to integrate with DataDog for system and service metrics. This requires a few things:

  1. In the container, we install and configure to run the following services:

  2. A source dependency on the python Datadog APM Client: ddtrace.

  3. Instrumentation of code using the ddtrace tracer interface.

  4. A deployment with the correct environment variables to configure dd-agent.

    • DD_API_KEY = Your DataDog API key.
    • DD_APP_KEY = Your DataDog application key.
    • DD_AGENT_MAJOR_VERSION = "7"
    • DD_HOSTNAME = "spellbot"
    • DD_ENV = "dev", "stage", or "prod"

  5. A datadog.yaml file including the following configuration:

    apm_config:
  enabled: true
  apm_non_local_traffic: true

  6. When starting the spellbot process, start it via ddtrace-run spellbot so that the APM client hooks are properly installed.

Locally in development you can use the following invocation to quickly spin up a dd-agent. And to set up your environment you should use the .env file.

docker run --rm \
    --name dd-agent \
    -p8125:8125 \
    -p8126:8126 \
    -v /var/run/docker.sock:/var/run/docker.sock:ro \
    -v /proc/:/host/proc/:ro \
    -v /sys/fs/cgroup/:/host/sys/fs/cgroup:ro \
    -e DD_API_KEY="Your DataDog API key" \
    -e DD_APP_KEY="Your DataDog application key" \
    -e DD_AGENT_MAJOR_VERSION="7" \
    -e DD_SERVICE="spellbot" \
    -e DD_HOSTNAME="localhost" \
    -e DD_ENV="dev" \
    gcr.io/datadoghq/agent:7