Skip to content

Commit

Permalink
Enable connecting to PG within private VPCs using a VPN tunnel
Browse files Browse the repository at this point in the history
  • Loading branch information
exAspArk committed Dec 20, 2024
1 parent b1cbd0f commit 4418002
Show file tree
Hide file tree
Showing 5 changed files with 167 additions and 3 deletions.
2 changes: 2 additions & 0 deletions docs/docs/changelog.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ keywords: ['Bemi Changelog', 'Bemi New Features', 'Postgres Audit Trails', 'Chan

## 2024-12

* Platform
* Enable connecting to PostgreSQL database within private VPCs using a [VPN tunnel](https://docs.bemi.io/postgresql/source-database#vpn-tunnel)
* [Bemi Core](https://github.com/BemiHQ/bemi)
* Stitch context with out of order LSN positions

Expand Down
168 changes: 165 additions & 3 deletions docs/docs/postgresql/source-database.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,19 +76,181 @@ Note that this column will always be recorded if there were updated values in ot

## SSH Tunnel

If your PostgreSQL source database is not accessible over the internet, you can specify SSH credentials to enable an SSH tunnel via a jump host.
![](/img/new-source-db-ssh-arch.png)

If your PostgreSQL source database is not accessible over the internet, you can specify SSH credentials to enable an SSH tunnel via your public jump host:

![dashboard](/img/new-source-db-ssh.png)

Once the source database connection settings are submitted, we'll generate a public SSH key.
Once the source database connection settings are submitted, we'll generate a public SSH key:

![](/img/new-source-db-ssh-key.png)

Add this public SSH key to your SSH host to allow Bemi workers to connect and SSH-tunnel to the PostgreSQL database:

```sh
touch ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys
echo 'ssh-ed25519 AAAAC3Nz...' >> ~/.ssh/authorized_keys
```

If you need a public SSH Key before you know the SSH host address, just specify any address and later reach out to us to update it.
Note: if you need a public SSH Key before you know the SSH host address, just specify any address and later reach out to us to update it.

## VPN Tunnel

If your PostgreSQL source database and SSH jump host can't be accessible over the internet, you can establish a secure VPN tunnel to our VPN server.

![](/img/new-source-db-vpn-arch.png)

You need to provision a VPN node and configure VPN using [WireGuard](https://www.wireguard.com/) that combines strong encryption with fast speeds:

```sh
# Install WireGuard
sudo apt update && sudo apt install -y resolvconf wireguard

# Generate a private and public keys
wg genkey | tee wg-privatekey | wg pubkey > wg-publickey

# Get your network interface name (e.g., ens5)
NETWORK_INTERFACE=$(ip route get 8.8.8.8 | grep 8.8.8.8 | awk '{print $5}')

# Reach out to us to get the VPN server public key and endpoint
BEMI_PUBLIC_KEY=
BEMI_ENDPOINT=

# Send us your public key and PostgreSQL private IP address
cat wg-publickey
dig your-prod-postgres.abcdef5ghij.us-west-1.rds.amazonaws.com A +short | tail -n 1

# Enable IP forwarding
echo "net.ipv4.ip_forward = 1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p

# Create a WireGuard configuration file with the following content
cat << EOF > wg0.conf
[Interface]
PrivateKey = $(cat ./wg-privatekey)
Address = 10.0.0.1/24
PostUp = iptables -t nat -A POSTROUTING -o $NETWORK_INTERFACE -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o $NETWORK_INTERFACE -j MASQUERAD
[Peer]
PublicKey = $BEMI_PUBLIC_KEY
AllowedIPs = 10.0.0.2/32
Endpoint = $BEMI_ENDPOINT:51820
PersistentKeepalive = 25
EOF
sudo mv wg0.conf /etc/wireguard/wg0.conf

sudo systemctl enable wg-quick@wg0.service
sudo systemctl start wg-quick@wg0.service
systemctl status wg-quick@wg0.service
```

Once you have the WireGuard running, we'll be able to connect to your PostgreSQL database over the VPN tunnel.

Here is a Terraform snippet to run all the commands above on AWS:

```tf
locals {
# For example, Ubuntu 24.04 LTS - arm64
ami = "ami-..."
instance_type = "t4g.micro"
# Your private subnet ID
private_subnet_id = "subnet-..."
# Bemi VPN server information
bemi_public_key = "..."
bemi_endpoint = "..."
# Optionally if you want to SSH to the VPN node
ssh_key_name = "..."
ssh_cidr_blocks = [".../32"]
}
resource "aws_security_group" "bemi-vpn-node" {
name = "bemi-vpn-node"
description = "Security group for bemi-vpn-node"
ingress {
description = "SSH"
from_port = 22
to_port = 22
protocol = "tcp"
cidr_blocks = local.ssh_cidr_blocks
}
egress {
from_port = 0
to_port = 0
protocol = "-1"
cidr_blocks = ["0.0.0.0/0"]
ipv6_cidr_blocks = ["::/0"]
}
tags = {
Name = "bemi-vpn-node"
}
}
resource "aws_instance" "bemi-vpn-node" {
ami = local.ami
instance_type = local.instance_type
key_name = local.ssh_key_name
associate_public_ip_address = false
subnet_id = local.private_subnet_id
security_groups = [aws_security_group.bemi-vpn-node.id]
tags = {
Name = "bemi-vpn-node"
}
user_data = <<-EOF
#!/bin/bash
sudo apt update
sudo apt install -y resolvconf wireguard
cd /home/ubuntu
wg genkey | tee wg-privatekey | wg pubkey > wg-publickey
echo "net.ipv4.ip_forward = 1" | sudo tee -a /etc/sysctl.conf
sudo sysctl -p
cat << EOF_WG_CONF > wg0.conf
[Interface]
PrivateKey = $(cat ./wg-privatekey)
Address = 10.0.0.1/24
PostUp = iptables -t nat -A POSTROUTING -o ens5 -j MASQUERADE
PostDown = iptables -t nat -D POSTROUTING -o ens5 -j MASQUERAD
[Peer]
PublicKey = ${local.bemi_public_key}
AllowedIPs = 10.0.0.2/32
Endpoint = ${local.bemi_endpoint}:51820
PersistentKeepalive = 25
EOF_WG_CONF
sudo mv wg0.conf /etc/wireguard/wg0.conf
sudo systemctl enable wg-quick@wg0.service
sudo systemctl start wg-quick@wg0.service
EOF
}
```

To check the WireGuard status, SSH to the VPN node and run:

```sh
# Check the WireGuard status
systemctl status wg-quick@wg0.service
sudo wg show

# Check the WireGuard configuration file
sudo cat /etc/wireguard/wg0.conf

# Check your public WireGuard key and share it with us
cat ./wg-publickey
```

## Bemi Static IPs

Expand Down
Binary file added docs/static/img/new-source-db-ssh-arch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/new-source-db-ssh-key.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/static/img/new-source-db-vpn-arch.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.

0 comments on commit 4418002

Please sign in to comment.