Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: azure deployment #217

Merged
merged 45 commits into from
Dec 8, 2024
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
5c1fae4
chore: vercel
gentlementlegen Dec 2, 2024
e41fbb5
Add or update the Azure App Service build and deployment workflow config
gentlementlegen Dec 2, 2024
61c8985
chore: added Azure
gentlementlegen Dec 2, 2024
4f3c8ea
chore: azure build flow
gentlementlegen Dec 2, 2024
a838d94
chore: added hono
gentlementlegen Dec 2, 2024
2bfccf4
chore: updated workflow
gentlementlegen Dec 2, 2024
c198930
chore: updated workflow
gentlementlegen Dec 2, 2024
e48f2c5
Add or update the Azure App Service build and deployment workflow config
gentlementlegen Dec 2, 2024
23b7350
chore: updated workflow
gentlementlegen Dec 2, 2024
471cca0
chore: tsup compilation
gentlementlegen Dec 2, 2024
0a1b606
chore: added post route
gentlementlegen Dec 2, 2024
3ca4e9f
chore: fixed hono env
gentlementlegen Dec 2, 2024
9f7901a
chore: added frozen lockfile
gentlementlegen Dec 2, 2024
0f7534f
chore: test diminish package
gentlementlegen Dec 2, 2024
a8e29de
feat: hono
whilefoo Dec 2, 2024
64681b8
fix: knip
whilefoo Dec 2, 2024
b053c86
chore: added Azure
gentlementlegen Dec 2, 2024
8161d7e
chore: added hono
gentlementlegen Dec 2, 2024
1334073
chore: moved entry point
gentlementlegen Dec 2, 2024
5a2dc5a
chore: updated deploy branch
gentlementlegen Dec 2, 2024
3f8adf3
Merge branch 'development' into feat/azure
gentlementlegen Dec 3, 2024
658b19f
chore: added welcome message
gentlementlegen Dec 3, 2024
d8531b5
chore: test generate settings
gentlementlegen Dec 3, 2024
161d7d1
chore: fixed bun deps
gentlementlegen Dec 3, 2024
26130a5
chore: disable gen
gentlementlegen Dec 3, 2024
e3c9776
chore: node 3
gentlementlegen Dec 3, 2024
70b83c3
chore: renamed secrets
gentlementlegen Dec 4, 2024
82b3136
Add or update the Azure App Service build and deployment workflow config
gentlementlegen Dec 4, 2024
51ebe87
chore: renamed secrets
gentlementlegen Dec 4, 2024
2cc2833
chore: renamed secrets
gentlementlegen Dec 4, 2024
1730972
chore: added local settings generation
gentlementlegen Dec 4, 2024
b9ee268
chore: added build step
gentlementlegen Dec 4, 2024
600d9f3
chore: added build step ubuntu
gentlementlegen Dec 4, 2024
705f520
chore: test no env
gentlementlegen Dec 4, 2024
09f2fa3
chore: cli install
gentlementlegen Dec 4, 2024
3cef087
chore: az functions
gentlementlegen Dec 4, 2024
9c9eb25
chore: removed vercel file
gentlementlegen Dec 4, 2024
711a807
chore: fixed cspell
gentlementlegen Dec 4, 2024
0791eb5
chore: fixed knip
gentlementlegen Dec 4, 2024
b73ed34
chore: updated example env
gentlementlegen Dec 4, 2024
72c1a92
chore: fixed formatting
gentlementlegen Dec 4, 2024
6ac73c8
chore: renamed example settings
gentlementlegen Dec 4, 2024
ab4c959
chore: updated deployment workflow
gentlementlegen Dec 5, 2024
fc13b67
chore: updated deployment workflow
gentlementlegen Dec 5, 2024
3da026b
chore: fixed spacing in package.json
gentlementlegen Dec 8, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .cspell.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,9 @@
"pavlovcik",
gentlementlegen marked this conversation as resolved.
Show resolved Hide resolved
"ghijklmnopqrstuvwxyz",
"GHIJKLMNOPQRSTUVWXYZ",
"ubiquityos"
"ubiquityos",
"azurefunc",
"marplex"
],
"dictionaries": ["typescript", "node", "software-terms"],
"import": ["@cspell/dict-typescript/cspell-ext.json", "@cspell/dict-node/cspell-ext.json", "@cspell/dict-software-terms"],
Expand Down
10 changes: 10 additions & 0 deletions .funcignore
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this file

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are we releasing packages for Azure?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's what Azure will pack to serve on the endpoint, files listed here won't be included in the build.

Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
*.js.map
*.ts
.git*
.vscode
__azurite_db*__.json
__blobstorage__
__queuestorage__
local.settings.json
test
tsconfig.json
4 changes: 2 additions & 2 deletions .github/knip.ts
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
import type { KnipConfig } from "knip";

const config: KnipConfig = {
entry: ["src/kernel.ts", "src/adapters/cloudflare-worker.ts", "deploy/setup-kv-namespace.ts"],
entry: ["src/kernel.ts", "src/adapters/cloudflare-worker.ts", "deploy/setup-kv-namespace.ts", "src/index.ts"],
project: ["src/**/*.ts"],
ignore: ["jest.config.ts"],
ignoreBinaries: ["i", "publish"],
ignoreBinaries: ["i"],
ignoreExportsUsedInFile: true,
ignoreDependencies: ["@mswjs/data", "esbuild", "eslint-config-prettier", "eslint-plugin-prettier", "msw", "ts-node"],
};
Expand Down
137 changes: 137 additions & 0 deletions .github/workflows/azure-deploy.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
# Docs for the Azure Web Apps Deploy action: https://github.com/azure/functions-action
# More GitHub Actions for Azure: https://github.com/Azure/actions

name: Build and deploy Node.js project to Azure Function App - ubiquity-os

on:
push:
branches:
- development
workflow_dispatch:

env:
AZURE_FUNCTIONAPP_PACKAGE_PATH: '.' # set this to the path to your web app project, defaults to the repository root
NODE_VERSION: '20.10.0'

jobs:
build:
runs-on: windows-latest
steps:
- name: 'Checkout GitHub Action'
uses: actions/checkout@v4

- name: Setup Node ${{ env.NODE_VERSION }} Environment
uses: actions/setup-node@v4
with:
node-version: ${{ env.NODE_VERSION }}

- uses: oven-sh/setup-bun@v2

- name: 'Resolve Project Dependencies Using Bun'
shell: pwsh
gentlementlegen marked this conversation as resolved.
Show resolved Hide resolved
run: |
pushd './${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}'
bun install --frozen-lockfile
bun run jest:test
bun run build
bun rimraf node_modules
bun install --frozen-lockfile --production
popd

- name: Upload artifact for deployment job
uses: actions/upload-artifact@v4
with:
name: node-app
path: .

update-environment:
runs-on: ubuntu-latest
needs: build

permissions:
id-token: write

steps:
- name: Generate local.settings.json
env:
GITHUB_SECRETS: ${{ toJson(secrets) }}
shell: bash
run: |
# Parse the JSON secrets
secrets=$(echo "$GITHUB_SECRETS" | jq -c '.')

# Create the base settings object
settings=$(jq -n '{
"IsEncrypted": false,
"Values": {}
}')

# Define excluded prefixes
excluded_prefixes=("GITHUB_" "ACTIONS_" "AZUREAPPSERVICE_")

# Iterate through secrets and add non-excluded ones
for secret_name in $(echo "$secrets" | jq -r 'keys[]'); do
# Check if the secret should be included
include=true
for prefix in "${excluded_prefixes[@]}"; do
if [[ "$secret_name" == "$prefix"* ]]; then
include=false
break
fi
done

# Add the secret if it passes the filter
if [ "$include" = true ]; then
secret_value=$(echo "$secrets" | jq -r ".[\"$secret_name\"]")
settings=$(echo "$settings" | jq --arg name "$secret_name" --arg value "$secret_value" '.Values[$name] = $value')
fi
done

# Output the final settings to local.settings.json
echo "$settings" | jq '.' > local.settings.json

- uses: azure/login@v2
with:
client-id: ${{ secrets.AZUREAPPSERVICE_CLIENTID }}
tenant-id: ${{ secrets.AZUREAPPSERVICE_TENANTID }}
subscription-id: ${{ secrets.AZUREAPPSERVICE_SUBSCRIPTIONID }}

- name: Azure CLI script to upload environment
uses: azure/cli@v2
with:
azcliversion: latest
inlineScript: |
az account show
az functionapp config appsettings set \
--name ubiquity-os \
--resource-group ubiquity-os_group \
--settings @local.settings.json

deploy:
runs-on: windows-latest
needs: build
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be run after update-environment? I was recently using Azure App Service and when I changed env I had to restart the app

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think both steps restart the app, so it doesn't matter much. The best way would be to combine the two steps, the command line can handle upload + local.storage.settings upload but the GitHub Action does not support it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GitHub Action does not support it.

That seems unlikely because its a fully capable ubuntu box.


permissions:
id-token: write #This is required for requesting the JWT

steps:
- name: Download artifact from build job
uses: actions/download-artifact@v4
with:
name: node-app

- name: Login to Azure
uses: azure/login@v2
with:
client-id: ${{ secrets.AZUREAPPSERVICE_CLIENTID }}
tenant-id: ${{ secrets.AZUREAPPSERVICE_TENANTID }}
subscription-id: ${{ secrets.AZUREAPPSERVICE_SUBSCRIPTIONID }}

- name: 'Run Azure Functions Action'
uses: Azure/functions-action@v1
id: fa
with:
app-name: 'ubiquity-os'
slot-name: 'Production'
package: ${{ env.AZURE_FUNCTIONAPP_PACKAGE_PATH }}

1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -120,3 +120,4 @@ junit.xml

*.pem
test-dashboard.md
local.settings.json
Binary file modified bun.lockb
Binary file not shown.
20 changes: 20 additions & 0 deletions host.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
{
"version": "2.0",
"logging": {
"applicationInsights": {
"samplingSettings": {
"isEnabled": true,
"excludedTypes": "Request"
}
}
},
"extensionBundle": {
"id": "Microsoft.Azure.Functions.ExtensionBundle",
"version": "[4.*, 5.0.0)"
},
"extensions": {
"http": {
"routePrefix": ""
}
}
}
12 changes: 12 additions & 0 deletions local.settings.json.example
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
{
"IsEncrypted": false,
"Values": {
"AzureWebJobsStorage": "UseDevelopmentStorage=true",
"FUNCTIONS_WORKER_RUNTIME": "node",
"APP_WEBHOOK_SECRET": "xxxxxx",
"APP_ID": "123456",
"APP_PRIVATE_KEY": "-----BEGIN PRIVATE KEY-----",
"ENVIRONMENT": "development | production",
"OPENAI_API_KEY": "123456"
}
}
19 changes: 15 additions & 4 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,7 @@
"version": "2.1.5",
"private": false,
"description": "The kernel for UbiquityOS.",
"module": "dist/index.mjs",
"main": "dist/index.js",
"main": "dist/*.js",
"typings": "dist/index.d.ts",
"files": [
"dist"
Expand All @@ -14,6 +13,7 @@
"engines": {
"node": ">=20.10.0"
},
"type": "module",
"scripts": {
"dev": "run-p worker proxy",
"predev": "tsx predev.ts",
Expand All @@ -32,7 +32,13 @@
"jest:test": "jest --coverage",
"plugin:hello-world": "tsx tests/__mocks__/hello-world-plugin.ts",
"setup-kv": "bun --env-file=.dev.vars scripts/setup-kv-namespace.ts",
"setup": "tsx ./scripts/setup.ts"
"setup": "tsx ./scripts/setup.ts",
"start:azure": "run-p proxy build:watch start",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the formatting correct or is it a render bug in my GitHub iOS client

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the space is a tab most likely, I will fix that.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you think run-s might make more sense, to run in series instead of parallel? I suggest that because build generally should complete before starting.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's true but since we are watching the code to rebuild on save, I think it makes more sense to run in parallel otherwise saving won't reload the code.

"start": "func start --port 8787",
"prestart": "bun run build",
"build": "tsup",
"build:watch": "tsup --watch",
"prebuild": "rimraf dist"
},
"keywords": [
"typescript",
Expand All @@ -42,7 +48,9 @@
"open-source"
],
"dependencies": {
"@azure/functions": "^4.6.0",
"@cfworker/json-schema": "2.0.1",
"@marplex/hono-azurefunc-adapter": "^1.0.1",
"@octokit/auth-app": "7.1.0",
"@octokit/core": "6.1.2",
"@octokit/plugin-paginate-rest": "11.3.3",
Expand All @@ -54,7 +62,7 @@
"@octokit/webhooks": "13.3.0",
"@octokit/webhooks-types": "7.5.1",
"@sinclair/typebox": "0.34.3",
"@ubiquity-os/plugin-sdk": "^1.1.0",
"@ubiquity-os/plugin-sdk": "^1.1.1",
"dotenv": "16.4.5",
"hono": "^4.6.12",
"openai": "^4.70.2",
Expand All @@ -78,6 +86,7 @@
"@types/jest": "29.5.12",
"@types/node": "20.14.10",
"@types/node-rsa": "^1.1.4",
"azure-functions-core-tools": "^4.0.6610",
"cspell": "8.9.0",
"esbuild": "0.23.0",
"eslint": "9.7.0",
Expand All @@ -95,10 +104,12 @@
"open": "^10.1.0",
"ora": "^8.1.1",
"prettier": "3.3.3",
"rimraf": "^6.0.1",
"smee-client": "^2.0.4",
"toml": "3.0.0",
"tomlify-j0.4": "3.0.0",
"ts-node": "^10.9.2",
"tsup": "^8.3.5",
"tsx": "4.16.2",
"typescript": "5.6.3",
"typescript-eslint": "7.16.0",
Expand Down
10 changes: 10 additions & 0 deletions src/functions/http-trigger.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import { app } from "@azure/functions";
import { azureHonoHandler } from "@marplex/hono-azurefunc-adapter";
import { app as honoApp } from "../kernel";

app.http("http-trigger", {
methods: ["GET", "POST"],
authLevel: "anonymous",
route: "{*proxy}",
handler: azureHonoHandler(honoApp.fetch),
});
5 changes: 5 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
import { app } from "@azure/functions";

app.setup({
enableHttpStream: true,
});
5 changes: 5 additions & 0 deletions src/kernel.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,9 +9,14 @@ import OpenAI from "openai";
import { Context, Hono, HonoRequest } from "hono";
import { env as honoEnv, getRuntimeKey } from "hono/adapter";
import { StatusCode } from "hono/utils/http-status";
import packageJson from "../package.json";

export const app = new Hono();

app.get("/", (c) => {
return c.text(`Welcome to UbiquityOS kernel version ${packageJson.version}`);
});

app.post("/", async (ctx: Context) => {
try {
const env = honoEnv(ctx);
Expand Down
13 changes: 7 additions & 6 deletions tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,8 @@
// "disableReferencedProjectLoad": true, /* Reduce the number of projects loaded automatically by TypeScript. */

/* Language and Environment */
"target": "es2021" /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */,
"lib": ["es2021"] /* Specify a set of bundled library declaration files that describe the target runtime environment. */,
"target": "ES2022" /* Set the JavaScript language version for emitted JavaScript and include compatible library declarations. */,
"lib": ["ES2022"] /* Specify a set of bundled library declaration files that describe the target runtime environment. */,
// "jsx": "react" /* Specify what JSX code is generated. */,
// "experimentalDecorators": true, /* Enable experimental support for TC39 stage 2 draft decorators. */
// "emitDecoratorMetadata": true, /* Emit design-type metadata for decorated declarations in source files. */
Expand All @@ -24,9 +24,9 @@
// "useDefineForClassFields": true, /* Emit ECMAScript-standard-compliant class fields. */

/* Modules */
"module": "commonjs" /* Specify what module code is generated. */,
"module": "ES2022" /* Specify what module code is generated. */,
// "rootDir": "./", /* Specify the root folder within your source files. */
"moduleResolution": "node" /* Specify how TypeScript looks up a file from a given module specifier. */,
"moduleResolution": "Bundler" /* Specify how TypeScript looks up a file from a given module specifier. */,
gentlementlegen marked this conversation as resolved.
Show resolved Hide resolved
// "baseUrl": "./", /* Specify the base directory to resolve non-relative module names. */
// "paths": {}, /* Specify a set of entries that re-map imports to additional lookup locations. */
// "rootDirs": [], /* Allow multiple folders to be treated as one when resolving modules. */
Expand All @@ -48,8 +48,9 @@
// "sourceMap": true, /* Create source map files for emitted JavaScript files. */
// "outFile": "./", /* Specify a file that bundles all outputs into one JavaScript file. If `declaration` is true, also designates a file that bundles all .d.ts output. */
"outDir": "./dist" /* Specify an output folder for all emitted files. */,
"rootDir": ".",
// "removeComments": true, /* Disable emitting comments. */
"noEmit": true /* Disable emitting files from a compilation. */,
// "noEmit": true /* Disable emitting files from a compilation. */,
// "importHelpers": true, /* Allow importing helper functions from tslib once per project, instead of including them per-file. */
// "importsNotUsedAsValues": "remove", /* Specify emit/checking behavior for imports that are only used for types */
// "downlevelIteration": true, /* Emit more compliant, but verbose and less performant JavaScript for iteration. */
Expand All @@ -67,7 +68,7 @@
// "preserveValueImports": true, /* Preserve unused imported values in the JavaScript output that would otherwise be removed. */

/* Interop Constraints */
"isolatedModules": true /* Ensure that each file can be safely transpiled without relying on other imports. */,
//"isolatedModules": true /* Ensure that each file can be safely transpiled without relying on other imports. */,
"allowSyntheticDefaultImports": true /* Allow 'import x from y' when a module doesn't have a default export. */,
"esModuleInterop": true /* Emit additional JavaScript to ease support for importing CommonJS modules. This enables `allowSyntheticDefaultImports` for type compatibility. */,
// "preserveSymlinks": true, /* Disable resolving symlinks to their realpath. This correlates to the same flag in node. */
Expand Down
10 changes: 10 additions & 0 deletions tsup.config.ts
gentlementlegen marked this conversation as resolved.
Show resolved Hide resolved
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't you use tsx

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tsx only runs code, it cannot build, as far as I know.

Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import { defineConfig } from "tsup";

export default defineConfig({
entry: ["src/functions/*.ts"],
splitting: false,
sourcemap: false,
clean: true,
format: ["esm"],
minify: true,
});
Loading