feat: azure deployment #217

gentlementlegen · 2024-12-02T21:49:29Z

Relates to #195
Resolves ubiquity/.github#111

List of changes

This introduces a pipeline and setup to deploy the Kernel on Azure. The testing endpoint is at https://ubiquity-os.azurewebsites.net/
This can be a good backup if we see that Cloudflare fails again (it is back to stable at this time somehow). The two deployments and instances can coexist, and Azure is pay as you go so if we don't use it it doesn't matter much.

New features

deploy to Azure
deploy the stored secrets in Action Env to Azure Env
local setup and scripts

What should be done

have working logs (nothing shows in the console)
have dev / prod setup (and eventually branches)
have an account for Ubiquity Org
properly override env variables if set via the script

QA

Meniole/text-conversation-rewards#31 (comment)

.cspell.json

0x4007 · 2024-12-03T04:09:15Z

src/adapters/cloudflare-worker.ts

@@ -0,0 +1,3 @@
+import { app } from "../kernel";


Suggested change

import { app } from "../kernel";

export { app } from "../kernel";

Can't you just do that

tsup.config.ts

# Conflicts: # bun.lockb # src/proxy.ts # tests/dispatch.test.ts # tests/main.test.ts

gentlementlegen · 2024-12-04T07:31:27Z

@whilefoo rfc

0x4007 · 2024-12-04T08:18:26Z

The two deployments and instances can coexist

This would be really interesting if we had load balancing, although implementation would be tricky if we have to manually check for output from every plugin before posting a new comment or taking an action.

I assume you mean we would have both deployments live but we would have to manually switch which responds to events from the GitHub App?

gentlementlegen · 2024-12-04T08:43:45Z

Yes we should change the webhook url within the GitHub App so it targets one or the other.

whilefoo · 2024-12-04T14:17:31Z

.github/workflows/azure-deploy.yml

+
+  deploy:
+    runs-on: windows-latest
+    needs: build


shouldn't this be run after update-environment? I was recently using Azure App Service and when I changed env I had to restart the app

I think both steps restart the app, so it doesn't matter much. The best way would be to combine the two steps, the command line can handle upload + local.storage.settings upload but the GitHub Action does not support it.

GitHub Action does not support it.

That seems unlikely because its a fully capable ubuntu box.

.github/workflows/azure-deploy.yml

rndquu · 2024-12-04T15:58:18Z

@gentlementlegen On bun install && bun dev getting this output:

✘ [ERROR] Build failed with 3 errors:

  ✘ [ERROR] Could not resolve "openai"
  
      src/kernel.ts:8:19:
        8 │ import OpenAI from "openai";
          ╵                    ~~~~~~~~
  
    The Yarn Plug'n'Play manifest forbids importing "openai" here because it's not listed as a
  dependency of this package:
  
      .pnp.cjs:4514:31:
        4514 │         "packageDependencies": [\
             ╵                                ~~
  
    You can mark the path "openai" as external to exclude it from the bundle, which will remove this
  error.
  
  
  ✘ [ERROR] Could not resolve "hono"
  
      src/kernel.ts:9:43:
        9 │ import { Context, Hono, HonoRequest } from "hono";
          ╵                                            ~~~~~~
  
    The Yarn Plug'n'Play manifest forbids importing "hono" here because it's not listed as a
  dependency of this package:
  
      .pnp.cjs:4514:31:
        4514 │         "packageDependencies": [\
             ╵                                ~~
  
    You can mark the path "hono" as external to exclude it from the bundle, which will remove this
  error.
  
  
  ✘ [ERROR] Could not resolve "hono/adapter"
  
      src/kernel.ts:10:46:
        10 │ import { env as honoEnv, getRuntimeKey } from "hono/adapter";
           ╵                                               ~~~~~~~~~~~~~~
  
    The Yarn Plug'n'Play manifest forbids importing "hono" here because it's not listed as a
  dependency of this package:
  
      .pnp.cjs:4514:31:
        4514 │         "packageDependencies": [\
             ╵                                ~~
  
    You can mark the path "hono/adapter" as external to exclude it from the bundle, which will
  remove this error.

Do you get the same errors?

0x4007 · 2024-12-04T16:36:31Z

Anything yarn related should 1. be removed from the project and 2. if there are still problems then it must be your environment. Everything should be using bun here.

.github/workflows/azure-deploy.yml

tsconfig.json

gentlementlegen · 2024-12-05T04:19:09Z

@gentlementlegen On bun install && bun dev getting this output:

✘ [ERROR] Build failed with 3 errors:

  ✘ [ERROR] Could not resolve "openai"
  
      src/kernel.ts:8:19:
        8 │ import OpenAI from "openai";
          ╵                    ~~~~~~~~
  
    The Yarn Plug'n'Play manifest forbids importing "openai" here because it's not listed as a
  dependency of this package:
  
      .pnp.cjs:4514:31:
        4514 │         "packageDependencies": [\
             ╵                                ~~
  
    You can mark the path "openai" as external to exclude it from the bundle, which will remove this
  error.
  
  
  ✘ [ERROR] Could not resolve "hono"
  
      src/kernel.ts:9:43:
        9 │ import { Context, Hono, HonoRequest } from "hono";
          ╵                                            ~~~~~~
  
    The Yarn Plug'n'Play manifest forbids importing "hono" here because it's not listed as a
  dependency of this package:
  
      .pnp.cjs:4514:31:
        4514 │         "packageDependencies": [\
             ╵                                ~~
  
    You can mark the path "hono" as external to exclude it from the bundle, which will remove this
  error.
  
  
  ✘ [ERROR] Could not resolve "hono/adapter"
  
      src/kernel.ts:10:46:
        10 │ import { env as honoEnv, getRuntimeKey } from "hono/adapter";
           ╵                                               ~~~~~~~~~~~~~~
  
    The Yarn Plug'n'Play manifest forbids importing "hono" here because it's not listed as a
  dependency of this package:
  
      .pnp.cjs:4514:31:
        4514 │         "packageDependencies": [\
             ╵                                ~~
  
    You can mark the path "hono/adapter" as external to exclude it from the bundle, which will
  remove this error.

Do you get the same errors?

I tested deleting all the node modules and the lockb, bun dev and bun start:azure are working fine. What version of bun are you using?

rndquu · 2024-12-05T08:10:31Z

I tested deleting all the node modules and the lockb, bun dev and bun start:azure are working fine. What version of bun are you using?

This was smth my local env related, after removing the node_modules it all started working

gentlementlegen · 2024-12-05T08:13:05Z

Glad it works. Once merge, I will have to set all the secrets for deployment in the repo as well.

0x4007 · 2024-12-06T17:15:49Z

tsup.config.ts

Why don't you use tsx

tsx only runs code, it cannot build, as far as I know.

0x4007 · 2024-12-06T17:16:46Z

package.json

@@ -32,7 +32,13 @@
    "jest:test": "jest --coverage",
    "plugin:hello-world": "tsx tests/__mocks__/hello-world-plugin.ts",
    "setup-kv": "bun --env-file=.dev.vars scripts/setup-kv-namespace.ts",
-    "setup": "tsx ./scripts/setup.ts"
+    "setup": "tsx ./scripts/setup.ts",
+		"start:azure": "run-p proxy build:watch start",


is the formatting correct or is it a render bug in my GitHub iOS client

I think the space is a tab most likely, I will fix that.

Do you think run-s might make more sense, to run in series instead of parallel? I suggest that because build generally should complete before starting.

It's true but since we are watching the code to rebuild on save, I think it makes more sense to run in parallel otherwise saving won't reload the code.

0x4007 · 2024-12-06T17:17:26Z

.funcignore

What is this file

It's the ignore file for Azure: https://learn.microsoft.com/en-us/azure/developer/azure-developer-cli/service-packaging-ignore-files?tabs=nodejs

Why are we releasing packages for Azure?

It's what Azure will pack to serve on the endpoint, files listed here won't be included in the build.

gentlementlegen and others added 20 commits December 2, 2024 18:21

chore: vercel

5c1fae4

Add or update the Azure App Service build and deployment workflow config

e41fbb5

chore: added Azure

61c8985

chore: azure build flow

4f3c8ea

chore: added hono

a838d94

chore: updated workflow

2bfccf4

chore: updated workflow

c198930

Add or update the Azure App Service build and deployment workflow config

e48f2c5

chore: updated workflow

23b7350

chore: tsup compilation

471cca0

chore: added post route

0a1b606

chore: fixed hono env

3ca4e9f

chore: added frozen lockfile

9f7901a

chore: test diminish package

0f7534f

feat: hono

a8e29de

fix: knip

64681b8

chore: added Azure

b053c86

chore: added hono

8161d7e

chore: moved entry point

1334073

chore: updated deploy branch

5a2dc5a

0x4007 reviewed Dec 3, 2024

View reviewed changes

rndquu mentioned this pull request Dec 3, 2024

feat: hono #215

Merged

gentlementlegen and others added 8 commits December 3, 2024 22:50

Merge branch 'development' into feat/azure

3f8adf3

# Conflicts: # bun.lockb # src/proxy.ts # tests/dispatch.test.ts # tests/main.test.ts

chore: added welcome message

658b19f

chore: test generate settings

d8531b5

chore: fixed bun deps

161d7d1

chore: disable gen

26130a5

chore: node 3

e3c9776

chore: renamed secrets

70b83c3

Add or update the Azure App Service build and deployment workflow config

82b3136

gentlementlegen marked this pull request as ready for review December 4, 2024 07:17

gentlementlegen added 2 commits December 4, 2024 16:26

chore: fixed formatting

72c1a92

chore: renamed example settings

6ac73c8

gentlementlegen changed the title ~~Feat/azure~~ feat: azure deployment Dec 4, 2024

gentlementlegen requested review from 0x4007 and rndquu December 4, 2024 07:31

whilefoo reviewed Dec 4, 2024

View reviewed changes

rndquu requested review from rndquu and removed request for rndquu December 4, 2024 15:47

0x4007 reviewed Dec 4, 2024

View reviewed changes

.github/workflows/azure-deploy.yml Outdated Show resolved Hide resolved

0x4007 reviewed Dec 4, 2024

View reviewed changes

.github/workflows/azure-deploy.yml Outdated Show resolved Hide resolved

0x4007 reviewed Dec 4, 2024

View reviewed changes

.github/workflows/azure-deploy.yml Show resolved Hide resolved

0x4007 reviewed Dec 4, 2024

View reviewed changes

tsconfig.json Show resolved Hide resolved

gentlementlegen added 2 commits December 5, 2024 13:11

chore: updated deployment workflow

ab4c959

chore: updated deployment workflow

fc13b67

rndquu approved these changes Dec 5, 2024

View reviewed changes

whilefoo approved these changes Dec 5, 2024

View reviewed changes

0x4007 approved these changes Dec 6, 2024

View reviewed changes

chore: fixed spacing in package.json

3da026b

gentlementlegen merged commit bb72c0a into ubiquity-os:development Dec 8, 2024
3 checks passed

gentlementlegen deleted the feat/azure branch December 8, 2024 20:17

ubiquity-os bot mentioned this pull request Dec 8, 2024

Cloudflare as a Risk ubiquity/.github#111

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: azure deployment #217

feat: azure deployment #217

gentlementlegen commented Dec 2, 2024 •

edited

Loading

0x4007 Dec 3, 2024

gentlementlegen commented Dec 4, 2024

0x4007 commented Dec 4, 2024

gentlementlegen commented Dec 4, 2024

whilefoo Dec 4, 2024

gentlementlegen Dec 5, 2024

0x4007 Dec 10, 2024

rndquu commented Dec 4, 2024 •

edited

Loading

0x4007 commented Dec 4, 2024

gentlementlegen commented Dec 5, 2024

rndquu commented Dec 5, 2024

gentlementlegen commented Dec 5, 2024

0x4007 Dec 6, 2024

gentlementlegen Dec 11, 2024

0x4007 Dec 6, 2024

gentlementlegen Dec 8, 2024

0x4007 Dec 10, 2024

gentlementlegen Dec 11, 2024

0x4007 Dec 6, 2024

gentlementlegen Dec 8, 2024

0x4007 Dec 10, 2024

gentlementlegen Dec 11, 2024

	import { app } from "../kernel";
	export { app } from "../kernel";

feat: azure deployment #217

feat: azure deployment #217

Conversation

gentlementlegen commented Dec 2, 2024 • edited Loading

List of changes

New features

What should be done

QA

Choose a reason for hiding this comment

gentlementlegen commented Dec 4, 2024

0x4007 commented Dec 4, 2024

gentlementlegen commented Dec 4, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rndquu commented Dec 4, 2024 • edited Loading

0x4007 commented Dec 4, 2024

gentlementlegen commented Dec 5, 2024

rndquu commented Dec 5, 2024

gentlementlegen commented Dec 5, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

gentlementlegen commented Dec 2, 2024 •

edited

Loading

rndquu commented Dec 4, 2024 •

edited

Loading