Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Azure Key Vault Extension #218

Merged
merged 40 commits into from
Apr 30, 2024
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
40 commits
Select commit Hold shift + click to select a range
8740472
Initialize file structure for keyvault extension.
galiacheng Apr 8, 2024
c38313e
use quarkus cli application for testing.
galiacheng Apr 9, 2024
99cf94e
add internal extension for azure identity
galiacheng Apr 16, 2024
5e2aca5
add dynamic-proxy.
galiacheng Apr 16, 2024
ee81987
add reflectiveHierarchyClass
galiacheng Apr 18, 2024
ccf8b97
complete integration test.
galiacheng Apr 18, 2024
cb93ba0
remove unused dependency.
galiacheng Apr 18, 2024
552ebed
rename test sample.
galiacheng Apr 18, 2024
4e6a06e
procude SecretAsyncClient.
galiacheng Apr 18, 2024
1ebbdbf
add ut.
galiacheng Apr 18, 2024
ea03ef6
complete integration test
galiacheng Apr 22, 2024
8dd8879
create doc for key vault extension.
galiacheng Apr 24, 2024
ea7b823
add ci for key vault extension.
galiacheng Apr 24, 2024
c2f3b72
code clean up
galiacheng Apr 24, 2024
50cb09a
clean up import
galiacheng Apr 24, 2024
1c5b5d0
update integration test.
galiacheng Apr 24, 2024
53b12c7
update top README.
galiacheng Apr 24, 2024
6ada004
Update README of integration-test for keyvault
galiacheng Apr 25, 2024
e993ce7
add KeyVaultSecretResourceIT.
galiacheng Apr 25, 2024
010b51b
fix com.microsoft.aad.msal4j.AbstractClientApplicationBase$Builder no…
galiacheng Apr 25, 2024
bf46fce
update azure-identity denpendency versions.
galiacheng Apr 25, 2024
6d2f6dc
clean up.
galiacheng Apr 25, 2024
690a3d7
build.yaml: specify -DskipTests before az login, as the key vault ext…
galiacheng Apr 25, 2024
9f0c28f
add script to delete keyvault resource.
galiacheng Apr 25, 2024
626f496
use reactive programing in integration test.
galiacheng Apr 26, 2024
b68c60b
remove unused import.
galiacheng Apr 26, 2024
1b0f9b4
apply Jianguo's comment.
galiacheng Apr 29, 2024
5b3010c
apply Zhihao's comments.
galiacheng Apr 29, 2024
a04324d
use -Dskip.azure.test=true to skip tests that require real azure serv…
galiacheng Apr 29, 2024
0db5e3b
use io.quarkus.logging.Log.
galiacheng Apr 30, 2024
b0cfd37
remove unused dependency.
galiacheng Apr 30, 2024
af62cb7
as we disscussed, move otherkeyvault API test to the next PR.
galiacheng Apr 30, 2024
d895ebe
fix build failure
galiacheng Apr 30, 2024
06eb3f8
sort imports used in Key Vault Extension.
galiacheng Apr 30, 2024
5fce42c
apply Ed's comments.
galiacheng Apr 30, 2024
a605c5b
add identitifier to track key vault secret clients.
galiacheng Apr 30, 2024
24cdcdb
format AzureQuarkusIdentifier.
galiacheng Apr 30, 2024
52a3125
format KeyVaultSecretClientProducer.java
galiacheng Apr 30, 2024
37c8dc9
add comment for azure-identity native resources.
galiacheng Apr 30, 2024
02eff46
fix length of application id cannot be greater than 24
galiacheng Apr 30, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 17 additions & 0 deletions .github/build-with-maven-native.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ set -Euo pipefail
# - RESOURCE_GROUP_NAME
# - STORAGE_ACCOUNT_NAME
# - APP_CONFIG_NAME
# - KEY_VAULT_NAME

# Create a resource group
az group create \
Expand Down Expand Up @@ -71,5 +72,21 @@ credential=$(az appconfig credential list \
export QUARKUS_AZURE_APP_CONFIGURATION_ID=$(echo "${credential}" | jq -r '.id')
export QUARKUS_AZURE_APP_CONFIGURATION_SECRET=$(echo "${credential}" | jq -r '.value')

az keyvault create \
--name ${KEY_VAULT_NAME} \
--resource-group ${RESOURCE_GROUP_NAME} \
--location eastus
Comment on lines +80 to +83
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mention this duplicates code in integration-tests/README.md.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See 5fce42c.


az ad signed-in-user show --query id -o tsv \
| az keyvault set-policy \
--name ${KEY_VAULT_NAME} \
--object-id @- \
--secret-permissions all

export QUARKUS_AZURE_KEYVAULT_SECRET_ENDPOINT=$(az keyvault show --name ${KEY_VAULT_NAME} \
--resource-group ${RESOURCE_GROUP_NAME}\
--query properties.vaultUri\
--output tsv)

# Build native executable and run the integration tests against the Azure services
mvn -B install -Dnative -Dquarkus.native.container-build -Dnative.surefire.skip -Dazure.test=true
3 changes: 3 additions & 0 deletions .github/delete-azure-resources.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,11 @@
# - RESOURCE_GROUP_NAME
# - STORAGE_ACCOUNT_NAME
# - APP_CONFIG_NAME
# - KEY_VAULT_NAME
majguo marked this conversation as resolved.
Show resolved Hide resolved

az appconfig delete --name ${APP_CONFIG_NAME} --resource-group ${RESOURCE_GROUP_NAME} --yes
az appconfig purge --name ${APP_CONFIG_NAME} --yes
az storage account delete --name ${STORAGE_ACCOUNT_NAME} --resource-group ${RESOURCE_GROUP_NAME} --yes
az keyvault delete --name ${KEY_VAULT_NAME} --resource-group ${RESOURCE_GROUP_NAME} --yes
az keyvault purge --name ${KEY_VAULT_NAME} --yes
az group delete --name ${RESOURCE_GROUP_NAME} --yes
majguo marked this conversation as resolved.
Show resolved Hide resolved
14 changes: 12 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,7 @@ jobs:
RESOURCE_GROUP_NAME: quarkus${{ github.run_id }}${{ github.run_number }}
STORAGE_ACCOUNT_NAME: storage${{ github.run_id }}${{ github.run_number }}
APP_CONFIG_NAME: appconfig${{ github.run_id }}${{ github.run_number }}
KEY_VAULT_NAME: kv${{ github.run_id }}${{ github.run_number }}
name: Build on ${{ matrix.os }}
strategy:
fail-fast: false
Expand All @@ -67,10 +68,10 @@ jobs:
cache: 'maven'

- name: Build with Maven
run: mvn -B clean install -Dno-format
run: mvn -B clean install -Dno-format -Dskip.azure.tests=true

- name: Build with Maven (Native)
run: mvn -B install -Dnative -Dquarkus.native.container-build -Dnative.surefire.skip
run: mvn -B install -Dnative -Dquarkus.native.container-build -Dnative.surefire.skip -Dskip.azure.tests=true
if: ${{ env.AZURE_CLIENT_ID == '' || env.AZURE_TENANT_ID == '' || env.AZURE_SUBSCRIPTION_ID == '' }}

- name: Az CLI login
Expand All @@ -85,6 +86,15 @@ jobs:
run: .github/build-with-maven-native.sh
if: ${{ env.AZURE_CLIENT_ID != '' && env.AZURE_TENANT_ID != '' && env.AZURE_SUBSCRIPTION_ID != '' }}

# Fix error: Client assertion is not within its valid time range.
- name: Az CLI login
uses: azure/login@v1
with:
client-id: ${{ env.AZURE_CLIENT_ID }}
tenant-id: ${{ env.AZURE_TENANT_ID }}
subscription-id: ${{ env.AZURE_SUBSCRIPTION_ID }}
if: ${{ env.AZURE_CLIENT_ID != '' && env.AZURE_TENANT_ID != '' && env.AZURE_SUBSCRIPTION_ID != '' }}
majguo marked this conversation as resolved.
Show resolved Hide resolved

- name: Delete Azure resources
run: .github/delete-azure-resources.sh
if: ${{ always() && env.AZURE_CLIENT_ID != '' && env.AZURE_TENANT_ID != '' && env.AZURE_SUBSCRIPTION_ID != '' }}
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/quarkus-snapshot.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ jobs:
RESOURCE_GROUP_NAME: quarkus${{ github.run_id }}${{ github.run_number }}
STORAGE_ACCOUNT_NAME: storage${{ github.run_id }}${{ github.run_number }}
APP_CONFIG_NAME: appconfig${{ github.run_id }}${{ github.run_number }}
KEY_VAULT_NAME: kv${{ github.run_id }}${{ github.run_number }}
name: "Build against latest Quarkus snapshot"
runs-on: ubuntu-latest
# Allow <ADMIN> to manually launch the ecosystem CI in addition to the bots
Expand Down
3 changes: 3 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ The following extensions allows you to interact with some of the Azure Services:

- [Quarkus Azure App Configuration Extension](https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-app-configuration.html): [Azure App Configuration](https://azure.microsoft.com/products/app-configuration)
is a fast, scalable parameter storage for app configuration.
- [Quarkus Azure Key Vault Extension](https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-keyvault.html): [Azure Key Vault](https://azure.microsoft.com/products/key-vault) is a cloud service for securely storing and accessing secrets.
- [Quarkus Azure Blob Storage Extension](https://docs.quarkiverse.io/quarkus-azure-services/dev/quarkus-azure-storage-blob.html): [Azure Blob Storage](https://azure.microsoft.com/products/storage/blobs/)
is a massively scalable and secure object storage for cloud-native workloads, archives, data lakes, high-performance
computing, and machine learning.
Expand All @@ -26,6 +27,8 @@ Example applications can be found inside the [integration-tests](integration-tes

- [Azure App Configuration sample](integration-tests/azure-app-configuration): REST endpoint using the Quarkus extension
to get the configuration stored in Azure App Configuration.
- [Azure Key Vault sample](integration-tests/azure-keyvault): REST endpoint using the Quarkus extension
to create secret via [SecretClient](https://learn.microsoft.com/java/api/com.azure.security.keyvault.secrets.secretclient) and [SecretAsyncClient](https://learn.microsoft.com/java/api/com.azure.security.keyvault.secrets.secretasyncclient) in Azure Key Vault.
- [Azure Blob Storage sample](integration-tests/azure-storage-blob): REST endpoint using the Quarkus extension to
upload and download files to/from Azure Blob Storage.

Expand Down
31 changes: 29 additions & 2 deletions bom/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,9 @@
<azure-sdk-bom.version>1.2.22</azure-sdk-bom.version>
<azure.core.http.client.vertx.version>1.0.0-beta.17</azure.core.http.client.vertx.version>
<!-- @sync com.azure:azure-identity:${azure-identity.version} dep:com.microsoft.azure:msal4j -->
<msal4j.version>1.15.0</msal4j.version>
<msal4j.version>1.14.3</msal4j.version>
majguo marked this conversation as resolved.
Show resolved Hide resolved
<!-- @sync com.azure:azure-identity:${azure-identity.version} dep:net.minidev:json-smart -->
<json-smart.verion>2.5.0</json-smart.verion>
<!-- needed for dependency convergence until Azure SDK is upgraded -->
<woodstox-core.version>6.6.2</woodstox-core.version>
</properties>
Expand All @@ -40,7 +42,12 @@
<type>pom</type>
<scope>import</scope>
</dependency>


<dependency>
<groupId>net.minidev</groupId>
<artifactId>json-smart</artifactId>
<version>${json-smart.verion}</version>
</dependency>
<dependency>
<groupId>com.microsoft.azure</groupId>
<artifactId>msal4j</artifactId>
Expand All @@ -67,6 +74,16 @@
</dependency>

<!-- Azure Services Extensions -->
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-identity</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-identity-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-core</artifactId>
Expand Down Expand Up @@ -107,6 +124,16 @@
<artifactId>quarkus-azure-jackson-dataformat-xml-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-keyvault</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-keyvault-deployment</artifactId>
<version>${project.version}</version>
</dependency>
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-storage-blob</artifactId>
Expand Down
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,2 +1,3 @@
* xref:quarkus-azure-app-configuration.adoc[Quarkus Azure App Configuration Extension]
* xref:quarkus-azure-keyvault.adoc[Quarkus Azure Key Vault Extension]
* xref:quarkus-azure-storage-blob.adoc[Quarkus Azure Blob Storage Extension]
29 changes: 29 additions & 0 deletions docs/modules/ROOT/pages/includes/quarkus-azure-keyvault.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@

:summaryTableId: quarkus-azure-keyvault
[.configuration-legend]
icon:lock[title=Fixed at build time] Configuration property fixed at build time - All other configuration properties are overridable at runtime
[.configuration-reference.searchable, cols="80,.^10,.^10"]
|===

h|[[quarkus-azure-keyvault_configuration]]link:#quarkus-azure-keyvault_configuration[Configuration property]

h|Type
h|Default

a| [[quarkus-azure-keyvault_quarkus.quarkus.azure.keyvault.secret.endpoint]]`link:#quarkus-azure-keyvualt_quarkus.azure.keyvault.secret.endpoint[quarkus.azure.keyvault.secret.endpoint]`


[.description]
--
The endpoint of Azure Key Vault.

ifdef::add-copy-button-to-env-var[]
Environment variable: env_var_with_copy_button:+++QUARKUS_AZURE_KEYVAULT_SECRET_ENDPOINT+++[]
endif::add-copy-button-to-env-var[]
ifndef::add-copy-button-to-env-var[]
Environment variable: `+++QUARKUS_AZURE_KEYVAULT_SECRET_ENDPOINT+++`
endif::add-copy-button-to-env-var[]
--|string
|required icon:exclamation-circle[title=Configuration property is required]

|===
2 changes: 2 additions & 0 deletions docs/modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,6 @@ include::./includes/attributes.adoc[]

include::quarkus-azure-app-configuration.adoc[leveloffset=+1, opts=optional]

include::quarkus-azure-keyvault.adoc[leveloffset=+1, opts=optional]

include::quarkus-azure-storage-blob.adoc[leveloffset=+1, opts=optional]
167 changes: 167 additions & 0 deletions docs/modules/ROOT/pages/quarkus-azure-keyvault.adoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,167 @@
= Quarkus Azure Key Vault Extension

include::./includes/attributes.adoc[]

include::./includes/support.adoc[]

https://azure.microsoft.com/en-us/products/key-vault[Azure Key Vault] is a cloud service for securely storing and accessing secrets. A secret is anything that you want to tightly control access to, such as API keys, passwords, certificates, or cryptographic keys.
This extension allows you to create and retrieve secret from Azure Key Vault by injecting the following object inside your Quarkus application.

* `com.azure.security.keyvault.secrets.SecretClient`
* `com.azure.security.keyvault.secrets.SecretAsyncClient`

The extension produces SecretClient and SecretAsyncClient using https://learn.microsoft.com/java/api/overview/azure/identity-readme#defaultazurecredential[DefaultAzureCredential].
Developers who want more control or whose scenario isn't served by the default settings should build client using other credential types.

== Installation

If you want to use this extension, you need to add the `io.quarkiverse.azureservices:quarkus-azure-services` extension first to your build file.

For instance, with Maven, add the following dependency to your POM file:

[source,xml,subs=attributes+]
----
<dependency>
<groupId>io.quarkiverse.azureservices</groupId>
<artifactId>quarkus-azure-keyvault</artifactId>
<version>{project-version}</version>
</dependency>
----

== How to Use It

Once you have added the extension to your project, follow the next steps, so you can inject `com.azure.security.keyvault.secrets.SecretClient` or `com.azure.security.keyvault.secrets.SecretAsyncClient` object in your application to manage secret.

=== Setup your Azure Environment

include::includes/azure-setup-environment.adoc[]

Create an Azure resource group with the `az group create` command.
A resource group is a logical container into which Azure resources are deployed and managed.

[source,shell]
----
az group create \
--name rg-quarkus-azure-keyvault \
--location eastus
----

Create a general-purpose key vault with the following command:

[source,shell]
----
az keyvault create --name kvquarkusazurekv0423 \
--resource-group rg-quarkus-azure-keyvault \
--location eastus
----

Key Vault provides secure storage of generic secrets, such as passwords and database connection strings.
All secrets in your key vault are stored encrypted. The Azure Key Vault service encrypts your secrets when you add them, and decrypts them automatically when you read them.

Access Control for secrets managed in Key Vault, is provided at the level of the Key Vault.
The following command uses your Azure AD account to authorize the operation to manage secret.
Even if you are the key vault owner, you need explicit permissions to perform operations against secret.

Assign all secret permissions(backup, delete, get, list, purge, recover, restore, set) to yourself:

[source,shell]
----
az ad signed-in-user show --query id -o tsv \
| az keyvault set-policy \
--name kvquarkusazurekv0423 \
--object-id @- \
--secret-permissions all
----

If you log into the http://portal.azure.com/[Azure portal], you can see the key vault you created. Select **Objects** -> **Secrets**, you will find the Secrets page.

image::quarkus-azure-keyvault-secret-portal.png[alt=Azure Portal showing Key Vault Secrets]

=== Configure the Azure Key Vault Secret Client

As you can see below in the _Configuration Reference_ section, the configuration option `quarkus.azure.keyvault.secret.endpoint` is mandatory.
To get the endpoint, execute the following Azure CLI command:

[source,shell]
----
az keyvault show --name kvquarkusazurekv0423 \
--resource-group rg-quarkus-azure-keyvault \
--query properties.vaultUri \
--output tsv
----

Then, in the `application.properties` file, add the following property:

[source,properties]
----
quarkus.azure.keyvault.secret.endpoint=https://kvquarkusazurekv0423.vault.azure.net/
----

=== Inject the Azure Key Vault Secret Client

Now that your Azure environment is ready and that you have configured the extension, you can inject the `com.azure.security.keyvault.secrets.SecretClient` object in your application, so you can interact with Azure Key Vault Secret.

This is a https://quarkus.io/guides/command-mode-reference[Quarkus CLI application]. The application will:

* Ask for a secret value.
* Create a secret with name `mySecret` and set its value.
* Retrieve and print the secret value.
* Delete the secret.

You can build and run the application in development mode using command:

```
quarkus dev
```

[source,java]
----
@QuarkusMain
public class KeyVaultSecretApplication implements QuarkusApplication {

@Inject
SecretClient secretClient;

@Override
public int run(String... args) throws Exception {

Console con = System.console();

String secretName = "mySecret";
System.out.println("Create secret: " + secretName);

System.out.println("Please provide the value of your secret > ");

String secretValue = con.readLine();

System.out.println("Creating a secret called '" + secretName + "' with value '" + secretValue + "' ... ");

secretClient.setSecret(new KeyVaultSecret(secretName, secretValue));

System.out.println("Retrieving your secret...");

KeyVaultSecret retrievedSecret = secretClient.getSecret(secretName);

System.out.println("Your secret's value is '" + retrievedSecret.getValue() + "'.");
System.out.println("Deleting your secret ... ");

SyncPoller<DeletedSecret, Void> deletionPoller = secretClient.beginDeleteSecret(secretName);
deletionPoller.waitForCompletion();

System.out.println("done.");
return 0;
}
}

----

After running the application, if you log into the http://portal.azure.com/[Azure portal], you can see the key vault and the secret you created.
As the secret is deleted, you will find the secret from **Objects** -> **Secrets** -> **Manage deleted secrets**.

image::quarkus-azure-keyvault-deleted-secret-portal.png[alt=Azure Portal showing the deleted secrets]

You can also inject `com.azure.security.keyvault.secrets.SecretAsyncClient` object to your application. For more usage, see https://learn.microsoft.com/java/api/com.azure.security.keyvault.secrets.secretasyncclient?view=azure-java-stable[com.azure.security.keyvault.secrets.secretasyncclient].

== Extension Configuration Reference

include::includes/quarkus-azure-keyvault.adoc[leveloffset=+1, opts=optional]
Loading
Loading