Release 0.6.0

### Features * Add --no-precommit flag to build_and_test_all_in_docker * Add command to generate snapshots to data cli * Add support for reading snapshots during server startup. * Implement a snapshot writer. * Produce PCR0.json for server EIF * Remove VCS commit info from server --buildinfo * Reorg build_and_test* scripts * Store and validate arch-specific PCR0 hash * update dev to staging copybara to include github workflows * update GitHub presubmit workflow to trigger on pull request * Upgrade to build-system 0.10.0 * Upgrade to build-system 0.5.0 * Upgrade to build-system 0.6.0 * Upgrade to gRPC v1.51.1 ### Bug Fixes * add missing "xray" to vpc_interface_endpoint_services references. * Adjust git global config * Attach initial_launch_hook to autoscaling group. * Avoid non-zero exit on PCR0 hash mismatch * Correct documentation on endpoint to test * Fix the region doc for local development. * Ignore builders/ when executing pre-commit * LifecycleHeartbeat only Finish once. Fixed unit test. * Upgrade to addlicense v1.1 * Use absolute path for kokoro_release.sh * Use bazel-debian to build and run test_serving_data_generator ### Build System * Add presubmit GitHub workflow * Upgrade to bazel 5.4.0 ### Documentation * Add a default AWS region to push command * Correct command to run server locally * Update ECR format and improve the AWS doc order Change-Id: Iff6ea5fe3b60a676daa4ea59b6002556ba9044ac Bug: 264395137 GitOrigin-RevId: 6f226eea6142202b60c2634dc85633360b243574
privacysandbox · Jan 11, 2023 · c956398 · c956398
1 parent aff371e
commit c956398
Show file tree

Hide file tree

Showing 209 changed files with 5,043 additions and 734 deletions.
diff --git a/.bazelrc b/.bazelrc
@@ -10,3 +10,4 @@ build --workspace_status_command="bash tools/get_workspace_status"
 test --test_output=errors
 # Disable ICU linking for googleurl.
 build --@com_google_googleurl//build_config:system_icu=0
+build --@io_opentelemetry_cpp//api:with_abseil=true
diff --git a/.bazelversion b/.bazelversion
diff --git a/.bazelversion b/.bazelversion
@@ -0,0 +1 @@
+5.4.0
diff --git a/.github/workflows/ci_presubmit.yaml b/.github/workflows/ci_presubmit.yaml
@@ -0,0 +1,37 @@
+# Copyright 2022 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+name: 'CI: Presubmit'
+
+on:
+  # Allows you to run this workflow manually from the Actions tab
+  workflow_dispatch:
+
+  pull_request:
+    branches:
+    - main
+    - 'release-*'
+    - 'prerelease-*'
+
+jobs:
+  build-debian:
+    name: Build debian-based build image
+    runs-on: ubuntu-latest
+    steps:
+    - name: Check out code
+      uses: actions/checkout@v3
+
+    - name: Build the K/V server
+      run: |
+       "${GITHUB_WORKSPACE}"/production/packaging/build_and_test_all_in_docker
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
@@ -18,13 +18,14 @@ exclude: (?x)^(
              bazel-(bin|out|testlogs|workspace)/.*|
              google_internal/coordinator_environments/.*$|
              .bazel_output/.*|
+             builders/.*|
              version.txt
          )$
 
 fail_fast: true
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v4.3.0
+  rev: v4.4.0
   hooks:
   - id: end-of-file-fixer
   - id: fix-byte-order-marker
@@ -65,7 +66,7 @@ repos:
     name: addlicense
     language: golang
     additional_dependencies:
-    - github.com/google/addlicense@v1.0.0
+    - github.com/google/addlicense@v1.1.0
     always_run: false
     pass_filenames: true
     entry: addlicense -v
@@ -76,7 +77,7 @@ repos:
     name: addlicense check
     language: golang
     additional_dependencies:
-    - github.com/google/addlicense@v1.0.0
+    - github.com/google/addlicense@v1.1.0
     always_run: false
     pass_filenames: true
     entry: addlicense -check
@@ -101,7 +102,7 @@ repos:
     - markdown
 
 - repo: https://github.com/DavidAnson/markdownlint-cli2
-  rev: v0.5.1
+  rev: v0.6.0
   hooks:
   - id: markdownlint-cli2
     name: lint markdown
@@ -136,7 +137,7 @@ repos:
     - --quiet
 
 - repo: https://github.com/psf/black
-  rev: 22.8.0
+  rev: 22.12.0
   hooks:
   - id: black
     name: black python formatter
diff --git a/BUILD b/BUILD
@@ -1,3 +1,17 @@
+# Copyright 2023 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#      http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
 load("@bazel_skylib//rules:common_settings.bzl", "string_flag")
 
 package(default_visibility = ["//:__subpackages__"])
@@ -52,7 +66,7 @@ config_setting(
     flag_values = {
         ":instance": "local",
     },
-    visibility = ["//visibility:private"],
+    visibility = ["//components/util:__pkg__"],
 )
 
 exports_files(

diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,54 @@
 
 All notable changes to this project will be documented in this file. See [commit-and-tag-version](https://github.com/absolute-version/commit-and-tag-version) for commit guidelines.
 
+## [0.6.0](https://github.com/privacysandbox/fledge-key-value-service/compare/release-0.5.0...release-0.6.0) (2023-01-10)
+
+
+### Features
+
+* Add --no-precommit flag to build_and_test_all_in_docker
+* Add command to generate snapshots to data cli
+* Add support for reading snapshots during server startup.
+* Implement a snapshot writer.
+* Produce PCR0.json for server EIF
+* Remove VCS commit info from server --buildinfo
+* Reorg build_and_test* scripts
+* Store and validate arch-specific PCR0 hash
+* update dev to staging copybara to include github workflows
+* update GitHub presubmit workflow to trigger on pull request
+* Upgrade to build-system 0.10.0
+* Upgrade to build-system 0.5.0
+* Upgrade to build-system 0.6.0
+* Upgrade to gRPC v1.51.1
+
+
+### Bug Fixes
+
+* add missing "xray" to vpc_interface_endpoint_services references.
+* Adjust git global config
+* Attach initial_launch_hook to autoscaling group.
+* Avoid non-zero exit on PCR0 hash mismatch
+* Correct documentation on endpoint to test
+* Fix the region doc for local development.
+* Ignore builders/ when executing pre-commit
+* LifecycleHeartbeat only Finish once.  Fixed unit test.
+* Upgrade to addlicense v1.1
+* Use absolute path for kokoro_release.sh
+* Use bazel-debian to build and run test_serving_data_generator
+
+
+### Build System
+
+* Add presubmit GitHub workflow
+* Upgrade to bazel 5.4.0
+
+
+### Documentation
+
+* Add a default AWS region to push command
+* Correct command to run server locally
+* Update ECR format and improve the AWS doc order
+
 ## [0.5.0](https://github.com/privacysandbox/fledge-key-value-service/compare/release-0.4.0...release-0.5.0) (2022-11-28)
 
 

diff --git a/WORKSPACE b/WORKSPACE
@@ -43,12 +43,30 @@ http_archive(
 )
 
 load("@bazel_gazelle//:deps.bzl", "gazelle_dependencies")
-load("@io_bazel_rules_go//go:deps.bzl", "go_register_toolchains", "go_rules_dependencies")
+load("@io_bazel_rules_go//go:deps.bzl", "go_rules_dependencies")
 
 go_rules_dependencies()
 
-go_register_toolchains(go_version = "1.18")
+### go_register_toolchains will be called by grpc_extra_deps
+# go_register_toolchains(go_version = "1.18")
 
+### gRPC
+http_archive(
+    name = "com_github_grpc_grpc",
+    sha256 = "b55696fb249669744de3e71acc54a9382bea0dce7cd5ba379b356b12b82d4229",
+    strip_prefix = "grpc-1.51.1",
+    urls = ["https://github.com/grpc/grpc/archive/refs/tags/v1.51.1.tar.gz"],
+)
+
+load("@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
+
+grpc_deps()
+
+load("@com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")
+
+grpc_extra_deps()
+
+### gazelle deps must be loaded after go toolchains registered
 gazelle_dependencies()
 
 http_archive(
@@ -64,22 +82,6 @@ load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")
 
 rules_pkg_dependencies()
 
-### gRPC
-http_archive(
-    name = "com_github_grpc_grpc",
-    sha256 = "ec19657a677d49af59aa806ec299c070c882986c9fcc022b1c22c2a3caf01bcd",
-    strip_prefix = "grpc-1.45.0",
-    urls = ["https://github.com/grpc/grpc/archive/refs/tags/v1.45.0.tar.gz"],
-)
-
-load("@com_github_grpc_grpc//bazel:grpc_deps.bzl", "grpc_deps")
-
-grpc_deps()
-
-load("@com_github_grpc_grpc//bazel:grpc_extra_deps.bzl", "grpc_extra_deps")
-
-grpc_extra_deps()
-
 ### googletest
 http_archive(
     name = "com_google_googletest",
@@ -133,3 +135,16 @@ quiche_dependencies()
 load("@com_google_protobuf//:protobuf_deps.bzl", "protobuf_deps")
 
 protobuf_deps()
+
+# Load OpenTelemetry dependencies after load.
+load("//third_party:open_telemetry.bzl", "open_telemetry_dependencies")
+
+open_telemetry_dependencies()
+
+load("@io_opentelemetry_cpp//bazel:repository.bzl", "opentelemetry_cpp_deps")
+
+opentelemetry_cpp_deps()
+
+load("@rules_foreign_cc//foreign_cc:repositories.bzl", "rules_foreign_cc_dependencies")
+
+rules_foreign_cc_dependencies()
diff --git a/builders/.pre-commit-config.yaml b/builders/.pre-commit-config.yaml
@@ -15,15 +15,13 @@
 # See https://pre-commit.com for more information
 # See https://pre-commit.com/hooks.html for more hooks
 exclude: (?x)^(
-             bazel-(bin|out|testlogs|workspace)/.*|
-             .bazel_output/.*|
              version.txt
          )$
 
 fail_fast: true
 repos:
 - repo: https://github.com/pre-commit/pre-commit-hooks
-  rev: v4.3.0
+  rev: v4.4.0
   hooks:
   - id: end-of-file-fixer
   - id: fix-byte-order-marker
@@ -48,7 +46,7 @@ repos:
   - id: script-must-have-extension
   - id: require-ascii
   - id: shellcheck
-    exclude: '^(production|tools|google_internal|builders/images)/.*$'
+    exclude: '^(tools|google_internal|images)/.*$'
 
 - repo: https://github.com/pre-commit/mirrors-clang-format
   rev: v14.0.6
@@ -64,7 +62,7 @@ repos:
     name: addlicense
     language: golang
     additional_dependencies:
-    - github.com/google/addlicense@v1.0.0
+    - github.com/google/addlicense@v1.1.0
     always_run: false
     pass_filenames: true
     entry: addlicense -v
@@ -75,7 +73,7 @@ repos:
     name: addlicense check
     language: golang
     additional_dependencies:
-    - github.com/google/addlicense@v1.0.0
+    - github.com/google/addlicense@v1.1.0
     always_run: false
     pass_filenames: true
     entry: addlicense -check
@@ -108,7 +106,7 @@ repos:
     - markdown
 
 - repo: https://github.com/DavidAnson/markdownlint-cli2
-  rev: v0.5.1
+  rev: v0.6.0
   hooks:
   - id: markdownlint-cli2
     name: lint markdown
@@ -143,7 +141,7 @@ repos:
     - --quiet
 
 - repo: https://github.com/psf/black
-  rev: 22.8.0
+  rev: 22.12.0
   hooks:
   - id: black
     name: black python formatter
diff --git a/builders/CHANGELOG.md b/builders/CHANGELOG.md
@@ -2,6 +2,73 @@
 
 All notable changes to this project will be documented in this file. See [commit-and-tag-version](https://github.com/absolute-version/commit-and-tag-version) for commit guidelines.
 
+## [0.10.0](https://team/kiwi-air-force-eng-team/build-system/compare/release-0.9.0...release-0.10.0) (2023-01-06)
+
+
+### Features
+
+* Drop ubuntu package version minor for curl
+
+## [0.9.0](https://team/kiwi-air-force-eng-team/build-system/compare/release-0.8.0...release-0.9.0) (2023-01-04)
+
+
+### Features
+
+* Add zlib1g-dev to build-debian per scp build dependency
+* Update hook versions
+
+
+### Bug Fixes
+
+* Correct non-zero error message and drop sourcing of tools/builder.sh
+* Elide warnings from bazel info
+* Revert from clang-format v15 to v14
+
+## [0.8.0](https://team/kiwi-air-force-eng-team/build-system/compare/release-0.7.0...release-0.8.0) (2022-12-29)
+
+
+### Features
+
+* Add bash and jq to test-tools image
+* Add script to normalize bazel- symlinks
+* Ensure run-tests includes all images
+* Skip symlinks that resolve in normalize-bazel-symlink
+
+## [0.7.0](https://team/kiwi-air-force-eng-team/build-system/compare/release-0.6.0...release-0.7.0) (2022-12-27)
+
+
+### Features
+
+* Add ghz wrapper script
+* Add test-tools image
+
+## [0.6.0](https://team/kiwi-air-force-eng-team/build-system/compare/release-0.5.0...release-0.6.0) (2022-12-12)
+
+
+### Features
+
+* Add EXTRA_DOCKER_RUN_ARGS support in aws-cli
+
+
+### Bug Fixes
+
+* Emit docker build output only on non-zero exit
+* Remove tempfile before exiting
+
+## [0.5.0](https://team/kiwi-air-force-eng-team/build-system/compare/release-0.4.4...release-0.5.0) (2022-12-06)
+
+
+### Features
+
+* Pin versions in presubmit image
+
+
+### Bug Fixes
+
+* Avoid cache for tar image
+* Update version pin for ca-certificates
+* Use images subdirs for image list
+
 ### [0.4.4](https://team/kiwi-air-force-eng-team/build-system/compare/release-0.4.3...release-0.4.4) (2022-11-18)