Skip to content

Commit

Permalink
Merge branch 'main' into 1.13-scaling-docs
Browse files Browse the repository at this point in the history
  • Loading branch information
JimBugwadia authored Oct 28, 2024
2 parents 9b66178 + 7451b09 commit f4b8d91
Show file tree
Hide file tree
Showing 29 changed files with 501 additions and 182 deletions.
2 changes: 1 addition & 1 deletion content/en/docs/applying-policies/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ The [Kyverno Policies](/policies/) repository contains several policies you can

On installation, Kyverno runs as a [dynamic admission controller](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/) in a Kubernetes cluster. Kyverno receives validating and mutating admission webhook HTTP callbacks from the Kubernetes API server and applies matching policies to return results that enforce admission policies or reject requests.

Policies with validation rules can be used to block insecure or non-compliant configurations by setting the [`validationFailureAction`](../writing-policies/validate.md#validation-failure-action) to `Enforce`. Or, validation rules can be applied using periodic scans with results available as [policy reports](../policy-reports/).
Policies with validation rules can be used to block insecure or non-compliant configurations by setting the [`failureAction`](../writing-policies/validate.md#failure-action) to `Enforce`. Or, validation rules can be applied using periodic scans with results available as [policy reports](../policy-reports/).

Rules in a policy are applied in the order of definition. During [admission control](https://kubernetes.io/docs/reference/access-authn-authz/extensible-admission-controllers/), mutation rules are applied before validation rules. This allows validation of changes made during mutation. Note that **all** mutation rules are applied first across all policies before any validation rules are applied.

Expand Down
11 changes: 6 additions & 5 deletions content/en/docs/installation/customization.md
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,7 @@ kyverno:reports-controller:core | -- | --


{{% alert title="Note" color="info" %}}
The Kyverno admission, background, and reports controller have a role binding to the built-in `view` role. This allows these Kyverno controllers view access to most namespaced resources. You can customize this role during Helm installation using variables like `admissionController.rbac.viewRoleName`.
The Kyverno admission, background, and reports controller have a role binding to the built-in `view` role. This allows these Kyverno controllers view access to most namespaced resources. You can customize this role during Helm installation using the variables `admissionController.rbac.viewRoleName`, `backgroundController.rbac.viewRoleName`, and `reportsController.rbac.viewRoleName`.
{{% /alert %}}

#### Customizing Permissions
Expand Down Expand Up @@ -365,17 +365,18 @@ The following flags can be used to control the advanced behavior of the various
| `cleanupServerPort` (C) | 9443 | Defines the port used by the cleanup server. Usually changed in tandem with `webhookServerPort`.|
| `clientRateLimitBurst` (ABCR) | 300 | Configures the maximum burst for throttling. Uses the client default if zero. |
| `clientRateLimitQPS` (ABCR) | 300 | Configures the maximum QPS to the API server from Kyverno. Uses the client default if zero. |
| `eventsRateLimitBurst` (ABCR) | 2000 | Configures the maximum burst for throttling for events. Uses the client default if zero. |
| `eventsRateLimitQPS` (ABCR) | 1000 | Configures the maximum QPS to the API server from Kyverno for events. Uses the client default if zero. |
| `disableMetrics` (ABCR) | false | Specifies whether to enable exposing the metrics. |
| `dumpPayload` (AC) | false | Toggles debug mode. When debug mode is enabled, the full AdmissionReview payload is logged. Additionally, resources of kind Secret are redacted. Should only be used in policy development or troubleshooting scenarios, not left perpetually enabled. |
| `dumpPatches` (A) | false | Toggles debug mode. When debug mode is enabled, the full patch payload is logged |
| `eventsRateLimitBurst` (ABCR) | 2000 | Configures the maximum burst for throttling for events. Uses the client default if zero. |
| `eventsRateLimitQPS` (ABCR) | 1000 | Configures the maximum QPS to the API server from Kyverno for events. Uses the client default if zero. |
| `enableConfigMapCaching` (ABR) | true | Enables the ConfigMap caching feature. |
| `enableDeferredLoading` (A) | true | Enables deferred (lazy) loading of variables (1.10.1+). Set to `false` to disable deferred loading of variables which was the default behavior in versions < 1.10.0. |
| `enablePolicyException` (ABR) | true | Set to `true` to enable the [PolicyException capability](../writing-policies/exceptions.md). |
| `enablePolicyException` (ABR) | false | Set to `true` to enable the [PolicyException capability](../writing-policies/exceptions.md). |
| `enableReporting` (ABCR) | validate,mutate,mutateExisting,generate,imageVerify | Comma separated list to enables reporting for different rule types. (validate,mutate,mutateExisting,generate,imageVerify) |
| `enableTracing` (ABCR) | false | Set to enable exposing traces. |
| `enableTuf` (AR) | | Enable tuf for private sigstore deployments. |
| `exceptionNamespace` (ABR) | | Set to the name of a Namespace where [PolicyExceptions](../writing-policies/exceptions.md) will only be permitted. PolicyExceptions created in any other Namespace will throw a warning. If not set, PolicyExceptions from all Namespaces will be considered. Implies the `enablePolicyException` flag is set to `true`. Neither wildcards nor multiple Namespaces are currently accepted. |
| `exceptionNamespace` (ABR) | | Set to the name of a Namespace where [PolicyExceptions](../writing-policies/exceptions.md) will only be permitted. PolicyExceptions created in any other Namespace will throw a warning. If set to "*", PolicyExceptions from all Namespaces will be accepted. Note that wildcards and multiple Namespace entries are not supported. It is required if the `enablePolicyException` flag is set to true. |
| `forceFailurePolicyIgnore` (A) | false | Set to force Failure Policy to `Ignore`. |
| `generateValidatingAdmissionPolicy` (A) | false | Specifies whether to enable generating Kubernetes ValidatingAdmissionPolicies. |
| `genWorkers` (B) | 10 | The number of workers for processing generate policies concurrently. |
Expand Down
2 changes: 1 addition & 1 deletion content/en/docs/installation/methods.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ reportsController:
replicas: 3
```
For all of the available values and their defaults, please see the Helm chart [README](https://github.com/kyverno/kyverno/tree/release-1.10/charts/kyverno). You should carefully inspect all available chart values and their defaults to determine what overrides, if any, are necessary to meet the particular needs of your production environment.
For all of the available values and their defaults, please see the Helm chart [README](https://github.com/kyverno/kyverno/tree/release-1.13/charts/kyverno). You should carefully inspect all available chart values and their defaults to determine what overrides, if any, are necessary to meet the particular needs of your production environment.
{{% alert title="Note" color="warning" %}}
All Kyverno installations require the admission controller be among the controllers deployed. For a highly-available installation, at least 2 or more replicas are required. Based on scalability requirements, and cluster topology, additional replicas can be configured for each controller.
Expand Down
7 changes: 7 additions & 0 deletions content/en/docs/installation/scaling.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,10 @@ API requests, operations, and activities which match corresponding Kyverno rules
| DELETE | ConfigMap | | 1 |

These figures were captured using K3d v5.4.9 on Kubernetes v1.26.2 and Kyverno 1.10.0-alpha.2 with a 3-replica admission controller. When testing against KinD, there may be one less DELETE AdmissionReview for Pod-related operations.

### Kubernetes api-server and etcd resource footprint

In clusters with many Kyverno policy resources, the resource footprint of some core Kubernetes components may be affected, in particular the kube-apiserver and etcd.

For example, if you create several thousand Kyverno policy resources, double check that the kube-apiserver pods have head room to increase its memory allocations, otherwise
the cluster may crash entirely.
2 changes: 1 addition & 1 deletion content/en/docs/installation/uninstallation.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ helm uninstall kyverno kyverno/kyverno -n kyverno

### Clean up Webhooks

Kyverno by default will try to clean up all its webhooks when terminated. But in cases where its RBAC resources are removed first, it will lose the permission to do so properly.
Kyverno by default will try to clean up all its webhooks when terminated for helm based installations. But in other cases where Kyverno's RBAC resources are removed first, it will lose the permission to do so properly. Kyverno supports deletion using finalizers to ensure resource deletion in right order. Use `--autoDeleteWebhooks` in admission-controller and cleanup-controller deployment to enable this feature, this feature will be enabled by default in future releases.

If manual webhook removal is necessary, use the below commands.

Expand Down
59 changes: 58 additions & 1 deletion content/en/docs/installation/upgrading.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,4 +16,61 @@ Direct upgrades from previous versions are not supported when using the YAML man

### Upgrade Kyverno with Helm

An upgrade from versions prior to Kyverno 1.10 to versions at 1.10 or higher using Helm requires manual intervention and cannot be performed via a direct upgrade process. Please see the 1.10 migration guide [here](https://github.com/kyverno/kyverno/blob/release-1.10/charts/kyverno/README.md#migrating-from-v2-to-v3) for more complete information.
An upgrade from versions prior to Kyverno 1.10 to versions at 1.10 or higher using Helm requires manual intervention and cannot be performed via a direct upgrade process. Please see the 1.10 migration guide [here](https://githubviewRoleName.com/kyverno/kyverno/blob/release-1.13/charts/kyverno/README.md#migrating-from-v2-to-v3) for more complete information.


## Upgrading to Kyverno v1.13

Kyverno version 1.13 contains the following breaking configuration changes:

1. **Removal of wildcard permissions**: prior versions contained wildcard view permissions, which allowed Kyverno controllers to view all resources including secrets and other sensitive information. In 1.13 the wildcard view permission was removed and a role binding to the default `view` role was added. See the documentation section on [Role Based Access Controls](./customization.md#role-based-access-controls) for more details. This change will not impact policies during admission controls but may impact reports, and may impact users with mutate and generate policies on custom resources as the these controller may no longer be able to view these custom resources.

To upgrade to 1.13 and continue to allow wildcard view permissions for all Kyverno controllers, use a Helm values file that grants these permissions as specified below:

```yaml
admissionController:
clusterRole:
extraResources:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- watch
backgroundController:
clusterRole:
extraResources:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- watch
reportsController:
clusterRole:
extraResources:
- apiGroups:
- '*'
resources:
- '*'
verbs:
- get
- list
- watch
```
**NOTE**: using wildcard permissions is not recommended. Use explicit permissions instead.
2. **Default exception settings**: the Helm chart values of the prior versions enabled exceptions by default for all namespaces. This creates a potential security issue. See **CVE-2024-48921** for more details. This change will impact users who were relying on policy exceptions to be enabled in all namespaces.
To maintain backwards compatibility, you can configure the Helm chart values to allow the same settings as the prior version. To upgrade to 1.13 and continue to allow configuring exceptions in all namespaces, set the Helm value `features.policyExceptions.namespace` to `*`:

```sh
helm upgrade kyverno kyverno/kyverno -n kyverno --set features.policyExceptions.enabled=true --set features.policyExceptions.namespace="*"
```

**NOTE**: limiting exceptions to a specific namespace is recommended.
4 changes: 2 additions & 2 deletions content/en/docs/introduction/quick-start.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ Next, select the quick start guide in which you are interested. Alternatively, s

In the validation guide, you will see how simple an example Kyverno policy can be which ensures a label called `team` is present on every Pod. Validation is the most common use case for policy and functions as a "yes" or "no" decision making process. Resources which are compliant with the policy are allowed to pass ("yes, this is allowed") and those which are not compliant may not be allowed to pass ("no, this is not allowed"). An additional effect of these validate policies is to produce Policy Reports. A [Policy Report](../policy-reports/_index.md) is a custom Kubernetes resource, produced and managed by Kyverno, which shows the results of policy decisions upon allowed resources in a user-friendly way.

Add the policy below to your cluster. It contains a single validation rule that requires that all Pods have the `team` label. Kyverno supports different rule types to validate, mutate, generate, cleanup, and verify image configurations. The field `validationFailureAction` is set to `Enforce` to block Pods that are non-compliant. Using the default value `Audit` will report violations but not block requests.
Add the policy below to your cluster. It contains a single validation rule that requires that all Pods have the `team` label. Kyverno supports different rule types to validate, mutate, generate, cleanup, and verify image configurations. The field `failureAction` is set to `Enforce` to block Pods that are non-compliant. Using the default value `Audit` will report violations but not block requests.

```yaml
kubectl create -f- << EOF
Expand All @@ -31,7 +31,6 @@ kind: ClusterPolicy
metadata:
name: require-labels
spec:
validationFailureAction: Enforce
rules:
- name: check-team
match:
Expand All @@ -40,6 +39,7 @@ spec:
kinds:
- Pod
validate:
failureAction: Enforce
message: "label 'team' is required"
pattern:
metadata:
Expand Down
13 changes: 6 additions & 7 deletions content/en/docs/kyverno-cli/usage/apply.md
Original file line number Diff line number Diff line change
Expand Up @@ -264,7 +264,6 @@ kind: ClusterPolicy
metadata:
name: cm-globalval-example
spec:
validationFailureAction: Enforce
background: false
rules:
- name: validate-mode
Expand All @@ -274,6 +273,7 @@ spec:
kinds:
- Pod
validate:
failureAction: Enforce
message: "The value {{ request.mode }} for val1 is not equal to 'dev'."
deny:
conditions:
Expand Down Expand Up @@ -362,7 +362,6 @@ kind: ClusterPolicy
metadata:
name: enforce-pod-name
spec:
validationFailureAction: Audit
background: true
rules:
- name: validate-name
Expand All @@ -378,6 +377,7 @@ spec:
values:
- managed
validate:
failureAction: Audit
message: "The Pod must end with -nginx"
pattern:
metadata:
Expand Down Expand Up @@ -446,7 +446,6 @@ metadata:
annotations:
pod-policies.kyverno.io/autogen-controllers: DaemonSet,Deployment,StatefulSet
spec:
validationFailureAction: Enforce
background: false
rules:
- name: example-configmap-lookup
Expand Down Expand Up @@ -495,7 +494,7 @@ policies:
dictionary.data.env: dev1
```

Policies that have their validationFailureAction set to `Audit` can be set to produce a warning instead of a failure using the `--audit-warn` flag. This will also cause a non-zero exit code if no enforcing policies failed.
Policies that have their failureAction set to `Audit` can be set to produce a warning instead of a failure using the `--audit-warn` flag. This will also cause a non-zero exit code if no enforcing policies failed.

```sh
kyverno apply /path/to/policy.yaml --resource /path/to/resource.yaml --audit-warn
Expand Down Expand Up @@ -551,7 +550,6 @@ kind: ClusterPolicy
metadata:
name: require-pod-requests-limits
spec:
validationFailureAction: Audit
rules:
- name: validate-resources
match:
Expand All @@ -560,6 +558,7 @@ spec:
kinds:
- Pod
validate:
failureAction: Audit
message: "CPU and memory resource requests and limits are required"
pattern:
spec:
Expand Down Expand Up @@ -703,7 +702,6 @@ kind: ClusterPolicy
metadata:
name: max-containers
spec:
validationFailureAction: Enforce
background: false
rules:
- name: max-two-containers
Expand All @@ -713,6 +711,7 @@ spec:
kinds:
- Pod
validate:
failureAction: Enforce
message: "A maximum of 2 containers are allowed inside a Pod."
deny:
conditions:
Expand All @@ -725,7 +724,7 @@ spec:
Policy Exception manifest (`exception.yaml`):

```yaml
apiVersion: kyverno.io/v2beta1
apiVersion: kyverno.io/v2
kind: PolicyException
metadata:
name: container-exception
Expand Down
7 changes: 4 additions & 3 deletions content/en/docs/kyverno-cli/usage/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -228,7 +228,6 @@ kind: ClusterPolicy
metadata:
name: disallow-latest-tag
spec:
validationFailureAction: Audit
rules:
- name: require-image-tag
match:
Expand All @@ -237,6 +236,7 @@ spec:
kinds:
- Pod
validate:
failureAction: Audit
message: "An image tag is required."
pattern:
spec:
Expand All @@ -249,6 +249,7 @@ spec:
kinds:
- Pod
validate:
failureAction: Audit
message: "Using a mutable image tag e.g. 'latest' is not allowed."
pattern:
spec:
Expand Down Expand Up @@ -544,7 +545,6 @@ kind: ClusterPolicy
metadata:
name: disallow-host-namespaces
spec:
validationFailureAction: Enforce
background: false
rules:
- name: host-namespaces
Expand All @@ -554,6 +554,7 @@ spec:
kinds:
- Pod
validate:
failureAction: Enforce
message: >-
Sharing the host namespaces is disallowed. The fields spec.hostNetwork,
spec.hostIPC, and spec.hostPID must be unset or set to `false`.
Expand All @@ -567,7 +568,7 @@ spec:
Policy Exception manifest (`delta-exception.yaml`):

```yaml
apiVersion: kyverno.io/v2beta1
apiVersion: kyverno.io/v2
kind: PolicyException
metadata:
name: delta-exception
Expand Down
Loading

0 comments on commit f4b8d91

Please sign in to comment.