PodSecurityPolicy (PSP) is being deprecated in Kubernetes 1.21. There are several external admission controllers available like OPA/Gatekeeper, but getting started with OPA can be challenging.
Otomi, a complete suite of integrated K8s apps and add-ons, combined with developer self-service, is using OPA/Gatekeeper as the standard for providing policy enforcement. All apps integrated into Otomi are pre-configured based on sane defaults and best practices. Configuration of all apps is managed using a version control system where the desired state is reflected as code and the cluster state is automatically updated.
One of the key principles of Otomi is that it is easy to use and provides security from the ground up. In this article, I'll shortly explain how to get started with OPA/Gatekeeper using Otomi.
First, install Otomi on a Kubernetes cluster in your cloud of choice. You can use the Otomi Quickstart to provision a Kubernetes cluster in AWS, Azure, GCP, or install Otomi using the Helm chart. To install Otomi using the Helm chart, make sure you have a Kubernetes cluster running with at least 16 CPU threads and 32GB+ RAM available in the worker node pool.
Add the Otomi repository:
helm repo add otomi https://otomi.io/otomi-core helm repo update
Create a values file with the following values:
cluster: k8sVersion: '1.20' # currently 1.18, 1.19, 1.20 and 1.21 are supported name: # the name of your cluster provider: # choose between aws, azure, google or onprem
Install the chart:
helm install -f values.yaml otomi otomi/otomi
When the installer job (in the default namespace) has finished, copy the URL and the generated password from the bottom of the logs and complete the post-installation steps.
You can now log in to the console. In the left pane (under Enterprise) you will see
Policies and when you click on it, you will see the list of all policies available in Otomi.
Managing policies with Otomi Console
Otomi contains a selection of 13 usable OPA policies and adapted them to be used by Conftest as well as for static analysis of manifests generated by Otomi.
You can turn policies on or off, and also set default parameters to be used. Select the policy you would like to edit (enable/disable or change default parameters), click
Submit and click
Deploy Changes. The changes will now be applied to the Otomi configuration.
Otomi supports 3 Gatekeeper modes:
- Permissive (default)
In both Enforcing and Permissive modes, individual policies can be switched on or off. By default, Gatekeeper is enabled in permissive mode (logging & non-blocking).
Customization of the policies is supported based on the Otomi schema or by using Otomi Console. In case of specific requirements, operators can add their own custom policies.
To change the mode, open Gitea (in the Apps) and navigate to
To switch to enforcing mode, edit the file and set
charts: gatekeeper-operator: disableValidatingWebhook: false
To disable policy enforcement, edit the file and set
charts: gatekeeper-operator: enabled: false
In the team section in Otomi console, developers can use the Gatekeeper dashboard to see all the policy violations relevant to their deployments. First, create a Team in Otomi if you haven't done it already. In the team section, click
Apps and then click
Running policy checks against any kind of YAML resource
With a simple command, you can test if a Helm chart is violating any policies.
The generated YAML files are streamed into Conftest and policies are tested one by one.
$ helm template stable/keycloak | conftest test --policy ./policies/ --all-namespaces - FAIL - Policy: container-limits - container <keycloak> has no resource limits FAIL - Policy: container-limits - container <keycloak-test> has no resource limits 162 tests, 160 passed, 0 warnings, 2 failures, 0 exceptions
By examining the log message, you can see that the container-limits policy is marking two resources as failures. Now, all you have to do is modify the templates to provide a "sensitive" amount of resource limits to the indicated containers and our policy checks will pass successfully!
This is pretty useful if you want to adopt new Helm applications, but don't want to deploy anything to the cluster unless it's well-examined for any violations. Conftest supports passing values to the policies using the –data option, which allows policy designers to configure different settings through parameters.
Policy exceptions capabilities
To create some flexibility, Otomi offers the ability to make exceptions by examining granular annotation information for every resource.
An example: using the following annotations will allow the entire pod or certain containers from the pod to exclude one or more policies:
# Annotation for the entire pod policy.otomi.io/ignore: psp-allowed-repos # Annotation for Istio sidecar based containers policy.otomi.io/ignore-sidecar: psp-allowed-users,psp-capabilities # Annotation for a specific container (name=app-container) policy.otomi.io/ignore/app-container: banned-image-tags
Implementing OPA/Gatekeeper can be a challenging endeavor. Some cloud providers have already integrated Gatekeeper into their managed Kubernetes service, but this results in a lock-in, while Otomi on the other hand is cloud-agnostic. Using integrated policies in a managed Kubernetes service is also not always very flexible. For instance, in Azure, policy enforcement can only be turned on or off. There is no option to run in permissive mode.
Otomi offers full Kubernetes security policies capabilities using OPA/Gatekeeper and Conftest. In the (default) permissive mode, developers can deploy their applications without having to directly comply. They can then see all the violations and adjust their deployments accordingly. With Otomi, you can have all OPA/Gatekeeper functionality with a sane set of implemented default policies, an easy user experience, and lots of flexibility, immediately available after installing Otomi.
To get started with Otomi today. Check out the GitHub repo here.