The concept of scanning an image for vulnerabilities is not new, but scanning is only the beginning.

Scans generate a list of vulnerabilities, but ideally we want to react on what was found in our CI - and maybe break the pipeline if what was found is too dangerous to let it continue.

Ideally, this would happen in a flexible, automated fashion - not only during build time, but also blocking pods using vulnerable images from being deployed (using Admission Controllers). On top of that - we also need to continuously monitor our runtime, because new vulnerabilities will also be discovered after the images have made it to our clusters.

These policies (what do I do with the result, how should I react during build, deployment or runtime?) should be manageable in a flexible, best: Policy as Code (GitOps) approach.

You see, scanning is just the beginning.

For your production needs, we would definitely recommend considering products such as Red Hat Advanced Cluster Security for Kubernetes as it provides

…​just to name a few highlights.

This is far beyond the scope of this Platform Engineering workshop - however, there is a specific OpenShift Security Roadshow that goes into detail on how you can secure your Kubernetes / OpenShift platform using Red Hat Advanced Cluster Security for Kubernetes and manage platform compliance.

This is not a product-specific workshop; we are focusing on the Platform Engineering concepts. As such, it is important we add the scanning capability to our CI - because we need to make this vulnerability information available to developers and architects as early as possible. We’ll add scanning to our CI pipeline - to show you the basic principles, but we will leave it at that.

In this workshop, we will add image scanning using Red Hat Quay's built-in image scanning capability - which is by the way the same scanning engine that Advanced Cluster Security uses.

For this developer build (remember, we are still working in our development branch), we aren’t using the "final" image that will ultimately be promoted to higher level stages - but still, we want to surface vulnerabilities, so our developers and architects can react.

We will add a similar task to our pre-merge and post-merge pipelines, to make sure that we do not introduce vulnerabilities by using different base images, for example. However, at this point, we want to simply show the scan results.

Remember, the earlier a developer gets this information, the sooner he can make an informed decision. ("Shift left").

As you might have noticed while testing the commit verification with our "Secure Parasol Store Development" template - some Vulnerability figures started to appear…​

That happened, because we already added a vulnerability scan step to the pipeline that our new template rolls out:

This task uses the Quay Registry image scanning feature and populates specific fields in the Tekton pipelineRun that the Tekton Red Hat Developer Hub plugin automatically parses - making this information easily accessible to developers.

Developers can also access the "Output" action now (since our task populates it) and follow the URL to the detailed Quay scan results page (login via quayadmin / {common_password}).

Alteratively, Developers can check the task logs for a detailed list of identified vulnerabilities.

So far, we have enabled our developers to view vulnerability information and to sign (and verify) their commits.

As Platform Engineers, we should also make sure that nothing possibly harmful or untrustworthy reaches higher stages, or even production. With that said, let’s add security guardrails to the pipelines that are triggered when we merge our changes back to the Parasol Store main branch:

These pipeline definitions are located in the Parasol Store Build Manifests repository.

So, before we (our developer-self) create a merge request for our parasol-store branch (new-feature-with-signed-commits), let’s augment the pre- and post-merge pipelines with security guardrails:

As Platform Engineer, open the Pipeline Eventlistener - the component listening for webhooks and calling various pipelines based on rules. This is Tekton-specific and will look different based on your CI of choice. The important part is - instead of calling a pipeline that simply checks if code can be built, we are calling a pipeline that also verifies commit signatures and scans the created image.

For the pre-merge pipeline:

For the post-merge pipeline:

For the "Open Prod-PR" pipeline:

Once you have committed your changes, you can open the Parasol Store Build OpenShift GitOps application syncing the build manifests and manually sync it - or wait a couple of minutes until it automatically syncs.

Press (1) Refresh, then (2) Sync

Now that we have enabled our security guardrails, we can create a merge request for our branch:

First, login as a developer (dev1 / {common_password}), then go to your branch in the parasol store and create a merge request (if you don’t see the box suggesting that).

From a GitLab perspective, nothing has changed - a pre-merge verification pipeline has started.

If you open the parasol component’s CI page in Red Hat Developer Hub, you can see that the pre-merge-check-secure pipeline has started (or has already finished).

However (as already noted) this would exceed the scope of this workshop. But please keep in mind that verification should include breaking the pipeline if your criteria are not met.