Introduction
Currently, among the paramount concerns for enterprise IT organizations, supply chain security holds utmost significance. This is particularly critical given the increasing integration and utilization of open source artifacts within business-critical systems and applications. As open source code constitutes 75% of application code bases, these components are now subject to heightened scrutiny, driven by a staggering 742% increase in software supply chain attacks since 2020. In response, customers are actively looking to incorporate guardrails into their software supply chain and development life cycles, aiming to foster innovation while upholding robust security measures.
An established software supply chain offers organizations the advantage of implementing DevSecOps practices and tools, ensuring early security measures for software components during the SDLC and automating security procedures throughout every phase of the software development life cycle.
One aspect of securing the software supply chain is to ensure that all the container images deployed in Openshift clusters are signed and more secured. In this Blog we will show how to leverage Cosign and RHACS tools to enforce the signature of the container images within the CI/CD Pipelines.
Prerequisite:
- Openshift cluster v4.11 or later
- Access to Quay.io and account created to be able to pull and push created images
- Red hat Openshift Pipeline operator v1.10 or later
- RHACS v.3.74 or later installed and configured
- Have Cosign, roxctl and tekton cli (optional) installed in the local environment
- Have access to the public repo which host the voting application used for the demo
- Have access to the public repo which contains our pipeline manifests.
Configure Quay.io repository
Access your Quay.io account and create a new public repository. Next create a robot account user and grant it an admin permission. Robot Account Token will be automatically created.
Add registry credentials to Openshift
We need to use the pull secret manifest that has been generated and apply it to the Openshift ‘demo’ project within our cluster. This will allow the service account to access the registry.
- Create a project for the sample application:
- Apply the pull secret containing the credentials for Quay registry to push/pull the images and signatures:
- Link the imagePullSecret to the ServiceAccount ‘pipeline’ :
oc project demo
oc create -f szemmour-acs-demo-secret.yml -n demo
oc secrets link pipeline <pull_secret_name> -n demo
Integrate Roxctl CLI with RHACS
The command-line interface (CLI) roxctl is used for running commands against RHACS.
Since our goal is to perform image checks for policy violations and reporting, an integration between roxctl cli and RHACS is needed. For that, the roxctl cli needs to be able to authenticate against the RHACS central cluster. Thus, the next step is to generate the API token to be used for the authentication.
Via the RHACS console, Go to Platform Configuration -> Integrations -> Authentication Tokens -> API Token and generate new API token as shown in below images:
Copy the generated token and export it into the ROX_API_TOKEN variable or store it in a file in the local environment. You will not be able to view it again.
export ROX_API_TOKEN=”<api_token>”
export ACS_ROUTE=$(oc get route -n stackrox central -o jsonpath='{.spec.host}')
To validate the authentication we can install Install the roxctl cli and use the roxctl with ‘check image’ option in order verify if the API Token is working properly:
roxctl --insecure-skip-tls-verify image check --endpoint $ACS_ROUTE:443 --image registry.access.redhat.com/ubi8/ubi-minimal:8.7
The command should output a couple of violated policies with their remediations similar to image below.
Since the roxctl will be executed as part of a task in the pipeline, we will need to provide both rox_api_token and the rox_central_endpoint as an openshift secret. Create the secret in the ‘demo’ namespace using the command below.
oc create secret generic roxauthsecret --from-literal rox_api_token="$ROX_API_TOKEN" --from-literal rox_central_endpoint="$ACS_ROUTE:443" -n demo
Generate Cosign keys
Before we can perform image signing as part of our pipeline and to enable RHACS to verify those images we need to create a key-pair signature using the Cosign tool. The key-pair needs to be stored as a secret in the application namespace, and the public key needs to be placed in RHACS for later verification. Note that for a production environment an integration with a KMS provider will procure more secure key management. This documentation contains detailed instructions on how to configure Cosign to work with KMS providers eg: Hashicorp Vault, AWS KMS …
Cosign uses keys stored in Openshift Secrets to sign and verify signatures. In order to generate a secret you have to pass the Cosign generate-key-pair command a k8s://[NAMESPACE]/[NAME] URI specifying the namespace and secret name as described here.
cosign generate-key-pair k8s://demo/cosign
The Cosign command prompts the user to enter the password for the private key. Type any password that makes sense to you and hit Enter.
After generating the key pair, Cosign will store it in Openshift Secret using your current context. The secret will contain the private and public keys, as well as the password to decrypt the private key.
To validate the creation of Cosign secret. List the secrets under the ‘demo’ namespace and make sure Cosign secret exist.
$ oc get secret -n demo
NAME TYPE DATA AGE
cosign Opaque 3 2m11s
…
In the same folder where the Cosign generate-key-pair command was executed you will notice the creation of the cosign.pub file. This file represents the public key of the Cosign private key. The same base64 content of the cosign.pub file is stored in the Cosign secret under the ‘demo’ namespace.
Later, when verifying an image signature in the Tekton pipeline using Cosign verify command the key will be automatically decrypted using the password stored in the openshift secret.
Configuring signature integration
Let’s add the Cosign public key generated in the previous steps into the ACS console in order for RHACS to execute image signature verification. More details could be found here
Under Integrations select Signature and then click on New Integration to create a new signature integration. Next add integration name and copy/paste the public key from the generated cosign.pub file into the public key value section as shown in the image below.
Once we've created the signature integration we need to create a policy that will use it.
Create ACS Policy for Image Signature Verification
ACS comes with a lot of default policies to help organizations better meet their security requirements. These policies apply across the build, deploy and runtime phases of the container application lifecycle. The different stages can be selected under the Lifecycle stages section of the Policy behavior configuration as shown below.
Let's create a policy that applies to the Build and Deploy stages of the container application lifecycle, and we also want to enable enforcement at deployment time.
The Build option means that when a container image check is initiated with roxctl the policy will be reported in the output. The Deploy option means that the policy applies when a deployment is created on a secured cluster. This invokes the RHACS admission controller to validate the workload against the enforced policies
Let's create a policy that applies to the Build and Deploy stages of the container application lifecycle, and we also want to enable enforcement at deployment time.
Under Platform Configuration -> Policy Management select Create Policy and choose all the options shown in below images.
Now we need to configure the policy criteria. To achieve this, add the Image signature policy into the Policy Criteria as shown in the image below.
Next, Click the Select button and specify the Cosign-integration signer.
Finally, review and save the policy.
Integration validation
To evaluate the integration, a pipeline will be created to build and deploy the vote api microservice. We will present two scenarios to demonstrate the capabilities of signature validation and enforcement. In the initial scenario, the pipeline will build and attempt to deploy the application utilizing an unsigned container image. The subsequent scenario follows a similar path, but in this case, the image will be signed before its deployment.
Deploy Tekton tasks and pipelines
Create the tasks which will be used by the pipelines later using the following command:
oc create -k "github.com/szemmour-rh/acs-cosgin-integration/pipelines/tasks/?ref=main" -n demo
You can take a look at the tasks you have created:
oc get task
Operator installs a few ClusterTask which you can see by executing:
Oc get clusterTask
We will be using buildah clusterTasks, which gets installed along with the Openshift Pipeline Operator. Buildah task builds source into a container image and then pushes it to a container registry. Create the pipelines and the PVC using the following command:
oc create -k "github.com/szemmour-rh/acs-cosgin-integration/pipelines/pipeline/?ref=main" -n demo
Build and deploy unsigned Images
Now let’s run the pipeline which builds an unsigned image, push it to Quay and deploy the application using the same unsigned image. For the purpose of the demo we will execute the apply-manifest task regardless of the result of the image-check task in order to be able to illustrate the state of the deployment.
oc create -f https://raw.githubusercontent.com/szemmour-rh/acs-cosgin-integration/main/pipelines/pipelinesrun/unsigned/pipelinerun-unsigned.yaml -n demo
We can observe that the image-check task failed and upon inspecting the logs of the task, we can see that the signature check policy is being violated.
When we look at our vote-api-unsigned application deployment status and event in the Openshift console we notice that the RHACS image verification policy via the admission controller is blocking the pipeline to use the unsigned image for deployment as shown in below images.
If you navigate to Violations in the RHACS console you will notice that a policy violation has been created. The policy violation lists the container image that was not signed. Under the Enforcement tab you can also verify the action taken which is that the deployment vote-api-unsigned has been scaled to 0 replicas in response to the policy violation.
Build and deploy signed image
This time we will include the image signing task after the build and we will use the newly generated image for the vote-api-signed deployment.
oc create -f https://raw.githubusercontent.com/szemmour-rh/acs-cosgin-integration/main/pipelines/pipelinesrun/signed/pipelinerun-signed.yaml -n demo
Compared to the previous PipelineRun, there’s no signature check policy listed under the Policy check results of the image-check task logs as shown in the image below.
To make sure the image has been signed we can look at the image Tag in Quay.io which will contain a validation sign with the message “This Tag has been signed via cosign”.
Finally, we notice that the deployment vote-api-signed was successful.
Summary
Delving into the realms of RHACS and Cosign Integration, this article offers a concise yet insightful glimpse into their synergy as open-source security endeavors, synergistically augmenting the safety of the software supply chain. Hence, by integrating these two initiatives, developers can foster a culture of heightened safety while constructing applications, fostering a robust and more secure development ecosystem.
Sobre o autor
Navegue por canal
Automação
Últimas novidades em automação de TI para empresas de tecnologia, equipes e ambientes
Inteligência artificial
Descubra as atualizações nas plataformas que proporcionam aos clientes executar suas cargas de trabalho de IA em qualquer ambiente
Nuvem híbrida aberta
Veja como construímos um futuro mais flexível com a nuvem híbrida
Segurança
Veja as últimas novidades sobre como reduzimos riscos em ambientes e tecnologias
Edge computing
Saiba quais são as atualizações nas plataformas que simplificam as operações na borda
Infraestrutura
Saiba o que há de mais recente na plataforma Linux empresarial líder mundial
Aplicações
Conheça nossas soluções desenvolvidas para ajudar você a superar os desafios mais complexos de aplicações
Programas originais
Veja as histórias divertidas de criadores e líderes em tecnologia empresarial
Produtos
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Red Hat Cloud Services
- Veja todos os produtos
Ferramentas
- Treinamento e certificação
- Minha conta
- Suporte ao cliente
- Recursos para desenvolvedores
- Encontre um parceiro
- Red Hat Ecosystem Catalog
- Calculadora de valor Red Hat
- Documentação
Experimente, compre, venda
Comunicação
- Contate o setor de vendas
- Fale com o Atendimento ao Cliente
- Contate o setor de treinamento
- Redes sociais
Sobre a Red Hat
A Red Hat é a líder mundial em soluções empresariais open source como Linux, nuvem, containers e Kubernetes. Fornecemos soluções robustas que facilitam o trabalho em diversas plataformas e ambientes, do datacenter principal até a borda da rede.
Selecione um idioma
Red Hat legal and privacy links
- Sobre a Red Hat
- Oportunidades de emprego
- Eventos
- Escritórios
- Fale com a Red Hat
- Blog da Red Hat
- Diversidade, equidade e inclusão
- Cool Stuff Store
- Red Hat Summit