As a person who spends time playing computer role playing games (cRPGs), I see a lot of discussion of the quality of life (QoL) aspects of these games. Does the game have an overly complicated inventory system, a burdensome user interface or require you to manually buff your characters before each fight? Improvements in these areas enhance the gamer's quality of life.
Similarly, there are a variety of QoL tips that can improve the experience of developers and operators when interacting with Red Hat OpenShift GitOps. This blog looks at a few of my favorites.
Use annotation tracking
Argo CD, the upstream project of OpenShift GitOps, needs to track the resources in the cluster it has deployed when deploying applications. By default, both Argo CD and OpenShift GitOps use labels for the tracking method. While label tracking does the job, it only includes a minimal amount of information due to the Kubernetes 63-character limit.
This limitation manifests itself when Argo CD deploys resources that, in turn, generate additional resources. This is most commonly experienced deploying custom resources for operators. The operator will often copy all of the labels and annotations from the custom resource to the additional resources it creates, including the Argo CD tracking label.
While this behavior is generally desirable, copying the tracking label causes Argo CD to treat the application as out of sync. This occurs because Argo CD sees the tracking label and thinks it deployed the resource, but it cannot find the resource in Git since it was created by something else.
You can work around this behavior by adding the IgnoreExtraneous annotation to custom resources where this situation occurs, but it results in additional work and maintenance.
A better solution is to change the tracking method annotations. With annotation tracking, Argo CD can include additional information since annotations do not have the same character limit as labels. Argo CD uses this additional information to automatically weed out false positives.
To change the tracking method in OpenShift GitOps, simply add the following to your Argo CD custom resource:
apiVersion: argoproj.io/v1beta1
kind: ArgoCD
metadata:
name: openshift-gitops
namespace: openshift-gitops
spec:
resourceTrackingMethod: annotation
...
Note that you will likely need to re-sync all applications after making this change. Fortunately, you can use the Argo CD user interface to quickly sync everything in one go.
![Screenshot of the "Sync Apps" button in the Argo CD user interface](/rhdc/managed-files/styles/wysiwyg_full_width/private/gitops-tips-img1.png.webp?itok=GB2Vk4_A)
There is an issue upstream with changing the default tracking method from labels to annotations (or annotations+labels). However, caution is warranted to avoid disruption in existing installations (that's why it isn't the default).
Override automatic sync for App of Apps
App of Apps is a common pattern in OpenShift GitOps where you have an Argo CD application that deploys several other applications. This can be useful when you want to simplify a bootstrap process or deploy applications in a specific order via sync waves.
When troubleshooting one of the dependent applications, however, you cannot temporarily disable automatic sync on the application. If you attempt to do so, the parent application will simply reset it back to its state in Git. The workaround is a two-step process of disabling auto-sync on the parent application and disabling it in the child application.
Fortunately, you can use the ignoreDifferences
feature in the parent application to simplify this process. To do so, add ignoreDifferences
to your parent App of App as per the example below, taken from my cluster bootstrap application:
apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
name: cluster-config-bootstrap
namespace: openshift-gitops
spec:
...
ignoreDifferences:
- group: argoproj.io
kind: Application
managedFieldsManagers:
- argocd-server
jsonPointers:
- /spec/syncPolicy/automated
This configuration tells Argo CD to ignore any changes made by the argocd-server
service account to the spec.syncPolicy.automated
field. The argocd-server
service account is used by the pod providing the Argo CD user interface. Any changes made to an application from the UI will use this service account.
Thus, this change enables you to use the Argo CD UI to override the automated field as needed since Argo CD will now ignore this difference.
I cannot take credit for this tip. I stumbled across it somewhere and have since lost where I got it. Kudos to the original contributor, whomever they may be!
Easily test custom health checks
I firmly believe in writing custom health checks to support the custom resources that various operators make available. These custom health checks enable Argo CD to understand the state of the resource (Healthy, Progressing, Degraded, etc.) and are required if you want to use sync waves with the resource.
The custom health checks are written in the Lua programming language. Testing complex checks in Argo CD can be challenging and time-consuming. You can use the argocd
command line interface to quickly and easily test these health checks independent of an Argo CD deployment on a cluster.
To test the resource, you need a YAML file copy of the argocd-cm
configmap and a copy of the resource you want to check. In OpenShift GitOps, the argocd-cm
configmap is automatically generated in the namespace where your Argo CD instance is located. For example, to get the configmap for the default openshift-gitops
instance, you can do the following to retrieve it and save it as a file:
oc get configmap argocd-cm -n openshift-gitops -o yaml > argocd-cm.yaml
Use a similar command when you want to retrieve the resource. Once you have these two files, you can test the health check with the following command:
argocd admin settings resource-overrides health <resource-yaml> --argocd-cm-path <argocd-cm-yaml>
While debugging a problematic health check for a subscription, I ran the command with the following results:
$ argocd admin settings resource-overrides health ./bad-sub.yaml --argocd-cm-path ./argocd-cm.yaml
INFO[0000] Starting configmap/secret informers
INFO[0000] Configmap/secret informer synced
STATUS: Healthy
MESSAGE: 1: CatalogSourcesUnhealthy | False
You can see the health check was successful and returned a Healthy
status. Also, note the MESSAGE
block. This is the message portion of the health check. It is useful to include additional information that could help teams to better understand why a specific status was returned.
Use kubectl-neat to export clean resources
It's a common GitOps practice to deploy things into a cluster initially using a GUI like the OpenShift console or tools like oc new-app
and then export the resources into a Git repository.
Resources in a Kubernetes cluster tend to have a lot of cruft in them (managed fields, status, etc.) that is painful to remove from the exported resources manually. A great tool for handling this automatically is the kubectl-neat plugin.
This tool is a plugin to kubectl and oc. To install it, get the binary from the GitHub repository releases and put it somewhere on your path. Then, when fetching a resource from the cluster, you can pipe the results to neat. Here's an example:
oc get service my-service -o yaml | oc neat > my-service.yaml
Use Global projects
In multi-tenant GitOps, the recommended practice is to use Argo CD Projects (AppProject) to manage access privileges and restrictions for the tenants. The AppProject enables the Argo CD administrator to restrict access to specific resources via allow- and denylisting, define team roles for role-based access control (RBAC), sync windows and more.
It's not uncommon in larger multi-tenant Argo CD instances to have many different projects for the various teams. Certain configuration items are often common across all projects or specific subsets of projects.
Rather than repeat this configuration over and over, you can define it once in a Global project instead. The other benefit is that changes to the global project will automatically be applied to the dependent projects, reducing the overall maintenance burden.
Here is an example of a Global project I use with my tenants in a cluster-scoped OpenShift GitOps instance. In this example, I denylisted all cluster-scoped resources along with specific namespace-scoped resources. The namespace-scoped resources remove some common ones, like ResourceQuota
and LimitRange
. It also removes some resources the OpenShift GitOps operator provides by default in cluster-scoped instances. It looks like this:
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: global
spec:
description: Global Configuration
clusterResourceBlacklist:
- group: '*'
kind: '*'
namespaceResourceBlacklist:
- group: ''
kind: Namespace
- group: ''
kind: ResourceQuota
- group: ''
kind: LimitRange
- group: operators.coreos.com
kind: '*'
- group: operator.openshift.io
kind: '*'
- group: storage.k8s.io
kind: '*'
- group: machine.openshift.io
kind: '*'
- group: machineconfiguration.openshift.io
kind: '*'
- group: compliance.openshift.io
kind: '*'
The OpenShift GitOps operator does not have direct support for configuring the match expression needed for other projects to reference it. However, you can add the configuration using the extraConfig
capability in the operator, as shown in this snippet:
apiVersion: argoproj.io/v1beta1
kind: ArgoCD
metadata:
name: argocd
spec:
extraConfig:
globalProjects: |-
- labelSelector:
matchExpressions:
- key: argocd.argoproj.io/project-inherit
operator: In
values:
- global
projectName: global
...
This Global project can then be referenced in other Projects by adding a label to it:
apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata:
name: product-catalog
labels:
argocd.argoproj.io/project-inherit: global
spec:
...
Wrap up
This blog went over several ways to improve your overall quality of life when working with OpenShift GitOps and Argo CD.
Sobre el autor
Más similar
Navegar por canal
Automatización
Las últimas novedades en la automatización de la TI para los equipos, la tecnología y los entornos
Inteligencia artificial
Descubra las actualizaciones en las plataformas que permiten a los clientes ejecutar cargas de trabajo de inteligecia artificial en cualquier lugar
Nube híbrida abierta
Vea como construimos un futuro flexible con la nube híbrida
Seguridad
Vea las últimas novedades sobre cómo reducimos los riesgos en entornos y tecnologías
Edge computing
Conozca las actualizaciones en las plataformas que simplifican las operaciones en el edge
Infraestructura
Vea las últimas novedades sobre la plataforma Linux empresarial líder en el mundo
Aplicaciones
Conozca nuestras soluciones para abordar los desafíos más complejos de las aplicaciones
Programas originales
Vea historias divertidas de creadores y líderes en tecnología empresarial
Productos
- Red Hat Enterprise Linux
- Red Hat OpenShift
- Red Hat Ansible Automation Platform
- Servicios de nube
- Ver todos los productos
Herramientas
- Training y Certificación
- Mi cuenta
- Soporte al cliente
- Recursos para desarrolladores
- Busque un partner
- Red Hat Ecosystem Catalog
- Calculador de valor Red Hat
- Documentación
Realice pruebas, compras y ventas
Comunicarse
- Comuníquese con la oficina de ventas
- Comuníquese con el servicio al cliente
- Comuníquese con Red Hat Training
- Redes sociales
Acerca de Red Hat
Somos el proveedor líder a nivel mundial de soluciones empresariales de código abierto, incluyendo Linux, cloud, contenedores y Kubernetes. Ofrecemos soluciones reforzadas, las cuales permiten que las empresas trabajen en distintas plataformas y entornos con facilidad, desde el centro de datos principal hasta el extremo de la red.
Seleccionar idioma
Red Hat legal and privacy links
- Acerca de Red Hat
- Oportunidades de empleo
- Eventos
- Sedes
- Póngase en contacto con Red Hat
- Blog de Red Hat
- Diversidad, igualdad e inclusión
- Cool Stuff Store
- Red Hat Summit