How we use a common CI/CD pipeline to deploy applications at scale

Continuous integration and continuous deployment (CI/CD) is a set of practices that enable development teams to merge code changes frequently into a central repository and confidently deploy them for users.

A critical component of the IBM CIO organization's hybrid cloud transformation is safely building and deploying applications at scale. Therefore, we're developing a common organization-wide CI/CD solution built on Red Hat's OpenShift Pipelines. This common pipeline approach allows our organization to build on previous work to scale best practices across the organization. But why are we doing this? I'll lay the foundation for why not just CI/CD but a common CI/CD solution is beneficial.

[ Learn 4 tips for implementing consistent configuration and automation standards. ]

What is CI/CD?

CI/CD is a two-step process that removes friction from application development and delivery using automation. CI is a practice where the development team frequently commits changes into the shared repository and then leverages automation to ensure the code works as expected. This happens by having each commit trigger a build and a series of automated tests to verify the application's behavior and the clean integration of updates.

This process enables the application team to deploy code frequently, which leads to continuous deployment. CD automates releasing an app to production. Because it is automated with no manual gates, CD relies heavily on well-designed test automation. Once the application passes all tests, a developer's change to an application can go live within minutes.

How CI/CD is implemented

CI/CD automates integrating, releasing, and deploying software while removing roadblocks. This is done with smaller code changes, continuous testing, real-time feedback, faster releases, improved customer satisfaction, and reduced costs.

• Smaller code changes: CI/CD encourages developers to integrate small pieces of code. Smaller code changes are more straightforward and easier to handle than trying to integrate a large amount of code. This is especially true when many developers work on the same code base.
• Continuous testing: Building on smaller changes, continuous testing allows these smaller pieces of code to be tested as soon as they are checked into the repository so that developers can see and correct problems quickly.
• Real-time feedback: CI/CD is a great way to get continuous feedback from end users and the developer team to improve team transparency and accountability. This real-time feedback increases visibility into problems with the team and encourages responsible accountability.
• Faster releases and improved customer satisfaction: CI/CD pipelines provide a means for consistent builds, tests, and deployments, enabling errors to be detected faster. This allows teams to safely release code more quickly and with new features and bug fixes. Development teams can meet users' needs through regular (even daily!) updates and responding to feedback with rapid, high-quality changes.
• Reduced costs: CI/CD pipelines help catch bugs before they reach production, significantly reducing the costs of addressing them. Deployment automation also reduces the chances of human error affecting deployment.

[ Get the ultimate CI/CD resource guide. ]

Build guide rails into processes

Many organizations do not know what components make up their systems or, in some cases, even what systems they are running. Enterprises need to build guide rails into their processes to help them gain better insight into their security positions, something a common CI/CD process can enable.

Enterprises and chief security officers worldwide have seen recent high-profile vulnerabilities as a wake-up call to rethink security. These new approaches include:

• Secrets and credentials detection
• Library inventories
• Usage approvals
• Vulnerability detection
• Code quality and automated test coverage
• Static Application Security Testing (SAST) vulnerability detection
• Dynamic Application Security Testing (DAST) vulnerability detection
• Container image vulnerability detection
• Artifact signing

These new activities may require significant compliance work for development teams. If you understand that CI/CD helps the team catch errors early and makes security part of the regular development process, CI/CD pipelines are a natural place to start. A common CI/CD makes it even easier. With a common CI/CD in place, you can add new security tools without affecting existing development processes and developers.

Our common CI/CD journey takes this approach. By adding security to our CI/CD process, we are introducing guide rails for our developers that help them focus on delivering business value while also protecting the enterprise. Our common approach:

• Implements our security policies, including scanning
• Reduces developer response time for issues
• Promotes application quality and consistency
• Offloads compliance and ongoing maintenance burdens

[ Download The path to GitOps to put the right tools, workflows, and structures in place to enable a complete GitOps workflow. ] 

In addition, on the CD side, we can provide:

• Production deployment approval auditing
• DAST processes
• Deployment frequency and duration metrics
• Deployment region location information

Economy of scale and friction reduction

Most development teams agree on the value of a CI/CD pipeline; however, they also may face barriers to adoption related to costs, skills, or time. As more requirements are placed on teams, these barriers can increase.

These barriers and ever-increasing compliance requirements create friction in the development process that can reduce developer productivity. From the organization's point of view, every team implementing its own CI/CD approach creates inefficiencies: Each team spends time building and maintaining the pipelines, procuring and managing the infrastructure, and integrating new tools. Combined, these increase the costs and delays in deploying features.

[ Learn more about how to set up a CI/CD pipeline. ]

A common CI/CD provides a way to address these problems. A single team is responsible for creating, maintaining, and standardizing the pipelines, supporting and managing the infrastructure, and integrating new tools. For our development teams, this means:

• Greater ability to focus on business value and not process and infrastructure
• Increased productivity and satisfaction
• Accelerated application modernization

This reduces the burden of CI/CD adoption for development teams and friction in the development process. Couple this with the ability to leverage a single infrastructure, and an economy of scale emerges.

How a common CI/CD approach helps generate insights

Moving to a common CI/CD approach helps our organization collect and analyze data from the pipelines. This enables greater insight into an application's source code to deployment from a standard set of tools pushing into a common data store. We've developed a data lake to bring the data together and a developer experience portal to provide a single pane of glass that makes data easy to understand.

For the IBM CIO organization, our approach allows us to:

• Better understand our open source usage
• Better understand application quality
• Gain insights into our application runtimes and languages
• Reduce response time when security issues arise

[ Check out Red Hat's Portfolio Architecture Center for a wide variety of reference architectures you can use. ]

Wrap up

CI/CD is a set of practices that enable teams to build and deploy applications safely. Introducing a common CI/CD solution provides a means to scale the benefits of CI/CD across an organization while reducing the overall adoption effort. With such an offering in place, we can add security compliance activities to the pipelines without overloading teams. This results in a safer, more knowledgeable organization and happier and more productive developers.

This originally appeared on Hybrid Cloud How-tos and is republished with permission.