State of platform engineering in the age of AI

Platform engineering has emerged in response to the increasing complexity of software development due to the growing amount of choices and the need for streamlined, efficient processes. In its early stages, platform engineering focused on building robust infrastructure and automating repetitive tasks to reduce the manual overhead for development teams. These efforts were aimed at enhancing operational efficiency, ensuring that the entire team could concentrate more on delivering the solution and less on managing infrastructure.

From its inception, however, platform engineering has grown alongside DevOps, shaping the modern software development landscape. Platform engineering focuses on building and maintaining a platform that streamlines development and deployment processes, including responsibilities like creating internal developer platforms (IDPs), enhancing the developer experience (DevEx), and ensuring security and compliance. This evolution has been advanced by the rise of DevOps practices, which emphasize collaboration and automation, leading to a more integrated approach. Platform engineering and DevOps both contribute to achieving scalability, efficiency, resiliency, flexibility, and security, especially in a cloud-native and multicloud environment with an ever-expanding range of technology choices.

As the landscape continues to develop, few factors will ultimately advance platform engineering—and indeed technology—more profoundly than artificial intelligence (AI). Relying on deep learning models trained on large data sets, common use cases include software code assistance, customer personalization, scientific research, and much more. Not surprisingly, organizations large and small are allocating significant resources and budget to infuse new and existing applications with these capabilities to increase productivity or drive revenue and differentiation. The impact on platform engineering teams will be wide ranging, with many organizations already altering tools, processes, and overall charters to allow developers, data scientists, and other teams to take advantage of this generational paradigm shift in technology.

This detail provides a comprehensive review of the State of Platform Engineering in the Age of AI report, conducted by Illuminas, examining its benefits, challenges, and future trends. It explores how organizations are adopting and implementing platform engineering practices, the key performance indicators (KPIs) they track, and the level of success they are achieving. The report also outlines the integration of cloud-native technologies, automation tools, and AI solutions, and their alignment with platform engineering principles.

Platform engineers (e.g., platform teams) or those who support this function play a role in shaping the future of software development within organizations. This report aims to provide an informative perspective, offering insights into the developing landscape of the discipline.

By examining trends, challenges, and best practices from industry practitioners, this report presents data to help inform the selection of a platform engineering strategy while highlighting potential challenges.

The report focuses on the following key objectives:

1. Assess perceptions and adoption of platform engineering: By evaluating the understanding and implementation of platform engineering across various organizations, the research aims to provide a clear view of current adoption levels and the factors contributing to its growth.
2. Understand top priorities for developer effectiveness and efficiency: The research explores key priorities for companies seeking to improve developer productivity and streamline workflows, examining strategies and investments in this area.

For this report, Red Hat sponsored a survey of 1,000 platform engineers and IT decision makers (ITDMs) in the United States (U.S.), the United Kingdom (U.K.), and the English-speaking Asia Pacific region (APAC) to gain insights into the evolution of platform engineering and better understand its overall adoption and organizational perceptions. Data was gathered through 20-minute online surveys with respondents sourced from 3rd-party databases. The survey was conducted in September and October 2024.

To ensure a comprehensive view of the platform engineering landscape, this study gathered insights from a diverse pool of individuals, equally representing both IT decision makers and platform engineers. This balance allowed for the capture of perspectives from those leading platform engineering initiatives, and those directly involved in their implementation and daily operations.

The companies represented in this research spanned a range of sizes, with 35% falling into the medium-sized enterprise category and 65% categorized as large enterprises. The size diversity ensures that the findings reflect the experiences and challenges of organizations at different scales, from those with leaner development teams to those with extensive and complex development organizations.

To further enrich the data, the surveys targeted representation across various industry sectors. This included professionals from software development, finance, retail, healthcare, and professional services, among others. This cross-industry representation ensures that the insights and trends identified in this report are broadly applicable and relevant to a wide range of organizations undergoing digital transformation and seeking to optimize their software development processes.

The survey reveals a correlation between platform engineering maturity and organizational success, highlighting the importance of dedicated teams and strategic investment in areas like infrastructure and security. The report also examines the growing role of generative AI in platform engineering and its potential to transform software development practices. Additionally, it explores common challenges faced by organizations adopting platform engineering, such as workflow integration, security risks, and skills gaps. By examining these findings, the report aims to provide organizations with valuable insights to navigate the evolving landscape of platform engineering and optimize their software development lifecycle.

Key findings include:

• 41% of organizations with mature platform engineering practices invest more in areas like developer productivity tools (61%) and track more KPIs (7 on average), ultimately achieving greater success (41% reporting significantly higher success).
• The reasons for adopting platform engineering are enhanced security (48%) and improved collaboration (44%).
• Investment in platform engineering expands as organizations mature, with 52% starting with infrastructure modernization and 59% consistently prioritizing security.
• Workflow integration (37%), security risks (37%), skills gaps (34%), and budget constraints (33%) are common challenges, even for advanced organizations facing tool incompatibility (28%) and platform instability (26%).
• Security concerns (48%), collaboration needs (44%), and addressing operational bottlenecks (39%) are key motivators for platform engineering adoption.
• Advanced organizations track more success metrics (7 on average) focused on productivity (45%), security (41%), and performance (40%), while early-stage organizations prioritize cost (35%).
• Early success with platform engineering, reported by 22% of organizations as extremely successful and 72% as moderately/very successful, drives further investment and deeper engagement.
• Dedicated platform engineering teams, present in 62% of organizations, demonstrate a strong commitment to this approach.
• Generative AI is considered strategically important for platform engineering, with 45% viewing it as a core component of their strategy.
• Generative AI is widely used (76%) for various purposes, including documentation (76%), code generation (74%), and intelligent code suggestions (59%), transforming software development.

Finding 1: The effect of platform engineering maturity

Experience and maturity truly matter. 41% of organizations with more mature platform engineering practices invest more, track more key performance indicators (KPIs), and achieve greater success than those earlier in their journey. (p. 11)

"The Platform Engineering Maturity Index is a model developed for this survey to assess the maturity of an organization's platform engineering practices. It's based on an analysis of survey responses to a series of questions designed to interpret an organization's progress in adopting and implementing platform engineering.

The index categorizes organizations into 4 distinct maturity levels: Exploring, Emerging, Established, and Advanced. Each level represents a different stage in the platform engineering journey, from initial exploration and implementation to advanced integration and optimization. This model helps to provide a structured understanding of the different stages of platform engineering maturity and enables the identification of key trends, challenges, and opportunities associated with each stage."¹
 
More mature organizations see productivity improvements, with 54% of advanced organizations have also seen an increase in innovation compared to those who are just starting their platform engineering journey. (p. 19)

This difference in outcomes highlights the importance of a continued commitment to platform engineering, showcasing that just building a foundation is not enough and ongoing optimization leads to improved results and a more effective transformation of the software development lifecycle. 

Finding 2: Security and collaboration are cornerstones for success

Security and collaboration are the cornerstones of successful platform engineering initiatives. Organizations are increasingly drawn to platform engineering because of its ability to enhance security (48%) and ease collaboration (44%) within development teams. (p. 16) 

By providing a security-focused and standardized platform for development, organizations can mitigate risks while empowering developers to work together. This focus on security and collaboration not only improves the efficiency of the development process but also contributes to a compliant end-product. 

Especially advanced organizations see a greater success in both increased developer productivity (52%) and improved security and compliance (46%). (p. 19)

Finding 3: Investment focus—From automation to compliance

While infrastructure modernization takes center stage for those just starting (55%) (p. 13), as organizations progress, they broaden their investments especially in advanced automation (85%). (p. 13)

Closely followed by investments into security and compliance (59%) (p.14) and developer productivity tools (55%) (p. 14). Demonstrating a more complete approach to platform engineering that propels efficiency, security, and innovation.

Finding 4: Hidden obstacles in platform engineering adoption

The path to platform engineering success is not without challenges. Integrating platform engineering into existing workflows and ensuring robust security are the 2 most commonly named hurdles for organizations (both at 37%) (p.15). Skills gaps and budget constraints especially for those in the early stages of adoption are at 40% (p.15). However, these challenges do not simply go away as organizations gain experience. One out of three (roughly 30%) (p.15) advanced organizations often struggle with tool incompatibility, platform instability, and a persistent lack of knowledge. 

This underscores the importance of continuous investment in training, support, and the careful selection of cohesive tools that are known to work effectively together. 

Finding 5: Platform engineering for operational excellence

Organizations are drawn to platform engineering for a broad variety of reasons. The desire to improve collaboration and streamline workflows plays a significant role, as companies try to break down isolated environments and foster more efficient teamwork (44%) (p. 16). Interestingly, 39% of early-stage organizations are also significantly motivated by operational bottlenecks, 36% by difficulty scaling, and 31% by the presence of knowledge silos. (pp. 16, 17)

The variety of reasons continues to amplify the need for a cohesive platform supporting all project members in reaching operational excellence and greater velocity. 

Finding 6: Success has many facets in platform engineering

Organizations with advanced platform engineering practices tend to track a wider range of metrics, averaging around 7 key performance indicators (KPIs). Their focus leans towards metrics that reflect productivity gains (45%), enhanced security (41%), improved application performance (40%), and increased developer and customer satisfaction (40%). 35% of organizations in the earlier stages of platform engineering tend to prioritize cost-related metrics. (p. 19)

This difference in focus highlights how the concept of success in platform engineering evolves as organizations mature and begin to recognize the broader benefits beyond simple cost reduction. 

Finding 7: Early success fosters deeper engagement

The growing adoption of platform engineering is driven by organizations quickly realizing its benefits with extremely successful implementations (22%), even in the early stages of their platform engineering journey. With more than two-thirds (72%) reporting very successful implementations. 

This early taste of success creates a positive feedback loop, encouraging teams to refine their strategies, optimize their platforms, and ultimately help scale best practices and share knowledge intrinsically. (p. 20)

Finding 8: Dedicated platform engineering teams

Over half of the organizations (62%) (p. 25) have a team fully dedicated to platform engineering, indicating a strong commitment to this approach. 

This approach suggests a shift towards greater specialization within development organizations, building upon the foundation of existing application platforms and DevOps teams. These dedicated platform teams can advance operational excellence by focusing on creating and refining internal developer platforms (IDPs), streamlining workflows with development teams, and fostering a more efficient and collaborative development environment.

Finding 9: Generative AI is strategically important

34% of organizations view generative AI (gen AI) as an important component of their platform engineering strategy, with nearly half (45%) considering it central. 

This emphasizes a growing understanding that AI capabilities are essential for propelling innovation, enhancing efficiency, and unlocking the full potential of platform engineering. And it is not only relevant for developer tools and automation approaches, but also for provisioning and safeguarding the runtimes for AI-infused applications. (p. 42) 

Finding 10: Empowering developers with diverse use cases for gen AI

Gen AI is used in platform engineering for a variety of use cases, including AI-generated documentation (76%), automated code generation (74%), intelligent code suggestions (59%), and more. (pp. 43, 47)

This widespread adoption highlights the versatility of gen AI and its ability to streamline various aspects of the software development lifecycle. By integrating gen AI into their workflows, organizations can boost developer productivity, improve code quality, and foster a more innovative development culture.

Links to resources about platform engineering

Overview: “What is platform engineering?”

Overview: “Red Hat OpenShift for platform engineers”

Video: “Platform engineering”

Blog: “What is platform engineering and why do we need it?”

E-book: Developer Portals: Prepare to perform with Red Hat Developer Hub

Get started with Red Hat Developer Hub: https://developers.redhat.com/rhdh/overview 

Empowering platform engineering success

Red Hat provides a robust ecosystem of tools and resources designed to support your organization's platform engineering journey and help you achieve your desired outcomes:

• Establish a solid foundation for your platform engineering initiatives with Red Hat® OpenShift®, a powerful Kubernetes based application platform that ensures consistency, scalability, and reliability.
• Enhance your automation capabilities with Red Hat Ansible® Automation Platform, allowing Infrastructure as Code (IaC) practices to streamline infrastructure provisioning, configuration management, and application deployment.
• Accelerate your software delivery lifecycle with Red Hat OpenShift Pipelines, a cloud-native CI/CD solution that integrates smoothly with GitOps principles for efficient and automated application deployments.
• Foster collaboration and knowledge sharing with Red Hat Developer Hub, a platform that provides developers with self-service access to tools, resources, and support.
• Deliver security-focused and compliant applications with Red Hat Trusted Application Pipeline, a solution that incorporates security best practices and compliance checks throughout the software development lifecycle.
• Empower your developers with Red Hat OpenShift Dev Spaces, providing them with preconfigured workspaces and on-demand access to the tools and resources they need for efficient and productive development.

By using Red Hat's comprehensive suite of solutions, your organization can confidently navigate the complexities of platform engineering, accelerate software development lifecycles, and advance innovation.