DenizBank empowers its data scientists

More than 100 data scientists working at DenizBank, a prominent private bank in Türkiye and the fifth largest in the country, wanted to convert its existing workflow into a less manual process with a more standardized approach. The bank’s IT subsidiary, Intertech, embarked on a project to provide a model development environment with automated pipelines and standards to improve productivity and time to market. One of the key improvements Intertech implemented was adopting Red Hat OpenShift AI for its self-service capabilities and capacity to scale model serving and improve operational efficiency. Data scientists can now focus on building models — models that are more robust and secure than ever.

Benefits: 

• Provided more than 120 data scientists, from different lines of business, with greater autonomy and more consistent standards
• Accelerated time-to-market while ensuring more robust and secure models
• Optimized GPU usage with slicing

As artificial intelligence (AI) and machine learning (ML) become part of our everyday lives, cutting-edge organizations have already embedded the technology deeply into their day-to-day operations. DenizBank, the fifth largest private bank in Türkiye, is one such financial institution. The bank has more than 120 data scientists spread across different business lines, spanning areas like risk management, marketing, and customer relations. In teams of 4 or 5, DenizBank’s highly skilled data scientists have developed more than 100 AI and ML models to date. “Our models primarily focus on predictions, such as what types of customers the bank should give credit to,” said Salih Eliguzel, Head of DevSecOps, Intertech – DenizBank’s technology company. “And, on fraud prevention, they help to identify anomalies that would be difficult to see otherwise, such as if a person suddenly takes a larger amount from an ATM than they normally would.”

Data scientists depend on IT experts at the bank’s technology company, Intertech, for the environment they need. However, although the environment was highly innovative, it was still reliant on complex processor-heavy and resource-intensive workstations that required setup for each individual model. The model development process is intricate and includes model creation, data preparation for model training, fine-tuning and validation, and uploading to the model registry from where it can be served (model inferencing).

“Such manual processes—from creating the model development environment on the workbench to preparing the data—significantly hindered the productivity of the data scientists, impeding their ability to focus on higher-value tasks. Additionally, they could not validate their models at the desired level,” said Eliguzel. “Environment variables were challenging. Model names, database integrations, code repositories, and versioning sites – there was no way to standardize or monitor any of these without using a special product.”

Lack of standardization resulted in inconsistent workbenches, with each model accessing data in its individual manner. It also meant that there was a lack of visibility into the complete environment of any given model. What’s more, data scientists were storing their code on their workbenches where nobody else could access it.

Intertech wanted to build a comprehensive, standardized, holistic solution for the data scientists that would improve time-to-market while delivering AI/ML process cost savings. The company specifically wanted to automate data science pipelines, provide self-service capabilities, scale model serving, and significantly increase operational efficiency.

DenizBank adopted Red Hat OpenShift more than 3 years ago, giving Intertech a deep familiarity with the platform and a clear understanding of the potential benefits it could offer.

Following a thorough evaluation of the adoption of Red Hat OpenShift AI, it became evident that this technology was the right solution for empowering its data scientists. “We chose Red Hat OpenShift AI for its ability to harness the power of Kubernetes, its scalability, and its ease of management,” said Eliguzel. “The declarative approach it enables—the ability to manage everything as code—is also really important for us.”

The Intertech team has a DevOps background and was already experienced in adopting a microservices approach in rearchitecting DenizBank’s applications – all of which they were able to bring to the platform designed specifically for data scientists. Red Hat Consulting helped the team design and architect the OpenShift AI solution at DenizBank, which consists of a total of 15 clusters deployed on bare metal, on premise.

“Red Hat Consulting listened to our vision for our microservices transformation and helped us adapt their technical solution to existing DevOps and GitOps best practices to suit our needs,” said Eliguzel. “This is one value advantage we admired in our partnership with them: discussing and adapting everything together was the best way to learn the best practices.”

Equipped with professional know-how, Intertech proceeded to define the processes and standards the data scientists would use for the platform: naming convention, how to create a workbench, computing resource capacity required, GPU usage requirements, and more. With these in place, Intertech then proceeded to implement GitOps adaptions.

“Thanks to Red Hat OpenShift AI, DenizBank data scientists now have access to a broader range of open source AI technologies,” said Taner Kılıç, Software Platforms and Architecture Group Lead, Intertech. “This will allow them to take full advantage of the power of the platform.”

OpenShift AI integrates the Kubeflow Notebook controller, model serving, and data science pipeline components to simplify AI/ML workflow deployment at scale. It provides pre-built or customized cluster images for building models using Jupyter Notebooks. And it tracks changes to Jupyter, TensorFlow, PyTorch, and other open source AI technologies.

Provided data scientists with greater autonomy and more consistent standards

OpenShift AI ensures data scientists have the autonomy they need to design and create models to build intelligence-driven applications using a strong set of standards. Gone is the chaos of having to reconfigure environments manually when moving from working on one model to another; they can spend more time building models.

“Adding Red Hat OpenShift AI to our arsenal has given our data scientists greater autonomy and better standards,” said Ömer Uyar, CTO, Intertech. 

When a data scientist begins the process of developing a new model, self-service capabilities guide them through building a tailored AI model development environment. They simply pick a suitable pre-built base image, or a custom image using their organization’s standard libraries and define their requirements. The platform then automatically creates everything for them.

“Red Hat OpenShift AI provides standardized templates and pre-built cluster images – all the libraries they need built in,” said Eliguzel. Those base images include GPU images for Python-specific models, for instance. Plug-ins allow for easy plug-and-play functionality when connecting a data source and more besides.

Accelerated time-to-market while ensuring more robust and secure models

 Intertech expects OpenShift AI to reduce the time to market for new models from around 1 week to just 10 minutes. “We have witnessed how automating the whole process of developing new microservices with OpenShift AI cuts time-to-market,” said Eliguzel. “We expect to achieve similar improvements for developing new AI/ML models as well.”

Besides automating environment builds, self-service capabilities allow data scientists to deploy their models automatically through the pipeline, eliminating potential delays in model deployment. While data scientists now have a great deal of autonomy, guardrails ensure they adhere to standards and regulations. “Data scientists can work independently and more efficiently,” said Eliguzel, “but we expect them to follow standards and best practices.”

Changing from manual to automated processes using best practice not only results in a faster timeto-market but also creates models that are more secure and more reliable. With code now stored in a central repository, code reviews can ensure models are fully validated. 

“As an invaluable AI-driven solution, Red Hat OpenShift AI provides a streamlined environment that enables our data scientists to build and deploy more robust and secure models,” said Okan Çetinkaya, CDO – CAO, DenizBank.

Moreover, underlying GitOps and best practices make possible the declarative approach Intertech sought. “You can destroy then rebuild an environment and have everything back up and running in next to no time,” said Eliguzel. “OpenShift GitOps provides us with the templates—Helm charts managed by ArgoCD—for rapid and consistent rebuilds.”

Optimized GPU usage with slicing

Integration with NVIDIA dashboards allow GPUs to be observed to optimize GPU usage for both training and serving. OpenShift AI then controls the size of the GPU for each model: “OpenShift AI helps us improve the optimization of our GPU usage,” said Eliguzel. “It automatically scales up the slices of GPU a model has access to, as needed.”

OpenShift AI takes advantage of NVIDIA’s Multi-Instance GPU (MIG) technology through the NVIDIA GPU Operator to partition GPUs into multiple isolated instances. Each instance has its own dedicated compute, memory, and bandwidth, allowing multiple pods across different projects to access GPU resources with guaranteed quality of service and fault isolation. This maximizes resource utilization, increases flexibility, and allows for more workloads to run simultaneously without the need for additional GPU hardware.

The main objectives for Intertech were bringing about more standardization and creating a self-service model development environment. Having established certain standards and pipeline automations, the focus is now squarely on migrating all data scientists and their models to OpenShift AI. Future priorities also include improving compute resource utilization (CPUs, GPUs, and memory) and model inferencing capabilities and times with OpenShift AI.

“Data scientists highlighted the challenges around the significant time required to set up new environments and train new models,” said Eliguzel. “Red Hat OpenShift provides full automation, advanced tools, and robust standards, specifically designed to enhance the productivity of data scientists. It is possible to build one’s applications and models side-by-side in Red Hat OpenShift AI.”