This post is the extension of the previous post, which talks about how to automate SAP HANA Scale-Up System Replication deployments on Red Hat Enterprise Linux (RHEL). This time, we will further automate the process using RHEL System Roles for SAP and Image Builder, and introduce the Red Hat High Availability solutions for managing SAP HANA System Replication.
RHEL System Roles for SAP as part of RHEL for SAP Solutions is fully supported. It consists of three system roles, which are included in the RHEL for SAP Solutions repositories.
The system role sap-preconfigure configures a system for SAP to be installed. It includes adding packages, configuring DNS, configuring SELinux, and more. The system role sap-hana-preconfigure can apply changes to prepare the system prior to SAP HANA being installed. It is intended to be run after the SAP pre-configure role and includes elements such as setting the sap-hana tuned profile. And, system role sap-netweaver-preconfigure configures a system for SAP Netweaver to be installed. Includes validating the machine and configuring the sap-netweaver tuned profile.
RHEL High Availability
One of the key things customers look for when deploying SAP workloads on premise or public cloud is having high availability (HA) set up for their business/mission-critical SAP applications. Red Hat provides SAP certified high availability solutions for various SAP components. In this blog post, we discuss how to utilize the sap_hana_ha_pacemaker Ansible role to set up the Red Hat HA solution for managing SAP HANA Scale-Up System Replication on Azure.
The RHEL for SAP Solutions subscription includes the Red Hat Enterprise Linux High Availability Add-on (RHEL HA Add-On), which provides all the necessary packages for configuring a pacemaker-based cluster that provides reliability, scalability, and availability to critical production services. See the Automated SAP HANA System Replication in Scale-Up in pacemaker cluster and Deploying Highly Available SAP NetWeaver-based Servers Using Red Hat Enterprise Linux HA add-on with Pacemaker knowledge base articles for guidelines.
Image Builder
RHEL 8 ships a new tool, called Image Builder, that allows you to create custom RHEL system images in a variety of formats. These include compatibility with major cloud providers and virtualization technologies available in the market. As a result, it enables you to quickly spin up new Red Hat Enterprise Linux (RHEL) systems in different platforms, according to your requirements.
In the recent RHEL updates, os-build replaces lorax-composer has been added as the new back-end for Image Builder. This updated back-end will allow for additional features to be delivered, the first of which is Push to Cloud (see the below picture). Additional new features will be added to os-build, such as building RHEL versions different from the local system or different architectures, which is expected in future RHEL releases.
In this blog post, we will discuss how to set up a bootstrap server for SAP HANA deployment using Image Builder and libvirt tool.
Automate your SAP HANA deployment
In the following demo, we will walk you through how to build a bootstrap server image for SAP HANA with Image Builder and libvirt tools, then discuss how to run the image on Azure Cloud, and finally discuss using the bootstrap server to set up SAP HANA Scale-Up System Replication on Azure Cloud.
![SAP HANA Scale-Up fig 2 VM Availability Set SAP HANA Scale-Up fig 2 VM Availability Set](/rhdc/managed-files/styles/wysiwyg_full_width/private/2%20VM%20Availability%20Set.png.webp?itok=j9yE3r0f)
![SAP HANA Scale-Up fig 3 Set Resource Group SAP HANA Scale-Up fig 3 Set Resource Group](/rhdc/managed-files/styles/wysiwyg_full_width/private/3%20Resource%20Group.png.webp?itok=XYZM75rN)
Customize OS image with Image Builder and libvirt
First, let’s set up Image Builder. You can refer to the section “Installing Image Builder” from the Red Hat Enterprise Linux 8 Image Builder: Building custom system images article. Once it is ready, you can build an OS image using Image Builder. And, you will first need to enable the required repositories on the server where the image will be created. Because you can choose the packages which are available on the server. To do that, run the following command lines.
Here are the repositories for RHEL for SAP Solutions:
# subscription-manager repos \ --enable="rhel-8-for-$(uname -m)-baseos-e4s-rpms" \ --enable="rhel-8-for-$(uname -m)-appstream-e4s-rpms" \ --enable="rhel-8-for-$(uname -m)-sap-solutions-e4s-rpms" \ --enable="rhel-8-for-$(uname -m)-sap-netweaver-e4s-rpms" \ --enable="rhel-8-for-$(uname -m)-highavailability-e4s-rpms"
# subscription-manager repos \ --enable=ansible-2.8-for-rhel-8-$(uname -m)-rpms
Set up repositories for Azure CLI:
# rpm --import https://packages.microsoft.com/keys/microsoft.asc # sh -c 'echo -e "[azure-cli]\nname=Azure CLI\nbaseurl=https://packages.microsoft.com/yumrepos/azure-cli\nenabled=1\ngpgcheck=1\ngpgkey=https://packages.microsoft.com/keys/microsoft.asc" > /etc/yum.repos.d/azure-cli.repo'
Once the repositories are set, log in to the Web Console of the server, and look for Image Builder on the left sidebar, which will show you all the blueprints. Create your own blueprint by clicking the “Create Blueprint” button on the top right corner, and you will see the following picture.
After inputting the Name and Description, click "Create," which will take you to the Edit Packages page. You can pick and choose which packages you want to include in your OS images. To build a bootstrap server for SAP HANA, you will need the following packages at least:
-
ansible
-
azure-cli
-
gcc
-
libffi-devel
-
platform-python-devel
-
openssl-devel
-
python3-pip
-
python3-wheel
-
rhel-system-roles-sap
-
git
-
nfs-utils
Add the above packages to the blueprint, and click "Commit" twice to save the changes to your blueprint:
![SAP HANA Scale-up fig 5 Bootstrap saphana SAP HANA Scale-up fig 5 Bootstrap saphana](/rhdc/managed-files/styles/wysiwyg_full_width/private/5%20Bootstrap%20saphana.png.webp?itok=EORjdl8O)
![SAP HANA Scale-up fig 6 bootstrap saphana customization SAP HANA Scale-up fig 6 bootstrap saphana customization](/rhdc/managed-files/styles/wysiwyg_full_width/private/6%20bootstrap%20saphana%20customization.png.webp?itok=g2E2gPzu)
![SAP HANA Scale-up fig 7 Azure Disk Image SAP HANA Scale-up fig 7 Azure Disk Image](/rhdc/managed-files/styles/wysiwyg_full_width/private/7%20Azure%20Disk%20Image.png.webp?itok=cy9qFS7P)
![SAP HANA Scale-up fig 8 image build in process SAP HANA Scale-up fig 8 image build in process](/rhdc/managed-files/styles/wysiwyg_full_width/private/8%20image%20build%20in%20process.png.webp?itok=nR-BlHJF)
![SAP HANA Scale-up fig 9 download image SAP HANA Scale-up fig 9 download image](/rhdc/managed-files/styles/wysiwyg_full_width/private/9%20download%20image.png.webp?itok=iuqWZXD8)
Customize the image with libvirt tools
Before we start this section, you will need to set up libvirt tools by installing the packages:
# yum install qemu-kvm libvirt libvirt-python libguestfs-tools virt-install
To automate the setup of the bootstrap server, we would like to have Ansible roles and modules and playbooks to be installed and downloaded when the image is up and running, so here we use the first boot script (i.e., run_on_bootstrap.sh
).
#!/usr/bin/bash # Download the required yml files and files git clone https://github.com/EdwardJ1n/sap-hana-hsr-with-ha-on-azure-example.git >> /root/first_run.sh.log # Install the required packages for the Azure Python SDK modules. pip3 install --upgrade pip >> /root/first_run.sh.log pip install 'ansible[azure]' --use-feature=2020-resolver >> /root/first_run.sh.log # Download Ansible roles for SAP HANA deployments ansible-galaxy install -r /root/saphana_deployment/roles/requirements.yml >> /root/first_run.sh.log
Use libvirt tool to embed your first boot script using option --firstboot:
# virt-customize -a /<DIR>/bootstrap.vhd \ --firstboot /root/run_on_bootstrap.sh \ --root-password "password:redhat" \ --hostname bootstrap \ --ssh-inject root:file:./.ssh/id_rsa.pub \ --selinux-relabel
With the above command, you also set up the ssh key to perform passwordless login.
Now, you have a bootstrap server image for SAP HANA and can upload the image to Azure. On Azure WebUI, you will first need to create a storage account (e.g., storageaccounthana) and a container (images) within the storage account. Change the access level of the container as follows:
And then, set up the Shared access signature:
Note: the steps shown in the above two pictures are needed for the playbook deploy_bootstrap.yml mentioned in the next section.
After that, generate a SAS and connection string by clicking the button “Generate SAS and connection string”:
Upload the image using the
az storage blob upload
command. It can take a hour or more to complete depending on your network bandwidth to Azure:
# sas='<SAS token>' # az storage blob upload --account-name storageaccounthana \ --container-name images --name bootstrap.vhd \ -f /ssd_vms/bootstrap.vhd --sas-token $sas
--- - hosts: localhost connection: local vars: resource_group: ResourceGroupHana storage_account: storageaccounthana storage_container: images vm_image: bootstrapImage image_name: bootstrap.vhd virtual_network: ResourceGroupHanaVnet virtual_subnet: ResourceGroupHanaSubnet virtual_machine: bootstrap public_ip: bootstrapPublicIP nic: bootstrapVMNic security_group: bootstrapSecgroup image_URI: "https://{{ storage_account }}.blob.core.windows.net/{{ storage_container }}/{{ image_name }}" tasks: - name: Create a resource group azure_rm_resourcegroup: name: "{{ resource_group }}" location: eastus - name: Create an Azure Image from the image uploaded azure_rm_image: resource_group: "{{ resource_group }}" name: "{{ vm_image }}" source: "{{ image_URI }}" os_type: Linux - name: Create a virtual network azure_rm_virtualnetwork: resource_group: "{{ resource_group }}" name: "{{ virtual_network }}" address_prefixes: "10.0.0.0/16" - name: Create a subnet azure_rm_subnet: resource_group: "{{ resource_group }}" name: "{{ virtual_subnet }}" address_prefix: "10.0.1.0/24" virtual_network: "{{ virtual_network }}" - name: Create a public ip azure_rm_publicipaddress: resource_group: "{{ resource_group }}" allocation_method: Static name: "{{ public_ip }}" - name: Create a security group that allows HTTP azure_rm_securitygroup: resource_group: "{{ resource_group }}" name: "{{ security_group }}" rules: - name: 'AllowSSH' protocol: Tcp destination_port_range: 22 access: Allow priority: 101 direction: Inbound - name: Create a network interface azure_rm_networkinterface: resource_group: "{{ resource_group }}" name: "{{ nic }}" virtual_network: "{{ virtual_network }}" subnet: "{{ virtual_subnet }}" security_group: "{{ security_group }}" ip_configurations: - name: ipconfig1 public_ip_address_name: "{{ public_ip }}" primary: True - name: Create a bootstrap server azure_rm_virtualmachine: resource_group: "{{ resource_group }}" name: "{{ virtual_machine }}" vm_size: Standard_DS1_v2 admin_username: adminUser admin_password: "adminUser#1" ssh_public_keys: - path: /home/adminUser/.ssh/authorized_keys key_data: <Insert publish ssh key> network_interfaces: "{{ nic }}" image: name: "{{ vm_image }}" resource_group: "{{ resource_group }}" managed_disk_type: Standard_LRS data_disks: - lun: 0 disk_size_gb: 40 managed_disk_type: Standard_LRS register: result - name: Show the information of the VM debug: var=result ...
Deploy SAP HANA Scale-Up System Replication
Now let’s take a look at the playbooks to deploy SAP HANA Scale-Up System Replication. Yaml file site.yml is the main playbook, which consists of four playbooks respectively to set up Azure credentials (setup_azure_credentials.yml), and provision Azure VMs as SAP HANA nodes (provision_azure_vms.yml) with the credentials, and then set up the Azure VMs (configure_azure_vms.yml) before SAP HANA deployment, finally deploy SAP HANA Scale-Up System Replication (deploy_sap_hana_hsr.yml).
![SAP HANA Scale-up fig 13 yml playbooks SAP HANA Scale-up fig 13 yml playbooks](/rhdc/managed-files/styles/wysiwyg_full_width/private/13%20yml%20playbooks.png.webp?itok=gtgIP5Lh)
Next, you will need to run setup_azure_credentials.yml to set up the authentication with Azure for Azure Resource Manager modules:
- name: Set up Azure credentials hosts: localhost command: > connection: local tasks: - name: Include Azure credentials include_vars: files/azure_cred.yml - name: Log into Azure az login -u "{{ azure_account }}" -p "{{ azure_secret }}" - name: Create a service principal command: > az ad sp create-for-rbac --name ServicePrincipalHana register: ad_info - set_fact: ad_vars: "{{ ad_info.stdout | from_json }}" - name: Get Azure account information command: > az account show register: account_info - set_fact: account_vars: "{{ account_info.stdout | from_json }}" - name: Insert Azure credentials configuration in /root/.azure/credentials blockinfile: path: /root/.azure/credentials create: yes block: | [default] subscription_id={{ account_vars.id }} client_id={{ ad_vars.appId }} secret={{ ad_vars.password }} tenant={{ ad_vars.tenant }} - name: Verify the Azure credentials by listing resource groups azure_rm_resourcegroup_facts: register: output - debug: var=output
# cat << EOF > ~/.azure_vault <vault password> EOF # ansible-vault encrypt_string --vault-id dev@~/.azure_vault '<string to encrypt>' --name 'azure_secret'
azure_rm_availabilityset
is added to create Azure Availability sets, which provide VM redundancy and availability.--- - hosts: localhost connection: local vars: resource_group: ResourceGroupHana virtual_network: ResourceGroupHanaVnet virtual_subnet: ResourceGroupHanaSubnet virtual_machine1: saphana01 virtual_machine2: saphana02 nic1: NicHana01 nic2: NicHana02 security_group: SecurityGroupHana location: eastus avaiability_set: AvailabilitySetHana tasks: - name: Ensure a security group that allows SSH azure_rm_securitygroup: resource_group: "{{ resource_group }}" name: "{{ security_group }}" rules: - name: 'AllowSSH' protocol: Tcp destination_port_range: 22 access: Allow priority: 101 direction: Inbound - name: Ensure an availability set with advanced options azure_rm_availabilityset: name: "{{ avaiability_set }}" location: "{{ location }}" resource_group: "{{ resource_group }}" platform_update_domain_count: 5 platform_fault_domain_count: 3 sku: Aligned - name: Ensure a network interface for VM saphana01 azure_rm_networkinterface: resource_group: "{{ resource_group }}" name: "{{ nic1 }}" virtual_network: "{{ virtual_network }}" subnet_name: "{{ virtual_subnet }}" security_group: "{{ security_group }}" ip_configurations: - name: ipconfig1 primary: True - name: Ensure a VM saphana01 to run SAP HANA azure_rm_virtualmachine: resource_group: "{{ resource_group }}" name: "{{ virtual_machine1 }}" vm_size: Standard_E8s_v3 admin_username: adminUser admin_password: "adminUser#1" availability_set: "{{ avaiability_set }}" ssh_public_keys: - path: /home/adminUser/.ssh/authorized_keys key_data: <Insert public ssh key of bootstrap server> network_interfaces: "{{ nic1 }}" image: offer: "RHEL" publisher: "RedHat" sku: "8.1" version: "8.1.2020082711" managed_disk_type: Standard_LRS data_disks: - lun: 0 disk_size_gb: 128 managed_disk_type: Premium_LRS - name: Ensure a network interface for VM saphana02 azure_rm_networkinterface: resource_group: "{{ resource_group }}" name: "{{ nic2 }}" virtual_network: "{{ virtual_network }}" subnet_name: "{{ virtual_subnet }}" security_group: "{{ security_group }}" ip_configurations: - name: ipconfig2 primary: True - name: Ensure a VM saphana02 to run SAP HANA azure_rm_virtualmachine: resource_group: "{{ resource_group }}" name: "{{ virtual_machine2 }}" vm_size: Standard_E8s_v3 admin_username: adminUser admin_password: "adminUser#1" availability_set: "{{ avaiability_set }}" ssh_public_keys: - path: /home/adminUser/.ssh/authorized_keys key_data: <Insert public ssh key of bootstrap server> network_interfaces: "{{ nic2 }}" image: offer: "RHEL" publisher: "RedHat" sku: "8.1" version: "8.1.2020082711" managed_disk_type: Standard_LRS data_disks: - lun: 0 disk_size_gb: 128 managed_disk_type: Premium_LRS ...
--- - hosts: localhost connection: local tasks: - name: Get facts for saphana01's network interface azure_rm_networkinterface_facts: resource_group: ResourceGroupHana name: NicHana01 register: result - set_fact: saphana01_ipaddr: "{{ result.networkinterfaces.0.ip_configurations.0.private_ip_address }}" - name: Get facts for saphana02's network interface azure_rm_networkinterface_facts: resource_group: ResourceGroupHana name: NicHana02 register: result - set_fact: saphana02_ipaddr: "{{ result.networkinterfaces.0.ip_configurations.0.private_ip_address }}" - name: Setup /etc/hosts blockinfile: path: /etc/hosts block: | {{ saphana01_ipaddr }} saphana01 saphana01.tamlab.com {{ saphana02_ipaddr }} saphana02 saphana02.tamlab.com - hosts: hana tasks: - name: Ensure that copy local /etc/hosts to saphana01 and saphana02 copy: src: /etc/hosts dest: /etc/hosts owner: root group: root mode: '0644' backup: yes - name: Disable RHUI repos by renaming repo file /etc/yum.repos.d/rh-cloud.repo command: mv /etc/yum.repos.d/rh-cloud.repo /etc/yum.repos.d/rh-cloud.repo.bk ignore_errors: yes - name: Extend root filesystem "/" of VMs to 10GB, the default to 2GB lvol: vg: rootvg lv: rootlv size: 10240 resizefs: yes - name: Resize root filesystem "/" command: > xfs_growfs / ...
--- - hosts: hana roles: - { role: redhat_sap.sap_rhsm } - { role: redhat_sap.sap_hana_mediacheck } - { role: redhat_sap.sap_hostagent } - { role: linux-system-roles.timesync } - { role: linux-system-roles.storage } - { role: sap-preconfigure } - { role: sap-hana-preconfigure } - { role: redhat_sap.sap_hana_deployment } - { role: redhat_sap.sap_hana_hsr } - { role: redhat_sap.sap_hana_ha_pacemaker } ...
# Variables required for 'sap_hana_ha_pacemaker' role sap_hana_ha_pacemaker_hana_sid: RHT sap_hana_ha_pacemaker_hana_instance_number: "01" sap_hana_ha_pacemaker_vip: 10.0.1.100 sap_hana_ha_pacemaker_hacluster_password: "MyPassw0rd!" sap_hana_ha_pacemaker_node1_fqdn: saphana01.tamlab.com sap_hana_ha_pacemaker_node2_fqdn: saphana02.tamlab.com sap_hana_ha_pacemaker_node1_ip: 10.0.1.6 sap_hana_ha_pacemaker_node2_ip: 10.0.1.7
# ansible-playbook --vault-password-file ~/.azure_vault site.yml
Note: While running the playbook make sure that your console will not be disconnected due to SSH connection timeout. Otherwise, you may need to clean up the VMs and start over again.
Here are some screenshots while the playbook is running.
Once the playbook is complete, you will see the collected HA cluster information stored in the form of node attributes that can be viewed by running the command
pcs status --full
:
Congratulations! Your SAP HANA Scale-Up System Replication with a High Availability cluster is up and running. Now, you can check out the Red Hat Insights application-focuses view for SAP HANA deployments, which provides automatic discovery, health and security assessment for SAP HANA on RHEL.
![sap hana scale-up fig 18 insights sap hana scale-up fig 18 insights](/rhdc/managed-files/styles/wysiwyg_full_width/private/18%20insights.png.webp?itok=TYcrfQob)
Summary
As you can see System Roles for SAP and Ansible modules can speed up your SAP HANA deployment along with Image Builder. These help to manage complex processes in the automated way, which is a perfect solution to use in infrastructure workflows. You can easily integrate them with other building tools for continuous integration and deployment of systems to work efficiently and enhance your productivity.
关于作者
Edward Jin has been working in IT for more than 12 years. He has a strong service mindset and design thinking and insights to IT systems and processes within the pharmaceutical , manufacturing and finance industry. Currently, he is helping customers on their digital journey using open source technology.
产品
工具
试用购买与出售
沟通
关于红帽
我们是世界领先的企业开源解决方案供应商,提供包括 Linux、云、容器和 Kubernetes。我们致力于提供经过安全强化的解决方案,从核心数据中心到网络边缘,让企业能够更轻松地跨平台和环境运营。