In a Kubernetes cluster, the control plane's stability, resiliency, and performance drive the overall cluster stability, resiliency, and scalability. Generally speaking, the various components of Kubernetes are stateless. For storing object or artifact definitions of a cluster, the etcd key-value store acts as the single source of truth and as the only stateful component. This distributed key-value store is both reliable and strongly consistent. The APIServer, part of the control plane components, is the sole Kubernetes component directly interacting with etcd. All other components use the APIServer to retrieve or set information on the etcd. For this reason, losing etcd would lead to a catastrophic failure of the cluster.
The OpenShift control plane is defined by three supervisor nodes hosting the Kubernetes control plane components, including the etcd. To balance Failure Tolerance and Write Performance, OpenShift uses pre-defined etcd profiles to set the ETCD_HEARTBEAT_INTERVAL and ETCD_ELECTION_TIMEOUT for etcd. These profiles are optimized for the specific type of underlying infrastructure (e.g., cloud provider, virtualization, baremetal, etc.). On infrastructures with high variability of network latency or storage latency, OpenShift automatically uses settings with higher timers.
Internally, the OpenShift etcd Operator has two validated profiles:
- Default profile: This profile applies the default etcd timers configuration
- Heartbeat Interval=100ms
- Election Timeout=1000ms
- Slow profile: This profile is optimized for infrastructure with high variability on network or storage latency
- Heartbeat Interval=500ms
- Election Timeout=2500ms
Until now, these profiles have been automatically set by the OpenShift etcd Operator.
In some cases, a cluster administrator may need to select a slower or faster profile. This could be necessary when working with servers that have traditional hard disk drives (HDDs), deploying on unstable or saturated networks, or deploying a cluster that spans multiple locations. While these situations may not be ideal for Kubernetes, certain country or industry regulations or limitations from legacy application dependencies may require OpenShift to operate under these conditions. With this in mind, OpenShift 4.14 introduces the selectable profiles for etcd with the option of “controlPlaneHardwareSpeed”. With this, a cluster administrator can change the etcd profile to better match their environment.
The valid values for the new “controlPlaneHardwareSpeed” attribute in the OpenShift etcd Operator CR are:
- "" (empty string): This is the default option. Maintains the traditional behavior of the openshift-etcd-operator which allows the system to decide which speed to use. Clusters upgraded to Openshift 4.14 will initially operate here.
- “Standard”: This profile applies the default etcd timers configuration:
- Heartbeat Interval=100ms
- Election Timeout=1000ms
- “Slower”: This applies an etcd timers configuration to operate in unfavorable storage and network conditions:
- Heartbeat Interval=500ms
- Election Timeout=2500ms
The cluster administrator should be cautious when selecting profiles for openshift-etcd-operator, as higher etcd timer values increase recovery time for control plane failures. Whenever possible, keep the default behavior.
# Change the value. Valid values are "", "Standard", "Slower"
# The default value is "" (empty string)
oc patch etcd/cluster --type=merge \
-p '{"spec": {"controlPlaneHardwareSpeed": "<VALUE>"}}'
# To set the Standard profile
oc patch etcd/cluster --type=merge \
-p '{"spec": {"controlPlaneHardwareSpeed": "Standard"}}'
# To set the Slower profile
oc patch etcd/cluster --type=merge \
-p '{"spec": {"controlPlaneHardwareSpeed": "Slower"}}'
# To return to default behavior
oc patch etcd/cluster --type=merge \
-p '{"spec": {"controlPlaneHardwareSpeed": ""}}'
The changes will be detected by the OpenShift etcd operator environment variable controller triggering a rolling out update of an updated etcd. Note: This action may cause a temporary service disruption during the rollout process for the new etcd instance.
The ability to choose the etcd timers profile to use has been a capability requested by customers deploying OpenShift on unfavorable storage or networking conditions. This is what the new “controlPlaneHardwareSpeed” configuration option for the OpenShift etcd operator brings for the cluster administrators. If the organization has a use case that can benefit from it, try it in your non-production environments first, and let us know how it works for you.
執筆者紹介
William is a seasoned professional with 25 years of experience enabling Telco business transformation through emerging technologies. He works with Telco and MSO partners and customers at the forefront of digital disruption on architecting solutions that transform markets.
チャンネル別に見る
自動化
テクノロジー、チームおよび環境に関する IT 自動化の最新情報
AI (人工知能)
お客様が AI ワークロードをどこでも自由に実行することを可能にするプラットフォームについてのアップデート
オープン・ハイブリッドクラウド
ハイブリッドクラウドで柔軟に未来を築く方法をご確認ください。
セキュリティ
環境やテクノロジー全体に及ぶリスクを軽減する方法に関する最新情報
エッジコンピューティング
エッジでの運用を単純化するプラットフォームのアップデート
インフラストラクチャ
世界有数のエンタープライズ向け Linux プラットフォームの最新情報
アプリケーション
アプリケーションの最も困難な課題に対する Red Hat ソリューションの詳細
オリジナル番組
エンタープライズ向けテクノロジーのメーカーやリーダーによるストーリー
製品
ツール
試用、購入、販売
コミュニケーション
Red Hat について
エンタープライズ・オープンソース・ソリューションのプロバイダーとして世界をリードする Red Hat は、Linux、クラウド、コンテナ、Kubernetes などのテクノロジーを提供しています。Red Hat は強化されたソリューションを提供し、コアデータセンターからネットワークエッジまで、企業が複数のプラットフォームおよび環境間で容易に運用できるようにしています。
言語を選択してください
Red Hat legal and privacy links
- Red Hat について
- 採用情報
- イベント
- 各国のオフィス
- Red Hat へのお問い合わせ
- Red Hat ブログ
- ダイバーシティ、エクイティ、およびインクルージョン
- Cool Stuff Store
- Red Hat Summit