Red Hat Insights is a managed service included in every Red Hat subscription. It continuously analyzes platforms and applications to help enterprises manage hybrid cloud environments. It also triggers events through its Notifications service. Each account configures how and who can receive these events, with the ability to perform actions.
A common example is to forward new recommendations found on Red Hat Enterprise Linux (RHEL) system configuration to specific teams by email and/or create a new ticket in an IT Service Management (ITSM) solution. Through its Integrations service, Insights provides endpoint integrations to Splunk, ServiceNow, Slack and any third-party application with its generic webhooks integration (HTTP POST requests with optional token authentication). Once configured, these integrations can forward Insights events to third-party applications and include Insights findings in the organization's operational workflow.
PagerDuty is a software-as-a-service (SaaS) incident response platform offering management of operations. The platform collects signals from different sources and applies analytics and intelligence to take appropriate actions. These include routing requests to correct teams, running automation, or forwarding to third-party applications.
This article covers configuring integration between Insights and PagerDuty. This consists of exposing an endpoint on PagerDuty that accepts JSON-formatted events from Insights. We also provide a Custom Event Transformer (CET) for parsing incoming events and triggering PagerDuty alerts and incidents.
PagerDuty's Services and Integrations functionality
PagerDuty offers an Event Integration feature and associated events API to handle incoming events from third-party applications. Received data can be turned into alerts and actionable incidents in PagerDuty. This feature is commonly used to integrate infrastructure monitoring tools (e.g., Nagios, SignalFX, Datadog), application performance monitoring tools (e.g., New Relic, AppDynamics) and external site checks (e.g., Pingdom, Wormly). In the rest of this article, we use this feature to handle incoming events received from Insights.
Configuring an event integration in PagerDuty is straightforward and documented. It consists of creating a new service and adding an events integration. In this example, select and add a Custom Event Transformer (CET) integration. This allows for inspecting incoming events and extracting relevant data (using an editable JavaScript script) for the alert and incident creation. Details and documentation about this functionality can be found on the PagerDuty Developer Platform.
![Screenshot Red Hat Insights Custom Event Transformer](/rhdc/managed-files/styles/wysiwyg_full_width/private/insights-integration-pagerduty-img1.png?itok=m96hH2_F)
The new CET integration provides an endpoint URL that can be used by third-party applications to post events. The URL respects the following format, where <integration key> is a unique identifier for your integration:
https://events.pagerduty.com/integration/<integration key>/enqueue
This endpoint URL must be conserved preciously as you'll use it in Insights when configuring Integrations & Notifications (see following section).
The following is an example of the JavaScript script for the CET. The source code is available on GitHub on the redhat-insights-pagerduty repository. Its basic functionality consists of extracting common fields from the JSON formatted Insights events and looping through the data to deduplicate events received from Insights.
var event = JSON.parse(PD.inputRequest.rawBody); var contexts = [{ "bundle": event.bundle, "application": event.application, "event_type": event.event_type, "org_id": event.org_id, "hostname": event.context.hostname, "inventory_id": event.context.inventory_id }]; // Filtering incoming events (optional) if (event.bundle != "rhel") { PD.fail("Event ignored: (" + event.bundle + ") " + event.application + "/" + event.event_type) } var application = event.source && event.source.application && event.source.application.display_name ? event.source.application.display_name : event.application; var event_type = event.source && event.source.event_type && event.source.event_type.display_name ? event.source.event_type.display_name : event.event_type; var hostname = event.context && event.context.display_name ? event.context.display_name : event.context.hostname; var client_url = event.context && event.context.host_url ? event.context.host_url : "https://console.redhat.com"; // Trigger one alert per recommendation (PagerDuty limits to 40) var normalized_event = []; for (var i=1; i<event.events.length+1; i++) { var evt = { event_type: PD.Trigger, description: application + "/" + event_type + " on " + hostname + " (" + i + "/" + event.events.length + ")", details: event.events[i-1], client: "Red Hat Insights", client_url: client_url, contexts: contexts }; normalized_event.push(evt); } PD.emitGenericEvents(normalized_event);
With this Custom Event Transformer in place, PagerDuty can receive and handle events from Insights and forward them for alerts and incident creation. Please note that the supplied integration is an open source contribution and is not supported by Red Hat. The script can be modified according to your organization's needs and/or for handling incoming events data differently.
Lastly, we recommend using the intelligent grouping functionality from PagerDuty set with a window of five minutes to automatically group all related alerts in one incident. This is handy for dealing with different events generated from the same RHEL system as part of one PagerDuty incident.
![Screenshot PagerDuty intelligent alert grouping option](/rhdc/managed-files/styles/wysiwyg_full_width/private/insights-integration-pagerduty-img2.png?itok=hEp4ifvP)
Configuring Red Hat Insights for forwarding events
Insights forwards HTTP POST requests to a receiver endpoint every time an application triggers an event. The request contains a JSON payload body that includes all relevant Insights data. The data varies depending on the source application that triggers the event.
For the integration with PagerDuty, use Insights' webhook integration. Add a new integration by navigating to Settings > Integrations. The integration type must be Webhook and the endpoint URL corresponds to the PagerDuty endpoint URL previously generated in the Custom Event Transformer configuration. Finally, enable SSL verification for security reasons.
![Screenshot Edit integration dialog in Red Hat Insights](/rhdc/managed-files/styles/wysiwyg_full_width/private/insights-integration-pagerduty-img3.png?itok=Ck2yteR-)
Once the integration is configured, one must ensure that Insights forwards events. Under Settings > Notifications, make sure at least one behavior group includes the newly configured PagerDuty integration as an action and that it is associated with at least one event (e.g., nNew recommendations triggered by Advisor application).
Note that PagerDuty integration can be used with other actions like email notifications and/or integrations to third-party applications such as Slack, Microsoft Teams, Google Chat, Splunk, ServiceNow and Event-Driven Ansible.
More details on Integrations & Notifications configuration can be found in the Hybrid Cloud Console product documentation.
Validating the integration end-to-end
It is time to test the integration between Red Hat Insights and PagerDuty end-to-end. You can monitor the integration status from the Settings > Integrations page in Insights and verify that events are successfully triggered from the Event Log.
Assuming Insights events are successfully forwarded via the integration, you can monitor new alerts triggered in PagerDuty for each incoming event under Incidents > Alerts.
![Screenshot of a list of triggered alerts in PagerDuty](/rhdc/managed-files/styles/wysiwyg_full_width/private/insights-integration-pagerduty-img4.png?itok=FJLhudGM)
As you can see, our intelligent grouping functionality has associated all related alerts from the same system in one incident. You can view it under Incidents > All incidents.
![Screenshot of all related alerts via intellligent grouping functionality](/rhdc/managed-files/styles/wysiwyg_full_width/private/insights-integration-pagerduty-img5.png?itok=WXcyIXZ7)
Each alert can be expanded and contains the data collected from the Insights event according to the Custom Event Transformer's JavaScript.
![Screenshot of expanded alert details](/rhdc/managed-files/styles/wysiwyg_full_width/private/insights-integration-pagerduty-img6.png?itok=4-vq0PhZ)
Further, a link to Insights is provided for additional details, troubleshooting and resolution.
Wrap up
This article demonstrates the configuration of PagerDuty and Insights integration through webhooks. The outcome is the automatic generation of alerts in PagerDuty and a fully detailed incident report, making it an excellent foundation for advanced filtering and handling.
執筆者紹介
Jerome Marc is a Red Hat Sr. Principal Product Manager with over 15 years of international experience in the software industry spanning product management and product marketing, software lifecycle management, enterprise-level application design and delivery, and solution sales.
チャンネル別に見る
自動化
テクノロジー、チームおよび環境に関する IT 自動化の最新情報
AI (人工知能)
お客様が AI ワークロードをどこでも自由に実行することを可能にするプラットフォームについてのアップデート
オープン・ハイブリッドクラウド
ハイブリッドクラウドで柔軟に未来を築く方法をご確認ください。
セキュリティ
環境やテクノロジー全体に及ぶリスクを軽減する方法に関する最新情報
エッジコンピューティング
エッジでの運用を単純化するプラットフォームのアップデート
インフラストラクチャ
世界有数のエンタープライズ向け Linux プラットフォームの最新情報
アプリケーション
アプリケーションの最も困難な課題に対する Red Hat ソリューションの詳細
オリジナル番組
エンタープライズ向けテクノロジーのメーカーやリーダーによるストーリー
製品
ツール
試用、購入、販売
コミュニケーション
Red Hat について
エンタープライズ・オープンソース・ソリューションのプロバイダーとして世界をリードする Red Hat は、Linux、クラウド、コンテナ、Kubernetes などのテクノロジーを提供しています。Red Hat は強化されたソリューションを提供し、コアデータセンターからネットワークエッジまで、企業が複数のプラットフォームおよび環境間で容易に運用できるようにしています。
言語を選択してください
Red Hat legal and privacy links
- Red Hat について
- 採用情報
- イベント
- 各国のオフィス
- Red Hat へのお問い合わせ
- Red Hat ブログ
- ダイバーシティ、エクイティ、およびインクルージョン
- Cool Stuff Store
- Red Hat Summit