Red Hat Satellite Ask Me Anything Q&A from June and August 2019

This blog covers the questions and answers during the June and August 2019 Satellite Ask Me Anything (AMA) calls. 

For anyone not familiar, the Satellite AMAs are an "ask me anything" (AMA) style event where we invite Red Hat customers to bring all of their questions about Red Hat Satellite, drop them in the chat, and members of the Satellite product team answers as many of them live as we can during the AMA and we then follow up with a blog post detailing the questions and answers.

The ground rules of the AMA are:

• In the interest of making everyone feel like they can truly ask any question, the Satellite AMA sessions are not recorded. 

• The Satellite AMA is not the appropriate place to ask questions about specific support cases or specific sales issues. While we may be able to give generic feedback about certain areas we cannot use this time to troubleshoot or dig into logs. For your support cases please continue to work with Red Hat support, and for any sales issues please work with your Red Hat or Partner sales rep.

• The AMA is presented using Blue Jeans Prime. All questions are asked via the Q&A panel which allows other attendees to vote on questions that are asked. Questions are read by an event moderator based on the popularity of the questions and answered live and in real time.

As we kicked off the Satellite AMAs we pointed out a few important items happening in the Satellite area:

• Red Hat Satellite 5.7 and earlier end of life (EOL) was on January 31, 2019. Content for these versions stopped on March 14th as part of the Red Hat Network (RHN) shutdown. To continue to receive content on your Satellite environment you need to upgrade to Satellite 5.8, if you have not yet done so.

• Satellite 5.8 EOL is scheduled for May 31, 2020. At that point all Satellite 5 versions will be EOL and you will need to transition to Satellite 6.

• Satellite 6.2 wient EOL on May 31 2019. 

• Satellite 6.3 will go EOL on October 31, 2019

• For more information about Satellite check out the latest version of the Satellite Frequently Asked Questions.

The next Red Hat Satellite AMAs will run at two times:

• October 31, 2019 at 3am GMT / 8:30am IST / 12:00 noon JST / 2pm AEDT

• October 31, 2019 at 3pm GMT / 4pm CET / 11am EDT / 8am PDT

Here are questions and answers (lightly edited for readability, grammar, spelling, etc.) from the June 13, 2019 Red Hat Satellite Ask Me Anything.

Question: What is the extent to which Ansible will be integrated into Satellite and what does that road map look like? Will I be able to use Satellite as a Tower instance anytime soon?

Answer: The extent of the Ansible integration is focused around Satellite usage. Run a role on a client. Build a client. State management with Ansible roles.

Tower is open ended, enterprise orchestration.

These are different use cases even though they both use Ansible technology. We are focused on integration between the two, not bringing Tower itself into Satellite.

Question: Is it planned that users can use OpenSCAP without Puppet? And if so will there be a directly integrated Ansible Solution?

Answer: Yes this is planned, as an Ansible solution. All of the client side tooling should have equivalent functionality with Ansible over time. This is planned for a future release.

Question: Is it still necessary to install Puppet on Satellite?

Answer: Installing the Puppet agent has never been necessary. Plenty of customers do not use Puppet. If you are using Puppet for config mgmt, then you would need to install it. For example, we have a puppet module for Installing Insights. That is a convenience for Puppet customers. We also have an Ansible role for this. Same thing for OpenSCAP. We have a Puppet module, but you can install by hand if you aren't using Puppet.

Additionally, Satellite ships the puppet server for the purposes of managing clients with Puppet. Today, if you don't use Puppet to manage clients, you STILL need to have the Puppet Server running on the Satellite/Capsule. If you aren't using Puppet, you can't disable/not-install it (yet). That's a future feature.

Question: Any information regarding Satellite and Smart Management roadmap especially as it relates to the new cloud Software-as-a-Service (SaaS) services announced at Summit?

Answer: Reach out to account team for a comprehensive roadmap. Smart Management roadmap is pretty large and includes multiple product managers. 

Question: How does one compare content views in Satellite 6.5? I don't consider using hammer to dump errata/RPMs to a file and then comparing the two files to be efficient.

Answer: Today that is the only way to compare them. You get a list of what is in each content view and compare the lists with a tool of your choosing. That is the best way to do it today.

Question: I have multiple Satellites; repo names for RHEL 8 on one Satellite are slightly different than what is on the others? I thought the names were supposed to be the same?

Answer: The name of repos are consistent across the entirety of Red Hat products. If you are not seeing this then you should file this as a bug. 

Question: Searching content views/repos seems daunting; why isn't there a simple search mechanism like on Satellite 5.8?

Answer: The searching capability (introduced in Satellite 6.4) allows for searching by label, scoping by repo type, etc. If there are usability improvements you would like to see please open a ticket.

Question: What is the point of the satellite-installer DNS & DHCP options that specify zones and ranges if they don't then automatically show up in Infrastructure --> Domains and Infrastructure --> Subnets?

Answer: The point of these installer options is to set up the Satellite server to be a DHCP or DNS server. They don't automatically show up partially because just because those functions are set up you may have a use case where you don't want those to be automatically imported.

Question: Is there a new Content Management Guide which describes how 6.5 Inter-Satellite Synchronization is done? Chapter 9 in the Red Hat Satellite 6.5 Content Management Guide is not sufficient to determine a content migration strategy.

Answer: Content management material is typically posted in the content management guide. Please file a support case provide feedback on any areas that you feel needs additional information or clarification.

Question: Interested in general info on what the upgrade process is going from 6.3.1 to the 6.5 and any concerns with doing so.

Answer: Check out the Red Hat Satellite Upgrade Helper. (Login required.) This will provide detailed upgrade steps that are tailored to your environment. You will need to go 6.3 --> 6.4 --> 6.5. Be sure to back up first. Consider also filing a proactive support ticket as well.

Question: Is there a way to get email notifications of new package updates for products that don't have errata information, like the emails you can get when new errata is synced?

Answer: Currently today we don't have this ability.  We had an RFE opened, but it was closed due to inactivity.

Question: When creating a host in Sat 6.5, is there any way to get the "Boot disk based" provisioning method to be the default on the Create Host>>Operating System tab as opposed to the "Network Based" provisioning method?

Answer: Please open a RFE for this capability so that it can be reviewed.

Question: Will the reports outputs get the extension that responds to the content type, instead of always .txt (e.g. a comma separated file gets a CSV extension and a webreport an htm or html extension) so an application that is linked to your browser will automatically open?

Answer: This is a bug, tracked here: https://bugzilla.redhat.com/show_bug.cgi?id=1658566

Question: Are there plans for Puppet to be decommissioned in Satellite?

Answer: We have no plans to remove Puppet for the lifetime of Satellite 6. We are continuing to upgrade the Puppet versions used in Satellite. We are also continuing to enhance our Ansible capabilities so customers have a choice.

Question: Satellite training is for version 6.2, close to EOL. Are there plans to update it soon?

Answer: Yes—this is planned and hopefully coming soon

Question: I'm extremely lazy. What's the best way to automate my host creation in Satellite? hammer-cli? write something with REST API? Ansible modules with using prompts?

Answer: This primarily depends on your preferred workflow. Hammer or Rest API are both easily automatible. Rich leans towards Hammer first. If a host that was provisioned from somewhere else (VMware VM for example), then use the bootstrap script. 

Start with Hammer, then progress to API. Pro-tip - Start in Hammer first with the debug flag.

Question: Is it possible to migrate the Red Hat subscription manifest from an old Satellite 6.2 server to a 6.4.3 instance without impacting the old server which still services various endpoints?

Answer: If you are planning to do this we are curious why you are migrating as opposed to upgrading. 

However, you address this issue by what is called manifest reuse. Both Satellites can have the manifest running at the same time as you move hosts over to the new Satellite.

Question: Why is Satellite 5.8 EOSL on May 31, 2020 and Red Hat Enterprise Linux 5 ELS goes EOSL on Nov 20, 2020?

Answer: Releases of RHEL and Satellite are only loosely coupled. As a general rule of Satellite we will support currently supported versions of RHEL. We will support this in a version of Satellite, but not every version of Satellite. We will still have support for RHEL 5 ELS in Satellite 6.

Question: Is there any way to add a single non-errata package to a new incremental content view without pushing a completely new content view version?

Answer: No. That breaks the idea of what a content view is. A point in time snapshot of repositories using filter. You can update the CV by updating the filters which pushes a new major version, or you can add incremental errata which pushes a minor version.

Question: Is there a document describing how to use the Ansible System Roles to update the host's static network configuration from the subnet config in Satellite?

Answer: We don't have a document today. Part of this is that there are different use cases depending on which system role is used. Point of system roles is to enable you to easily define the desired state of a system regardless of the RHEL version used. 

We think this ask is how to use the info in Satellite to fill out the info in system roles. Please open a ticket so we can investigate this further

Question: Now that RH has been acquired by IBM will we ever have the opportunity to manage content for AIX systems? (We have a lot of legacy AIX here and I assume the answer is definitely no.)

Answer: At this moment we have not had sufficient customer requests to consider managing AIX. Please open a support case and ask for an RFE for us to consider. 

Question: If a fictional corp has 2 Red Hat accounts (1 for the platform team & 1 for middleware), can the MW team create a manifest for their subscriptions even though the Satellite sub are under the platform account?

Answer: There are two accounts, and the Satellite subscription are under the platform account. You can create a Satellite manifest in the MW account, and you need Smart Management subscription to cover the hosts in that manifest. Then you can import into the Satellite. One manifest per org, so you would need multiple orgs.

Question: Are there plans for custom products/subscriptions that the containing repos are not enabled on default or switching this for each repository differently?

Answer: We do have an open RFE for this and are looking to add this capability in the future.

In order to have repos aligned to the version of the OS that is registering we have to have multiple custom products, which creates product sprawl. Or we have different activation keys which shifts the sprawl to another area of Satellite, which is not ideal. We are working on enhancing this.

Question: In the settings page when creating a remote execution credential, are there any plans to encrypt/hide the password setting? As is, when entering the password it is clear text which has caused hesitation for some customers?

Answer: Today, on the settings page, the credential is shown in the clear. There is an open bug to address that. In lieu of using that application wide setting (on the administer->settings page), they can use a parameter (which can be associated with an Organization, Location, or Hostgroup), which can be made 'hidden'

Question: To patch a disconnected Satellite server, the ISO method described in the documentation appears disjointed and overly complex. Isn't there a better way if ISS is in use? Can this be documented?

Answer: The best practice to patch a disconnected Satellite Server is to have a single connected Satellite for content sync and export. This will enable to to sync content on your desired schedule. 

You could have all of the latest content, across multiple repos, then export that data whenever you want and however you want. This is generally the recommended method and is documented here.

Question: Are there any plans for hammer to support external authentication sources configured in Satellite?

Answer: Hammer can support LDAP for auth, but not Kerberos or 2FA sources. (Those are future features we have planned.)

Question: In Satellite 6.5, if I need to sync ALL my content from a connected to a disconnected Satellite, is the new hammer content-view version --export-dir command appropriate, or is there a different method?

Answer: The default org view of Satellite is one big content view - you can run a single command to export it all. 6.5 doesn't change that, but adds the capability to export a content view (less than everything) then import that on the disconnected host.

Question: Will there be any Satellite-Ansible-Modules like there are modules for IDM?

Answer: We do have Ansible modules in the upstream and are looking to include those in Satellite in the future. We are trying to make it easy for you to manage things with Ansible.

Question: Using OpenSCAP is there a smart way to remediate the errors, not using Puppet or Red Hat Insights?

Answer: Today, not a smart way, but it is possible. 

Inside the SCAP security guide we also ship remediations. When you view the report in Satellite it will show you the remediation for specific rules which may be Ansible or Bash.

Using Satellite 6.4 or better you can create a remote execution job and run that job across your environment. This does not require Puppet or Insights.

Question: How to handle the OpenSCAP Ansible snippets?

Answer: In the OpenSCAP report you get a snippet of Ansible code that can be used to remediate that client. You can take the snippet, create a remote execution template, and run that as a job. 

Question: You mention using the Organization Default Content View (e.g. in the export-everything question)...but the Default CV is cannot be used in some places in the GUI. As an example, I can't add it to an Activation Key. Can that be fixed?

Answer: You can add default org views to the activation keys. It doesn't show in the list but you can select it.

Question: Does the new 6.5 ISS support incremental updates, or is a full export required each time? Do we need to publish a new content-view version before each export now? ("Default Organization View" export did not require republishing).

Answer: ISS in previous versions also supported incremental updates. Use the --since flag. A new export each time is not required. 

With a CV version the --since flag doesn't make sense though.

Question: When using Puppet modules via Satellite, is there a way to push new module updates only to a hostgroup? Currently I build a Puppet module and then put it in sequence in DEV, TEST, PROD. But whenever I go to make a change in the module after I've pushed it everywhere. I just want it to go to say, DEV.

Answer: Sounds like you want to manage Puppet content differently from the rest of the content. You have the option to select a different Puppet environment in the host group which would allow you a little more flexibility. Make the changes in the Puppet environment and associate it with the host group.

Questions and answers (lightly edited for readability, grammar, spelling, etc.) from the August 14, 2019 Red Hat Satellite Ask Me Anything.

Question: Any new Ansible integrations coming up?

Answer: Today with regards to Ansible we can do remote execution and can manage desired state with system roles. The remove execution is used by the REX engine as well as by the find it, fix it of Insights. For many of the capabilities of cloud.redhat.com, those tools can generate Ansible playbooks that can be used to remediate findings. 

We will add a similar find it, fix it workflow for the other cloud.redhat.com capabilities.

The other area is around Ansible Tower - if you have both today you can use Satellite as a dynamic inventory in Tower and you can callback to Tower post-provisioning.

Question: Is there a supported mechanism for doing scheduled upgrades with a reboot afterwards once the patches are applied, for example, something like https://github.com/RedHatSatellite/sat6-rex-patching

Answer: The built-in remote execution capabilities do have a pre/post option and you can use those to perform a reboot after the patches are applied. 

We would recommend using a shutdown command with a 15 second grace period after patching so that we have enough time to complete the REX command after patching.

Question: What configuration items in Satellite can be source controlled with git. Kickstart snippets?

Answer: All the textual things can—provisioning templates, kickstart tables, report templates, REX templates. 

A few releases ago we released the template sync functionality "Pull template from git". 

Satellite pulls the objects in from git or Satellite can be the master source and you can back them up to git.

Question: Will there be a system monitoring solution (Nagios/Icinga/Zabbix/etc.) added?

Answer: No—Satellite doesn't want to be managing/maintaining monitoring tools. Satellite 6.4.1 or better has a Satellite monitoring guide that shows you how to use Performance Co-Pilot.

Question: What features can be added to make auditing and maintaining large amounts of licenses? Can rules be added to Candlepin to set preferences/weights on licenses (ie use physical over Openshift licenses) and to automatically remove vdc licenses when v-hosts haven't been seen by virt-who in 30 days?

Answer: Can rules be added to set preferences and weights - That is one of the reasons for system purpose, introduced with RHEL 8. A subscription is effectively a selection of content, technical attributes, and business rules like SLAs. 

Today system purpose is good at separating Desktop, Workstation, Server, etc. In the future we want to use this to identify "I'm using this for a Satellite Server" or an OpenShift server for example. So I'll use a different sub to meet that need.

For removing hosts that haven't been seen by virt-who, once a system has a sub we never touch it again unless that system is no longer entitled. We don't actively prune the system. We don't know why it is off and don't want to touch it. In that scenario you should use an API to delete inactive systems and you can set the rules per your requirement. Red Hat doesn't have enough info to make intelligent pruning decisions.

There is a katello-cleanup script that can be used to remove hosts that haven’t checked in, available here: https://github.com/RedHatSatellite/katello-cleanup.

For what other features - we are planning to introduce subscription reporting tools as part of cloud.redhat.com in the future. This will allow you to go into a console and see your consumption, which will give you an ability to self-govern.

Question: Can Host Collections be added as a matcher for Smart Class Parameters similar to fqd/host group including the same functionality for prioritizing fqdn/hostgroup/os? It would add a more Hiera-like experience.

Answer: There was an RFE for this, but it was closed due to not enough customer activity.

Question: I inherited a Satellite 6.2 install where the previous administrator wrote custom scripts to directly access the repositories on the file system. Do you know if upgrading will change the file system layout? I'm trying to decide if I can upgrade, or need to build a parallel system to migrate away.

Answer: Upgrading will not change the filesystem layout. There are a couple of ways you can get current...

To upgrade you must stairstep - 6.2 > 6.3 > 6.4 > 6.5. You could also build a parallel 6.5 and migrate the systems over—we generally don't recommend this since you would have to rebuild all of your lifecycles, CVs, etc.

Either way the support team will always help you upgrade from an unsupported version to a supported version.

In general we have done a lot of work improving and streamlining the upgrade process.

Question: Enhancement request, add matcher based on multiple attributes, if attr1 = x and attr2 = y , values is 1.

Answer: Open up a support case on this one so we can expand smart class matchers to support boolean logic

Question: Any plans to remove Puppet? Will remote execution be replaced by an internal Ansible.

Answer: Remote Execution today (6.4 and above) is already driven by Ansible. 

In 6.2 Rex was a custom engine, but this was prior to the Ansible acquisition. It made sense to use Ansible since we had it already.

For the lifetime of Satellite 6 we will support Puppet like it is today. In a later release it will be "bring your own Puppet."

Question: If Sat6 goes to a "bring your own Puppet" will that impact the Puppet modules lifecycle/content-view functionality of Sat6? My entire workflow is built around the Puppet module lifecycle....

Answer: To be determined. We have not solidified the strategy at this time.

Question: When will Puppet support modularity/AppStream? There currently isn't a way to say, for example, I'd like the equivalent of 'dnf install postgresql:9.6/client' in Puppet code.

Answer: With regards to Puppet, a new resource type is required. When Puppet ships that in the upstream we will include it in a future release of Puppet. We plan to skip Puppet 6 since it is a short term release and plan to go to Puppet 7. 

Question: Would like to hear the roadmap for Satellite 6.6 / 6.7 - what features are coming up and is there anything I should be doing now to prepare?

Answer: To prepare for future versions, go ahead and start preparing to get to the newest Satellite release, currently 6.5. Satellite upgrades are sequential and it is helpful to be on the latest version to prepare for the next version.

Also note that we are roughly on about a 6 month cadence for Satellite releases - we are trying to make this predictable to help you plan your upgrades.

Question: Will Sat 6 be adding Serial Over Lan support for BMC/IPMI for remote console?

Answer: We currently do not have this on the roadmap. If this is important please open an RFE with the business justification.

Question: Would it be possible to add functionality to the customer portal to activate multiple L3 keys as a single contract and combine existing like contracts?

--Someone with 500+ contracts.

Answer: We can't address the feasibility of this since we do not own the customer portal. But we can send that feedback along. We understand the question and the impact to you, but sadly this exists outside of our sphere of direct influence.

Question: Will nested organizations be coming to Sat 6?

Answer: Not at this time.

Question: What's the best way to manage host groups? It appears we have to have one for every lifecycle environment, and then every capsule, and then further for each subnet in order to facilitate proper provisioning.

Answer: Generally we recommend using nested hostgroups.

Question: Will there be a way to export/import Smart Class Parameters into/from Hiera?

Answer: No—You can use Hiera if you want to, but we don't plan to build tooling to go back and forth between them.

Question: Are there any performance differences between MongoDB Satellite vs PostgreSQL only (Speed/storage/memory...)? Were other object oriented NoSQL DB's like Cassandra considered?

Answer: Today we use both MongoDB (Pulp) and PostgreSQL (everything else). The intention is to move everything over to PostgreSQL since all of the data is relational, so a relational database makes the most sense. PostgreSQL is mature, performant, and handles our use cases well. We're also working with some 3rd parties for possible HA options in the future.

Question: What's the best approach in migrating clients from Sat 5.8 to 6.5?

Answer: The best workflow is to build the Satellite 6.5 up parallel to the 5.8 environment. When it is ready, use the bootstrap script to migrate the hosts (unregister them from Sat5, register them to Sat 6 and put them in the right hostgroups).

There is also an Ansible playbook for bootstrap that was shipped in Satellite 6.5 and newer.

Question: When will Ansible for remote execution be available?

Answer: Already available now as part of 6.4 and above.

Question: there is no place to specify Ansible capsule for the host so in case of multi-capsule environment does it always go through first capsule that was installed? 

Answer: Your subnet definitions have the option to list the preferred capsule for that subnet.

Start with the subnets which are explicit on which capsules to use.

If a system is not matched to a subnet, then there is a hierarchy we go through to figure out which is the best capsule to use. The most common would be to look at which capsules are being used for content. Worst case the Satellite itself will be used.

Question: Will there be a solution soon to the issues with Download policies and capsules that cause the kickstart not to be available? https://access.redhat.com/solutions/4192431

Answer: The recommendation for kickstart repos is to use the immediate policy which will ensure the boot media is available. Even if using the on demand for the other repos, for the provisioning stuff you should consider the immediate policy to ensure it is already on disk.

This is just for the kickstart repo—not for the download policy on the capsule.

Question: Is there a way to make specific graphs for the Dashboard or Statistics? A "Exec Summary" with custom widgets for example.

Answer: Currently No—the widgets are fixed and can only be moved around. 

A way to solve this problem might be to get the data you need out of Satellite in a format that makes sense, then visualize that in your preferred tool. You can use the new 6.5 reporting engine or you can pull the info from the API.

Question: What do you recommend in applying patching in quarterly basis? Per lifecycle environment and host collections.

Answer: Patching is a complex issue and is often driven by your organizational requirements.

Generally speaking, the first thing to consider is how to set up your content views. RHEL6 & RHEL 7 are different lifecycles, and should be different CVs. Each RHEL revision will have its own CV, and probably a different CV per architecture as well.

Layout the lifecycle environments that are in a linear model - dev, QA, prod for example. 

How to maintain? Recommend using CV filtering. This is RHEL 7, but include content up until the end of Q1. When you go to patch, update the filter with a new date which will add the new content. Then republish the CV to get the new content.

This gets the content into the right spot. For patching, leverage the remote execution framework. Fast, scalable, lightweight and doesn't require any additional agents. Rex can do the patching for you.

Which systems to patch? Use host collections. Host collections are essentially a tag - this is a test/dev/whatever system. They you build a search query and save them as bookmarks. Then Rex jobs can operate on the bookmarks.

We would also recommend taking a look at the content management guide that covers this in more detail.

Join us for the next AMAs

The October Red Hat Satellite Ask Me Anything is scheduled to run at two different times:

• October 31, 2019 at 3am GMT / 8:30am IST / 12:00 noon JST / 2pm AEDT

• October 31, 2019 at 3pm GMT / 4pm CET / 11am EDT / 8am PDT

Please join us and bring any questions about Satellite that you might have. We look forward to hearing from you!