Base64 encoding: What sysadmins need to know

As a sysadmin, you may have heard of Base64 encoding in the context of certificates or tools that accept and store data in an encoded format. Base64 encoding is used in OpenSSL, Kubernetes secrets, email applications, and many other technologies.

Despite Base64's ubiquity, many systems administrators have limited knowledge of how this encoding format actually works. Understanding Base64 can help you better comprehend how data is stored and used by the tools you rely on and support. It also enables you to use Base64 encoding in your own applications and scripts.

This article walks you through the theoretical aspects of Base64 encoding. Once I explain this simple encoding format, I will introduce the base64 command-line utility that you can use to encode and decode data. Finally, I provide some practical examples of Base64 encoding that you will likely encounter as a sysadmin.

How Base64 works

Fundamentally, Base64 is used to encode binary data as printable text. This allows you to transport binary over protocols or mediums that cannot handle binary data formats and require simple text.

[ Download now: A sysadmin's guide to Bash scripting. ]

Base64 uses 6-bit characters grouped into 24-bit sequences. For example, consider the sentence Hi\n, where the \n represents a newline. The first step in the encoding process is to obtain the binary representation of each ASCII character. This can be done by looking up the values in an ASCII-to-binary conversion table.

Image

ASCII uses 8 bits to represent individual characters, but Base64 uses 6 bits. Therefore, the binary needs to be broken up into 6-bit chunks.

Image

Finally, these 6-bit values can be converted into the appropriate printable character by using a Base64 table.

Image

Since Base64 uses 24-bit sequences, padding is needed when the original binary cannot be divided into a 24-bit sequence. You have probably seen this type of padding before represented by printed equal signs (=). For example, Hi without a newline is represented by only two 8-bit ASCII characters (for a total of 16 bits). Padding is removed by the Base64 encoding schema when data is decoded.

Image

Encode and decode Base64 at the command line

Now that you understand how Base64 encoding works, you can work with it at the command line. You can use the base64 command-line utility to encode and decode files or standard input. For example, to encode the previous example:

$ echo Hi | base64
SGkK

[ Keep your favorite Git commands, aliases, and tips close at hand. Download the Git cheat sheet. ]

You can decode Base64-encoded data using the -d flag:

$ echo SGkK | base64 -d
Hi

Try some Base64 examples

Using Base64 encoding may seem esoteric, but there are many common scenarios where you will encounter Base64 in systems administration. Knowing how to recognize, encode, and decode Base64 is valuable due to its prevalence as an encoding approach.

One of the most common use cases is certificate encoding. If you have worked with the OpenSSL tool, you have seen several examples of Base64-encoded certificates. The Base64-encoded certificate structure is commonly referred to as the Privacy Enhanced Mail (PEM) format, but a certificate is fundamentally binary data. A PEM certificate can be converted to its binary Distinguished Encoding Rules (DER) equivalent by decoding the Base64-encoded data.

To see this in action, you can obtain the PEM-encoded certificate and the binary DER certificate for redhat.com:

# Download the certificate in Base64 encoded PEM format
$ echo | openssl s_client -connect www.redhat.com:443 2>/dev/null | openssl x509 -outform pem -out redhat.com.pem

# Download the certificate in binary DER format
$ echo | openssl s_client -connect www.redhat.com:443 2>/dev/null | openssl x509 -outform der -out redhat.com.cer

# Confirm that the PEM encoded certificate contains printable, Base64 encoded data
$ head redhat.com.pem 
-----BEGIN CERTIFICATE-----
MIIHNjCCBh6gAwIBAgIQCVe4E0h49mzI0NcSqMy1+jANBgkqhkiG9w0BAQsFADB1
MQswCQYDVQQGEwJVUzEVMBMGA1UEChMMRGlnaUNlcnQgSW5jMRkwFwYDVQQLExB3
d3cuZGlnaWNlcnQuY29tMTQwMgYDVQQDEytEaWdpQ2VydCBTSEEyIEV4dGVuZGVk
IFZhbGlkYXRpb24gU2VydmVyIENBMB4XDTIxMTIwMTAwMDAwMFoXDTIyMTIwMTIz
NTk1OVowgcoxHTAbBgNVBA8MFFByaXZhdGUgT3JnYW5pemF0aW9uMRMwEQYLKwYB
BAGCNzwCAQMTAlVTMRkwFwYLKwYBBAGCNzwCAQITCERlbGF3YXJlMRAwDgYDVQQF
EwcyOTQ1NDM2MQswCQYDVQQGEwJVUzEXMBUGA1UECBMOTm9ydGggQ2Fyb2xpbmEx
EDAOBgNVBAcTB1JhbGVpZ2gxFjAUBgNVBAoTDVJlZCBIYXQsIEluYy4xFzAVBgNV
BAMTDnd3dy5yZWRoYXQuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKC

The Base64-encoded PEM certificate can then be decoded using the base64 command. You can see that it is exactly the same as the binary file that was saved using OpenSSL earlier:

# Decode the certificate and save it to redhat.com.decoded
$ tail -n +2 redhat.com.pem | head -n -1 | base64 -di > redhat.com.decoded

# Confirm that the decoded file and the original binary file are the same
$ md5sum redhat.com.decoded redhat.com.cer
8085b86934f6ff4abddac42793aed92f  redhat.com.decoded
8085b86934f6ff4abddac42793aed92f  redhat.com.cer

# Confirm that OpenSSL can inspect the decoded certificate file
$ openssl x509 -inform der -in redhat.com.decoded -noout -dates
notBefore=Dec  1 00:00:00 2021 GMT
notAfter=Dec  1 23:59:59 2022 GMT

Another common use for Base64 encoding is Kubernetes secrets. A Kubernetes secret contains key/value pairs where the value is Base64-encoded. For example, creating and reading back a secret shows that the value of the secret is Base64-encoded:

# Create a secret
$ kubectl create secret generic --from-literal=api-key=SuperSecret123! api-secrets
secret/api-secrets created

# The secret value appears to be encoded
$ kubectl  get secret api-secrets -o yaml
apiVersion: v1
data:
  api-key: U3VwZXJTZWNyZXQxMjMh
kind: Secret
metadata:
  creationTimestamp: "2022-07-26T18:35:00Z"
  name: api-secrets
  namespace: default
  resourceVersion: "5419"
  uid: ac23136b-4b51-4df1-b732-cff0195fff6b
type: Opaque

Existing secrets can be decoded using the base64 command after retrieving them with kubectl:

# Read back the encoded secret using a jsonpath
$ kubectl  get secret api-secrets -o jsonpath={.data.api-key}
U3VwZXJTZWNyZXQxMjMh
 
# Decode the secret by piping it into the base64 command
$ kubectl  get secret api-secrets -o jsonpath={.data.api-key} | base64 -d
SuperSecret123!     

Skip to the bottom of list

Image

Download now

Base64 encoding uses printable characters to encode binary data. This enables another interesting use case: You can encode multiline files into Base64 strings. While rare, you may come across software or scripts that accept an entire configuration file as a Base64-encoded string.

Consider the configuration file below, which is a YAML dictionary:

$ cat config.yaml 
allowed_users:
  - name: alice
    group: admins
  - name: bob
    group: powerusers
allowed_groups:
  - admins
  - powerusers
  - secops

Attempting to store this as an environment variable doesn’t work because the newlines are lost:

$ CONFIG=$(cat config.yaml)

$ echo $CONFIG
allowed_users: - name: alice group: admins - name: bob group: powerusers allowed_groups: - admins - powerusers - secops

However, you can Base64-encode the configuration file and store this printable string in an environment variable. This can then be decoded without any loss of fidelity:

# Encode the config file using base64 and disable line wrapping with -w 0 to avoid newlines in the output
$ CONFIG=$(base64 -w 0 config.yaml)

$ echo $CONFIG
YWxsb3dlZF91c2VyczoKICAtIG5hbWU6IGFsaWNlCiAgICBncm91cDogYWRtaW5zCiAgLSBuYW1lOiBib2IKICAgIGdyb3VwOiBwb3dlcnVzZXJzCmFsbG93ZWRfZ3JvdXBzOgogIC0gYWRtaW5zCiAgLSBwb3dlcnVzZXJzCiAgLSBzZWNvcHMK

# Decode the environment variable, revealing the original config file contents
$ echo $CONFIG | base64 -d
allowed_users:
  - name: alice
    group: admins
  - name: bob
    group: powerusers
allowed_groups:
  - admins
  - powerusers
  - secops

Wrap up

In this article, you saw how Base64 encodes data from a theoretical and an applied perspective. You also saw how to apply these ideas using the base64 command and how this can be useful in common situations, such as with Kubernetes secrets.

As with any technology, having a deeper understanding of the underlying concepts behind Base64 enables you to leverage this encoding format properly when appropriate.

[ Learn how to manage your Linux environment for success. ]