Python vs. Bash? Why not both with Python's subprocess module

Python is a versatile, user-friendly programming language that excels in string manipulation, analytics, and reporting. For this reason, many system administrators use Python to accomplish routine tasks but fall back on Bash for familiar and often rudimentary chores. It's a good combination, but why use Bash for some tasks and Python for others when you can use both?

[ Download A sysadmin's guide to Bash scripting. ]

Using Bash to run repetitive tasks or low-level system commands and then Python for further analysis or processing the output into a readable format is an excellent way to harness the strength of both of these languages. Fortunately, Python's subprocess module easily integrates Bash into your Python scripts.

To run the examples in this guide, you must install Python 3.5 or above. The subprocess module is a standard library in Python and does not require manual installation.

Subprocess module basics

The subprocess module contains several practical methods to run the Bash commands. The primary subprocess method you will use is run. With run, you can write any Bash command in single, double, or triple quotes as the first argument in the method. For example:

subprocess.run("ls")

If you want to capture the output, you can add the capture_output=True argument:

subprocess.run("ls", capture_output=True)

While there are many other useful methods with the subprocess library, this guide focuses only on the run method.

Note: if you are using macOS, you must add the shell=True argument to the run method.

For more information on the subprocess module, please visit the docs.

When to use Python's subprocess module

The subprocess module works best when running simple Bash commands inside your Python script. The subprocess module is limited with more complex Bash commands or if you need to create a new shell.

For example, getting an rsh command to work with subprocess is extremely difficult. If you have a long Bash command that you want to run in subprocess, you can enclose it in triple quotes. For instance:

subprocess.run('''ls -l | awk -v OFS='\t' 'BEGIN { printf "%s\t%s\n", "Name", "Size"} {print $9,$5}''')

However, the best way to run complicated Bash commands is to create a Bash script with your commands and then use the subprocess module to run it. Depending on what you want to do, you can capture the output and then continue processing it in Python.

proc = subprocess.run(["./sample_Bash.sh"], shell=True, capture_output=True)

Practical Python

Python and Bash make a great combination for the busy sysadmin. I've covered the basics of the subprocess module in this article. In my next article, I'll demonstrate some practical ways you can combine Python with Bash on your system.

[ Learn how to manage your Linux environment for success. ]