Linux tools: Getting the message out with dmesg

The dmesg command is one of those easily forgotten troubleshooting tools that should stay at the top of your sysadmin arsenal. It contains so much information about your system that dmesg should be the first place you look when something goes wrong. The output from dmesg is long, as you can see for yourself if you type it in at a command prompt because it reports information from all aspects of your system when there are no error messages—other than those that might appear at boot.

Formally, dmesg prints or controls the kernel ring buffer. The default action is to display all messages from this buffer.

For future reference and comparison, the best time to look at dmesg is just after boot. I usually send dmesg information to a text file using a command such as the following:

$ dmesg > dmesg.`date +%m.%d.%Y`.txt

This command creates a text file named dmesg.12.11.2019.txt. Between the file creation date and the next boot, you can check new messages generated by the kernel.

Possible post-boot messages include system errors, device errors, and information about any USB device that someone might plug in. For example, the following dmesg information appeared after inserting a USB drive:

[ 9189.631808] usb 1-1: new full-speed USB device number 2 using ohci-pci
[ 9189.909896] ohci-pci 0000:00:06.0: frame counter not updating; disabled
[ 9189.909958] ohci-pci 0000:00:06.0: HC died; cleaning up
[ 9194.910072] usb usb1-port1: attempt power cycle

To see a full listing of USB-related messages, issue the dmesg command and grep for usb:

$ dmesg |grep -i usb
[    0.052580] ACPI: bus type USB registered
[    0.052594] usbcore: registered new interface driver usbfs
[    0.052598] usbcore: registered new interface driver hub
[    0.052605] usbcore: registered new device driver usb
[    0.414901] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.414907] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.415398] ohci-pci 0000:00:06.0: new USB bus registered, assigned bus number 1
[    0.468262] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 4.18
[    0.468264] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.468266] usb usb1: Product: OHCI PCI host controller
[    0.468268] usb usb1: Manufacturer: Linux 4.18.0-80.el8.x86_64 ohci_hcd
[    0.468269] usb usb1: SerialNumber: 0000:00:06.0
[    0.468454] hub 1-0:1.0: USB hub found
[    0.468842] uhci_hcd: USB Universal Host Controller Interface driver
[    0.468885] usbcore: registered new interface driver usbserial_generic
[    0.468889] usbserial: USB Serial support registered for generic
[    0.470765] usbcore: registered new interface driver usbhid
[    0.470765] usbhid: USB HID core driver
[ 9189.631808] usb 1-1: new full-speed USB device number 2 using ohci-pci
[ 9194.910072] usb usb1-port1: attempt power cycle

As you can see, I used the grep command with the -i option so it ignores case and I will see everything associated with USB devices regardless of that (Usb, usb, or USB). This practice is good for any subsystem or filter that you want to use. Always ignore your filter's case.

For example, to see all disk (block devices) attached to your system, use the following command:

$ dmesg |grep -i sd
[    0.000000] Command line: BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-80.el8.x86_64 root=UUID=f695f641-e489-4674-afd8-8f354533811d ro crashkernel=auto rhgb quiet
[    0.000000] ACPI: RSDP 0x00000000000E0000 000024 (v02 VBOX  )
[    0.000000] ACPI: XSDT 0x000000003FFF0030 00003C (v01 VBOX   VBOXXSDT 00000001 ASL  00000061)
[    0.000000] ACPI: DSDT 0x000000003FFF0470 0022EA (v02 VBOX   VBOXBIOS 00000002 INTL 20100528)
[    0.000000] ACPI: SSDT 0x000000003FFF02A0 0001CC (v01 VBOX   VBOXCPUT 00000002 INTL 20100528)
[    0.000000] Kernel command line: BOOT_IMAGE=(hd0,msdos1)/vmlinuz-4.18.0-80.el8.x86_64 root=UUID=f695f641-e489-4674-afd8-8f354533811d ro crashkernel=auto rhgb quiet
[    1.545750] sd 2:0:0:0: [sda] 33554432 512-byte logical blocks: (17.2 GB/16.0 GiB)
[    1.545756] sd 2:0:0:0: [sda] Write Protect is off
[    1.545757] sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    1.545764] sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[    1.546960]  sda: sda1 sda2
[    1.547316] sd 2:0:0:0: [sda] Attached SCSI disk
[    1.975545] XFS (sda2): Mounting V5 Filesystem
[    2.092251] XFS (sda2): Starting recovery (logdev: internal)
[    2.137813] XFS (sda2): Ending recovery (logdev: internal)
[    3.993219] sd 2:0:0:0: Attached scsi generic sg1 type 0
[    5.909006] XFS (sda1): Mounting V5 Filesystem
[    5.959833] XFS (sda1): Starting recovery (logdev: internal)
[    5.962287] XFS (sda1): Ending recovery (logdev: internal)

The results show more than just a list of block devices, so you can further filter your results by specifying the filesystem, XFS in this case:

$ dmesg |grep -i xfs
[    1.965741] SGI XFS with ACLs, security attributes, no debug enabled
[    1.975545] XFS (sda2): Mounting V5 Filesystem
[    2.092251] XFS (sda2): Starting recovery (logdev: internal)
[    2.137813] XFS (sda2): Ending recovery (logdev: internal)
[    5.909006] XFS (sda1): Mounting V5 Filesystem
[    5.959833] XFS (sda1): Starting recovery (logdev: internal)
[    5.962287] XFS (sda1): Ending recovery (logdev: internal)

The use of the -i option here is superfluous but I always include it just in case there are any results that I wouldn't see otherwise. I use the up arrow key to replay my last command (a most excellent Bash feature) and just backspace over the last thing I searched for and replace it with my new keyword, so once I enter the command, I never have to bother with anything except what I'm searching for. No harm is done either way.

If you'd like to see if your remote system is equipped with a CD/DVD drive, try this command:

$ dmesg |grep -i cd
[    0.000000] clocksource: kvm-clock: mask: 0xffffffffffffffff max_cycles: 0x1cd42e4dffb, max_idle_ns: 881590591483 ns
[    0.414901] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[    0.414907] ohci_hcd: USB 1.1 'Open' Host Controller (OHCI) Driver
[    0.468262] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001, bcdDevice= 4.18
[    0.468268] usb usb1: Manufacturer: Linux 4.18.0-80.el8.x86_64 ohci_hcd
[    0.468842] uhci_hcd: USB Universal Host Controller Interface driver
[    1.328589] ata2.00: ATAPI: VBOX CD-ROM, 1.0, max UDMA/133
[    1.329773] scsi 1:0:0:0: CD-ROM            VBOX     CD-ROM           1.0  PQ: 0 ANSI: 5
[    1.577662] cdrom: Uniform CD-ROM driver Revision: 3.20
[    1.578616] sr 1:0:0:0: Attached scsi CD-ROM sr0

The last four lines display information about the CD-ROM drive. Although the CD-ROM drive is virtual on this system, if the virtual machine's complement of hardware includes it the drive can load ISO image files as if they were a bootable image on physical media.

The dmesg command isn't big and flashy. It doesn't do a lot of things or have a long list of options. Instead, it is elegant in its simplicity and as practical as that your pocket protector. Rather than as an afterthought, you should get into the habit of running dmesg on a regular basis on your systems. And, when something goes wrong, run it again to find out what the kernel knows about the problem. You might save yourself some grief and a few troubleshooting steps. You also might also look like a hero to your coworkers and management for finding the problem so quickly. Remember, time is money and you're trying to save it and the day.

Want to try out Red Hat Enterprise Linux? Download it now for free.