How to find interrupts for block device

George Shuklin
OpsOps
Published in
2 min readMay 9, 2022

It’s really complicated.

First, for every device you may have more than one queue, and, may be more than one IRQ. Also, everyone lies. Not exactly lying, but misleading.

Second, controller interrupt and device interrupt(s) may be very different thing.

Look at naive (non-working) approach:

Run lspci. Got:

3c:00.0 Non-Volatile memory controller: Toshiba Corporation XG4 NVMe SSD Controller (rev 01) (prog-if 02 [NVM Express])
Subsystem: Toshiba Corporation XG4 NVMe SSD Controller
Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0, IOMMU group 17
ls /proc/irq/16/
... i2c_designware.0 i801_smbus idma64.0 ...

i801_smbus is definitely not the nvme device…

Look at the server:

c1:00.0 RAID bus controller: Broadcom / LSI MegaRAID 12GSAS/PCIe Secure SAS39xx
Subsystem: Dell PERC H755N Front
Flags: bus master, fast devsel, latency 0, IRQ 277, NUMA node 1

There is no IRQ 277 on the server!

ls -1d /proc/irq/27*
/proc/irq/27
/proc/irq/278
/proc/irq/279

The proper way is

My laptop:

ls $(realpath /sys/block/nvme0n1/device/)/../../msi_irqs
132 133 134 135 136

For my laptop’s nvme there were 4 IRQs.

You can confirm that by looking on /proc/irq/132, there is nvme0q0 (and so on).

The server:

The actual interrupts for /dev/sdb (the device I’m interested in) is:

ls $(realpath /sys/block/sdb/device/)/../../../msi_irqs

There were whooping 128 interrupts dedicated for a single raid device. Talking about server stuff here!

The device name in /proc/irq/… is non-descriptive: megasas12-msix0, which is for me means ‘a… something there’.

Why were three ‘../../..’ for server and ‘../..’ for laptop?

The reason is that this is ‘end of pci’, which is different for different buses.

realpath for server device returns:

/sys/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.0/host12/target12:3:111/12:3:111:0

host12 and target12 — is SAS stuff. well as target and so on. We are interested on the pci path.

Here it was /sys/devices/pci0000:c0/0000:c0:01.1/0000:c1:00.0

For my laptop nvme it was simpler: /sys/devices/pci0000:00/0000:00:1d.0/0000:3c:00.0/nvme/nvme0 , and pci address was /sys/devices/pci0000:00/0000:00:1d.0/0000:3c:00.0

The general rule

  1. realpath /sys/block/{you_device}/device
  2. Find pci part of the path.
  3. cat {this_path}/msi_irqs
  4. Reconfirm IRQ by looking to /proc/irq/{any_of_number_above}. There going to be a reasonable name, related either to device, or controller, or, at least, to the driver.

--

--

George Shuklin
OpsOps

I work at Servers.com, most of my stories are about Ansible, Ceph, Python, Openstack and Linux. My hobby is Rust.