LVM SSD Caching

LVM has a cool feature that allows you to cache slow LVs on faster media. It has some limitations like not being able to cache an entire volume group but it does work. I had an extra Intel 520 240GB SSD that I was not using so I decided to add it to one of my servers to speed up an mdadm array.

I can probably say that using Gitlab is a bit snappier through the web interface after implementing the SSD cache.

  • vg0
    • Slow array (4 x 4TB spinning disks)
  • /dev/mapper/ssd
    • LUKS encrypted fast SSD

I will be adding a cache to existing LVs.

  1. Create the PV on the SSD

    • pvcreate /dev/mapper/ssd
  2. Extend the original volume group

    • vgextend vg0 /dev/mapper/ssd
  3. Create LV caches

    • 1
      2
      3
      lvcreate -L 40G -n cache0 vg0 /dev/mapper/ssd
      lvcreate -L 40G -n cache1 vg0 /dev/mapper/ssd
      lvcreate -L 40G -n cache2 vg0 /dev/mapper/ssd
  4. Add the cache to the existing logical volumes

    • 1
      2
      3
      lvconvert --type cache --cachepool vg0/cache0 vg0/lv0
      lvconvert --type cache --cachepool vg0/cache1 vg0/lv1
      lvconvert --type cache --cachepool vg0/cache2 vg0/lv2

That’s pretty much it. You can see utilization with this command: lvs -a -o +devices. I wouldn’t recommend using writeback caching unless your data is easily replaceable.

There is also bcache but I believe I’d have to wipe the array and start over. It’s probably not worth doing when a simpler solution exists.