Boosting Linux Storage Performance with LVM Striping
In the realm of storage management, achieving optimal disk performance takes precedence. The need for faster data access, reduced latency, and improved I/O operations and throughout has led to the adoption of advanced techniques. One such technique is LVM Striping
, a powerful feature of the Logical Volume Manager LVM
that can significantly enhance volume performance.
I — Linear LVM vs Striping LVM:
I .1 — Linear LVM:
The linear configuration, often considered the standard LVM setup, involves adding multiple physical volumes (disks) to a volume group in a linear fashion. This means that data is sequentially stored across these volumes, utilizing one disk before moving on to the next.
While linear LVM provides a straightforward approach to storage expansion, it may not fully exploit the potential for parallel processing and increased throughput.
I .2 — Striping LVM:
In contrast, Striping LVM takes a more advanced approach by distributing data across multiple physical volumes simultaneously. This striping process creates a logical volume that spans the disks, allowing for parallel read and write operations. The result is a substantial boost in performance, making Striping LVM an attractive option for environments with high I/O and throughput demands.
- Stripe Creation:
- The data is divided into segments known as “
stripes
” - Each stripe is written to a specific disk in the striped logical volume
LV
.
2. Parallel Writing of Stripes:
- These stripes are simultaneously written across the multiple physical volumes
PVs
(disks).
3. Balanced Distribution:
- The stripes are distributed evenly across the disks, preventing a single disk from becoming a bottleneck.
4. Enhanced IOPS and Throughput:
- Through the parallel writing of stripes, LVM Striping significantly boosts the overall throughput and performance of the logical volume.
I .3 — Scenario :
Consider a scenario with three disk drives allocated to three physical volumes PVs. If each individual physical volume can achieve a total of 125M/s
as throughput :
- Employing “LVM Striping” would result in a volume group capable of
375M/s
. - In contrast, using “LVM Linear”, the throughout remains at
125M/s
, and no matter how many disks we add in LVM.
II — Setting up “Linear LVM“ and “Striping LVM”:
In this section, we’ll establish two volume groups (VGs): the initial one for Linear LVM utilizing 3 disks, and the second one for Striping LVM, also with three disks. Following the VG setup, we’ll proceed to create Logical Volumes (LVMs) on each group, then formating and mounting them
II.1 — Initial check
Environment :
— Server : AWS EC2 instance = m4.10xlarge ,
— EBS Volumes : 6 x 20G EBS volumes .
— Each EBS volume has the following characteristics : { Type : GP3, IOPS = 3000 , Throughput =
125MiB/s
}— OS/image : Amazon Linux 2
Here the list of the volumes :
Let’s check our EC2 instance and see our disks
# lsblk
II.2 — Setting up the VGs:
In order to be able to create the LVs, we need first to create the VG for each one of them :
# vgcreate vg_linear /dev/sdb /dev/sdc /dev/sdd
# vgcreate vg_striping /dev/sde /dev/sdf /dev/sdg
# vgs
II.3 —Create a Logical Volume as “Linear LVM”
# lvcreate -l 100%FREE -n lv_linear vg_linear
-l 100%FREE
: The size of LV is set to match the entirety of the available free space within the VGvg_linear
.-n lv_linear
: LV namevg_linear
: Target VG
II.4 — Create a Logical Volume as “Striping LVM”
# lvcreate -l 100%FREE -i 3 -I 64k -n lv_striping vg_striping
-l 100%FREE
: The size of LV is set to match the entirety of the available free space within the VGvg_striping
.-n lv_striping
: LV namevg_striping
: Target VG-i 3
: stripes Number-I 64k
: stripe size
II.5 — Check the LVs
Let’s check the new LVs:
- Let’s start by
lv_linear
# lvdisplay -m /dev/vg_linear/lv_linear
- Let’s check new
lv_striping
# lvdisplay -m /dev/vg_striping/lv_striping
==> Here we can see the different details about the striping that we configured .
II.6 — Formating and mounting the LVs
# mkfs.xfs /dev/vg_linear/lv_linear
# mkfs.xfs /dev/vg_striping/lv_striping
# mkdir /mnt/linear
# mkdir /mnt/striping
# mount /dev/vg_linear/lv_linear /mnt/linear/
# mount /dev/vg_striping/lv_striping /mnt/striping/
# df -h
==> The LVs are now mounted correctly under the appropriate directories
III — Benchmarks the LVs/disks
In order to benchmark our LVs/disks, we will use the fio
tool. So let us install it before :
# yum install fio -y
III .1 — Benchmark “lv_linear”
In order to benchmark the LV, we will use the fio
config file below that will help us generating traffic for 400M
as throughput :
# cat fio_config-1.fio
[global]
ioengine=libaio
runtime=60
time_based
direct=1
rw=write
size=10G
bs=512K
rate=400M
numjobs=16
[job1]
filename=/mnt/linear/testfile
Execute now the fio
tool as below using the defined file above:
# fio fio_config-1.fio
- In the same time, run the
iostat
tool to monitor the disks usage in a separate terminal :
# iostat -xdmt 2
==> We can see from the output above that ONLY the first disk xvdb
is used and that the throughput is equal to 125M
which is the baseline value of one EBS volume itself.
The same thing is showing by the summary given the fio
tool :
==> WRITE operations were exclusively performed on a single disk.
III .2 — Benchmark “lv_striping”
Use the same fio
config file above but just change the filename
parameter :
filename=/mnt/striping/testfile
Use iostat
to monitor the disks
==> All 3 disks used by the lv_striping
participated in the WRITE operations, resulting in a combined throughput equal to the sum of the individual disk throughputs ( 125M X 3 = 375M
).
The same thing is showing by the summary given the fio
tool :
Conclusion:
LVM Striping stands as a robust solution for organizations seeking to unlock the full potential of their storage infrastructure. By distributing data intelligently across multiple disks, LVM Striping not only boosts performance but also provides a scalable and flexible approach to storage management. Embrace LVM Striping to harness the power of parallelism and take your disk performance to new heights.
If you found the post enjoyable and helpful, WHY LEAVE WITHOUT ANY REACTION 😔? At least show your support by clapping, leaving a comment, and following me. 😄
— Thanks