Adding Storage To Your Azure VM

Today we’ll be reviewing how to add additional storage to your Microsoft Azure Linux VM. This tutorial will review running a MySQL database server on an Azure VM and increasing the amount of storage available to you. There are many options on where you may want to host a database, but today we’ll focus on VM hosted and self-management. If you are looking for a fully managed MySQL database service, check out Azure Database for MySQL.

Getting Started:

I’ve created a MySQL server on a Debian 9 VM and have separate disks for OS and the data:

Next, we’ll look into how to add an additional disk to this VM to store backups.

The server we are currently using has three disks attached to it, which you can see here:

root@mymysql0:~# df -h Filesystem Size Used Avail Use% Mounted onudev 1.7G 0 1.7G 0% /devtmpfs 344M 4.8M 339M 2% /run
/dev/sda1 30G 1.6G 27G 6% /
tmpfs 1.7G 0 1.7G 0% /dev/shmtmpfs 5.0M 0 5.0M 0% /run/locktmpfs 1.7G 0 1.7G 0% /sys/fs/cgroup
/dev/sdb1 49G 53M 47G 1% /mnt/resource
/dev/sdc1 49G 202M 47G 1% /bitnamitmpfs 344M 0 344M 0% /run/user/1001
tmpfs 344M 0 344M 0% /run/user/1000

By using the df command we’re able to view the three mounted disks on the system.

sda1 = this is the primary OS partition — we’re using this to store the operating system.

sdb1 = this is an ephemeral disk used for temporary storage, including swap — we won’t concern ourselves with modifying this since we don’t expect any long term usage of any objects stored on this disk.

sbc1 = this is the data disk for our database binary files — as MySQL creates new records in the database and persists to disk, we’ll see these in the directory selected for MySQL data storage:

Microsoft Azure comes with a number of options for how your storage on the VMs will be managed long term. You can select standard disks or managed disk storage.

Managed disks simplify disk management for virtual machines by managing the storage accounts associated with the VM disks and provides granular access control with RBAC.

Migration is not reversible. Once completed, unmanaged disks can no longer be attached to this virtual machine. The source disks will not be deleted. After migration, configuration of the virtual machine will not be changed. Learn more by clicking here.

The virtual machine undergoing a migration to managed disks will be stopped and restarted after migration is complete. At this time we will skip this step and move on to the volume addition and volume increase on our own. We’ll work with management of our storage in another tutorial

Adding a new hard disk volume to Azure Linux VM

This process is less invasive that resizing a disk because there’s no need to reboot if simply adding a new volume. We’ll need to create the volume in Azure, associate it with our VM and then provision the disk to the local operating system so we can begin storing backups on it.

Let’s go to the Azure VM we’ll be modifying in the portal, and then select disks:

Next, we’ll provision a new unmanaged disk and then select a storage account. We’ll provision this to the same storage account currently used by the mymysql group we created. We have options to increase the disk type, the volume contents (if we have an existing disk image containing files you’d like to attach) and modify the IOPS limit.

I’ve selected standard options and will be provisioning 1023 GiB to the server:

The next step requires you to specify a container name for this new disk, in this case I’ve stated the container name for my storage is “mysqlstorage” — this is just so I can tell which is which.

I have left our RBAC settings to private, meaning this is not a shared disk for other systems:

Now we’re ready to finalize the disk creation details; you’ll be provided with full details on the disk we are creating.

Once you’ve finished applying the container details and click “OK”, you’ll be brought to this details screen:

Click “Create” and it will then permit you to finalize details on adding the disk in the portal. You’ll then be brought to the section to give the disk a logical name and eventually can click “Save” to attach the LUN associated with the disk to the VM:

After clicking “save” your account will be updated and the disk will now be attached:

Partition the disk

The disk is now physically attached to our VM, we can validate this information by looking in the dmesg data again on the server like we did before:

[ 3095.666567] sd 5:0:0:1: [sdd] Attached SCSI diskroot@mymysql0:~# dmesg |egrep sdd[ 3095.559898] sd 5:0:0:1: [sdd] 2145386496 512-byte logical blocks: (1.10 TB/1023 GiB)[ 3095.596385] sd 5:0:0:1: [sdd] Write Protect is off[ 3095.614696] sd 5:0:0:1: [sdd] Mode Sense: 0f 00 10 00[ 3095.614943] sd 5:0:0:1: [sdd] Write cache: enabled, read cache: enabled, supports DPO and FUA
[ 3095.666567] sd 5:0:0:1: [sdd] Attached SCSI disk

To begin the process of mounting, first we’ll need to fdisk and create a partition (bold are commands you will enter):

fdisk /dev/sdd Welcome to fdisk (util-linux 2.29.2). Changes will remain in memory only, until you decide to write them.Be careful before using the write command.Device does not contain a recognized partition table.Created a new DOS disklabel with disk identifier 0x1a9df8d1.Command (m for help):

We’ll create a new primary partition by pressing n and then pressing p:

Command (m for help): nPartition type p primary (0 primary, 0 extended, 4 free) e extended (container for logical partitions)
Select (default p): p

We do not have any existing partitions on this disk, so we’ll create partition 1 and then select defaults to create one large partition. If we wanted to split the 1TB disk into several partitions we can do here, but there’s no requirements to do so for this backup disk.

We’ll select the defaults for sector sizes by just hitting enter:

Partition number (1-4, default 1): 1First sector (63-2145386495, default 63):Last sector, +sectors or +size{K,M,G,T,P} (63-2145386495, default 2145386495): Created a new partition 1 of type 'Linux' and of size 1023 GiB.

Finally, use the w command to write the partition to the disk:

Command (m for help): wThe partition table has been altered.Calling ioctl() to re-read partition table.Syncing disks. root@mymysql0:~#

Mounting the partition

Our disk is now partitioned and ready for mounting so we can begin using it on our VM. We can view the new disk in dmesg again by running the following command:

root@mymysql0:~# dmesg |egrep sdd1[ 4105.968336] sdd: sdd1[ 4105.992448] sdd: sdd1

Lets format this new disk like you would in almost any operating system. We can use standard ext4 here to format our disk using the following command:

root@mymysql0:~# mkfs.ext4 /dev/sdd1mke2fs 1.43.4 (31-Jan-2017)Creating filesystem with 268173304 4k blocks and 67043328 inodesFilesystem UUID: a218a0a0-587e-4152-8e51-90ae6837cb1fSuperblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000, 7962624, 11239424, 20480000, 23887872, 71663616, 78675968, 102400000, 214990848 Allocating group tables: doneWriting inode tables: doneCreating journal (262144 blocks):

The disk is now ready to be made usable. First I will create a mountpoint directory; in this case I’ll create /backup:

root@mymysql0:~# mkdir /backup

Now we can get the disk mounted by getting the blkid information and adding it to fstab:

root@mymysql0:~# blkid -s UUID/dev/sdb1: UUID="d3414a55-9eee-4df1-a636-07c195bbd85c"/dev/sda1: UUID="33d85eb3-1450-4c55-94f5-a437ad57a36c"/dev/sdc1: UUID="4a279854-1d28-458d-8390-6785e101b1a3"root@mymysql0:~#

The blkid program is the command-line interface to working with libblkid(3) library. It can determine the type of content (e.g. filesystem, swap) a block device holds, and also attributes (tokens, NAME=value pairs) from the content metadata (e.g. LABEL or UUID fields).

My existing /etc/fstab contains the following existing information for the primary disk and it’s blkid:

root@mymysql0:~# cat /etc/fstab /etc/fstab: static file system information.
UUID=33d85eb3-1450-4c55-94f5-a437ad57a36c / ext4 errors=remount-ro 0 1

I will need to add an additional line to represent the new disk. For example, to get the blkid information on sdd1 so we may use this, we’d mount it like so:

root@mymysql0:~# blkid /dev/sdd1/dev/sdd1: UUID="a218a0a0-587e-4152-8e51-90ae6837cb1f" TYPE="ext4" PARTUUID="1a9df8d1-01"

Now I can add the following to the /etc/fstab file which will ensure the disk will be mounted on reboot (do not use the data below, every disk has a unique blkid UUID):

UUID=a218a0a0-587e-4152-8e51-90ae6837cb1f /backup ext4 errors=remount-ro 0 1

Once you’ve savedthe file, just type mount /backup to finish your disk mount creation and look at the disk:

root@mymysql0:~# mount /backup
root@mymysql0:~# df -h /backup/Filesystem Size Used Avail Use% Mounted on
/dev/sdd1 1006G 77M 955G 1% /backup

Touch a file to test all is working, and you’re set:

root@mymysql0:/backup# touch test
root@mymysql0:/backup# ls -altotal 24drwxr-xr-x 3 root root 4096 Oct 15 17:57 .drwxr-xr-x 24 root root 4096 Oct 15 17:14 ..drwx------ 2 root root 16384 Oct 15 16:51 lost+found-rw-r--r-- 1 root root 0 Oct 15 17:57 test
root@mymysql0:/backup#

Final Notes

There’s a ton more to learn about storage on Azure beside just adding disks. To get started learning about all the different options for caching, securing and sharing disks, check out the documentation on the website.

Originally published at jaydestro.org on October 15, 2018.