Have You Tried Azure Storage?
Hello,
I hope you are well. In this article, I will explain different types of Azure Storage. My original intention was to cover Azure Functions as well, since it would be nice for you to create scenarios for storage services and Azure Functions, and to project them. However, it is a comprehensive subject as it is. But no problem. 😊 I will be discussing Azure Functions in my next article.
Here, I will be sharing with you the GitHub link of a project where I use Azure service types and Azure Functions together.
I wish you a pleasant and useful adventure.
Shall we start? 😊
What is the Azure Storage Services?
“Azure Storage is a cloud storage solution for modern data storage scenarios,” as reported in the Microsoft Azure documentation.
Azure Storage is a cloud storage solution for modern applications that is scalable, highly available, secure, durable, and based on availability to meet the needs of its customers.
Its pay-as-you-go model provides you with the flexibility to pay only as much as you use.
Storage capacity is virtually unlimited. It offers many options by supporting different clients, such as .NET, Ruby, Java, etc.
You must have an Azure Storage account to access any of the storage services. To do this, first create an Azure account. Let me briefly explain:
How to create a storage account?
In the Azure portal ➜ click on Storage Accounts ➜ then on Add.
Azure Store Explorer
Carrying out the work to be done on the data held in Azure Storage over the cloud can be costly in terms of the pay-as-you-go method. Ultimately, we may want to add, delete, query, and briefly test many data points in the development process, and we may not want to pay for our needs in this whole process. At this point, Azure Storage Explorer is an effective solution. In short, you use Azure Storage Explorer to manage the contents of your Storage account in your local area while in the development process, free of charge. This allows you to easily work across different operating systems such as Windows, Mac, and Linux. It gives you several ways to connect with your Storage accounts. For example, you can connect and manage your local storage and accounts associated with the Azure subscription. I used local storage in my sample project, where I shared the GitHub link. I won’t go into how to connect our current Azure account to Azure Storage Explorer because the scope of the article is getting too big.
Azure SDK
We also need the Azure SDK to use Azure services in our development environment. Azure SDK is a tool based on Visual Studio that allows us to interact with Azure services.
Here’s briefly how to install it:
Visual Studio ➜ Tools ➜ Get Tools and Features ➜ Visual Studio Installer ➜ Azure Development
That’s all.
Well, we said, “Let’s use Azure Storage.”
Let’s use it. But why?
Azure Storage features and benefits
So, how does storing data in the cloud affect businesses or people in general?
The first and foremost advantage is that it does not require any hardware or physical space. When you don’t need any infrastructure, you also save money. You can grow or shrink your storage according to your needs.
- Durable and Highly Available
The most important thing for us is data. The stored data is replicated and kept in different geographical locations. Therefore, in case of any hardware failure, your copies, hence your data, will be safe. In addition, in the event of a natural disaster, your data will remain safe, and you can take advantage of them quite easily.
- Scalable
Scalability ensures access to data at the same time, regardless of the number of clients. Data scales automatically to meet the highest demands.
- Secure
All our data stored in Azure is encrypted. Azure Storage uses a shared key model to authenticate a user. Using a shared access signature (SAS) can restrict access to data.
- Accessible
You can access your data stored in Azure Storage in different ways from anywhere in the world. It can be HTTP or HTTPS; you can write your code on Azure PowerShell or Azure CLI. It also provides resources for various programming languages.
Now that you know why it’s important to store data in the cloud, let’s see what Azure Storage is.
Azure Storage Services by Data Types
In fact, in this article, I will discuss in detail the storage types that can be accessed with the API. Do not worry, I will briefly touch on the types designed for VMs. 😉
Azure Table Storage
It is schemaless, meaning that the data has no fixed structure. It is a derivative of the NoSQL database. You can easily store datasets that do not require any joins or foreign keys. You can denormalize them for faster access. They will also scale as demand for tables increases. Table names must be unique to your account and no longer than 63 characters.
It will allow you to make queries for similar volumes of data at a lower cost than traditional SQL methods. In addition, NoSQL approaches yield very favorable results in queries made with the OData protocol.
There is no transaction support in Azure Table Storage. This does not mean that databases that have adopted the NoSQL approach do not have transaction support. MongoDB is an example of this; that is, it has adopted the NoSQL approach and has transaction support.
- Account
This is the storage account where you want to save your tables.
https://< your storage account>.table.core.windows.net/
Using this address, you can provide direct access or query via the URL with the OData protocol. It also supports LINQ.
It is an object that consists of properties and is a whole that models that object. An entity has a RowKey, a Partition Key I, and a timestamp. Because every entity must inherit from TableEntity in Azure Storage.
like that → public class Person: TableEntity
✓ RowKey: It can be thought of as a Primary Key. It is a unique value.
✓ PartitionKey: It is used for grouping and fast searching of entities. For example, in my example project, I used the City property as the partition key in the Person table.
✓ TimeStamp: It automatically keeps the date when the data was added.
- Table
Tables are collections of entities. Since they adopt the NoSQL approach, the entities to be kept in do not have to be uniform. It corresponds to the MongoDB collection in NoSQL.
Properties
Properties are key-values. The property name must not exceed 255 characters and is case-sensitive. The following types are supported for properties:
Common Language Runtime type
Byte[], Bool, DateTime, Double, Guid, Int32 or int, Int64 or long
Microsoft Azure Table Storage is designed to store NoSQL structured data. Storage is massively scalable and inexpensive to store data. However, it becomes more expensive if you access the files frequently.
Microsoft Azure is useful if you find SQL too costly and can manage without its structure and architecture.
You can use it by installing the Microsoft.Azure.Cosmos.Table NuGet package in your project.
You can implement a repository for crud operations in Azure Table Storage as follows.
Azure Blob Storage
The definition of blob is “Binary Large Object.”
We can define the blob concept as almost anything consisting of bits and bytes. so, in a sense, we can think of a blob as a data block. These blobs are stored in Containers in the Storage Account. From here, we can deduce that Blob Storage architecture has three layers. Storage Account, Container, and Blob trio. You can have multiple Blob Containers in your Azure Storage Account in your Azure Structure. You can store your data in them. For example, I can put the data I want from outside (mp3, png, jpg, pdf, etc.) into this blob element, regardless of the contents of the box. Why? Because it streams (binary) files and hence the related file can be of any type. So, the data type and size don’t matter. There is no limit to the number of containers or blobs. However, a Storage Account can have a maximum of 500 TB in total, there is the restriction. On the other hand, each of these blobs or items has a unique address. Therefore, when we want to receive any data, we can receive it directly and quickly without any problems. The point we need to pay attention to here is that we can see and find exactly where these data are.
Objects in Blob Storage can be accessed from anywhere in the world via HTTP or HTTPS. Users or client applications can access blobs via URLs, Azure Storage REST API, Azure PowerShell, Azure CLI, or an Azure Storage client library. The storage client libraries are available for multiple languages, including .NET, Java, Node.js, Python, PHP, and Ruby. So, when we want to make the data publicly available or store application data privately, we can have a Blob Storage address.
In many cases, you can choose Blob Storage;
- to keep files directly in the cloud,
- to create video and audio streams,
- to backup, restore, recover, or archive data in case of disasters,
- for data doping in analysis studies.
Rather, it is used to store and scale files in a single place in applications that are raised as more than one instance. Thus, the files are centralized, and the level of manageability is increased. If you’re familiar with AWS S3, containers work pretty much the same as S3 buckets. You can store any number of blob files up to a total size of 500 TB and apply security policies such as S3.
Blobs are divided into three:
→ Block Blobs
The file is not saved as a whole, but by fragmenting it block by block, and only after you complete the upload, the file/blob is combined into one piece. Each block takes a maximum of 100 MB of data, and a blob can have a maximum of 50 thousand blocks. This corresponds to 5 terabytes of data in a single blob.
When the added file names or contents are desired to be changed, no change is made; the relevant file is overwritten. “Append Blobs” should be preferred to keep modifiable files.
→ Append Blobs
It consists of blocks like block blobs. But it is optimized for insertions. For example, files with log.txt extensions, whose content is constantly changing where logging information is kept.
→ Page Blobs
A page block consists of pages. A page is 512 bytes, and the blob size can be up to 1 TB. This is usually used for fast reads and writes.
Lastly, I will mention the .net development environment. You need to install the Azure.Storage.Blobs library in the application to use it in your applications.
You can perform operations in Blog Storage as follows:
Azure Queue Storage
The purpose of message queue systems is to securely and asynchronously queue a message received from one source and transfer it to another. The asynchronous feature provides a structure that does not require the sender and receiver to be online at the same time, thus reducing the server cost. Azure Queue Storage is one of the message queue systems offered by Microsoft Azure.
The size of the message sent to Azure Queue Storage can be up to 64 KB, and a queue can contain millions of messages. The default lifetime of each message is 7 days. Messages are automatically deleted after 7 days. You can change the message lifetimes in a custom way or keep them in the queue for life, provided that you give the value -1. Messages are not guaranteed to be processed in any particular order. (FIFO is not valid.)
The subscription system (Subscribe) is not supported. Each message can only have one recipient.
It can be addressed using the following URL format:
https: // <storage account>.queue.core.windows.net /<queue>
Azure Queue Storage Concepts:
→ Storage Account: This is your Azure Storage account containing your queue storage, and we can create multiple queues within this Storage Account.
→ Queues: They are structures in which messages are sent and kept. There are some standards we must follow when naming queues:
- The queue name must begin with a letter or number and can contain only letters, numbers, and the hyphen (-) character.
- The first and last letters of the queue name must be alphanumeric. The hyphen (-) character cannot be the first or the last character. Consecutive dashes are not allowed in the queue name.
- All letters in the queue name must be lowercase.
- The queue name must be between 3 and 63 characters long.
→ Message: A message can be sent in any format up to 64 KB.
When we create an Asp.NET Core application, we need to add the Azure.Storage.Queues NuGet package to our application. And in the development environment, you can perform operations in Queue Storage as follows:
Azure File Storage
Microsoft Azure File Storage is the second storage type that was designed to support the needs of the Azure VM environment. That storage is, in essence, a network share. You can store files there that can be accessed from different Virtual Machines.
It allows the ability to read and write shared files. It is used for file sharing in in-house applications. The drive connected to the storage can be mounted on the personnel computers, and all transfers made over this drive facilitate the data transfer to Azure Storage. To mount Azure File Storage on your computer:
Azure portal➜ Overview ➜ File shares.
Azure Disk Storage
Microsoft Azure Disk Storage is based on Page Blobs. It is a service that allows you to create disks for your virtual machines. A disk created in Disk Storage can be accessed from only one virtual machine. In other words — it is your local drive. Yes, it’s that simple.
Here, you have two options for the speed of your disks:
- HDDs that are cheap but slow are called standard storage.
- SSDs that are fast but expensive are called premium storage.
And two options for disk management:
- Unmanaged disk: You should manage the disk storage and corresponding account yourself.
- Managed disk: Azure does everything for you. You need to select only the size of the disk and the desired type (standard or premium)
I’ve left an example of Azure Storage in my GitHub Profile here.
https://github.com/adessoTurkey-dotNET/MT.AzureStorageCreateFile
I hope it will come in handy and thank you for reading.
Sağlıcakla kalın. :)
https://learn.microsoft.com/en-us/azure/storage/common/storage-introduction
https://intellipaat.com/blog/tutorial/microsoft-azure-tutorial/azure-storage/?US
https://cloudacademy.com/blog/azure-storage-service-overview/
https://www.sqlshack.com/different-azure-storage-types-file-blob-queue-table/
https://www.msp360.com/resources/blog/microsoft-azure-storage-types-explained/
https://www.edureka.co/blog/azure-storage-tutorial/
