Microsoft Azure Storage Blob: Download multiple Blobs from the Azure Storage Blob Containers using SemaphoreSlim Class
Azure Blob Storage is the Object storage in the Cloud. This storage is optimized for storing massive amounts of unstructured data. We use the Blob storage for storing large-size files, images, videos documents, etc. The images or documents can be directly served to the browser. In this article, we will implement a simple .NET 8 Console Application to download multiple Blobs.
To download multiple Blobs we will be using the SemaphoreSlim class. This is one of the superb classes provided on .NET. This class is used to limit the number of threads that can access a resource or pool of resources concurrently. The SemaphoreSlim Class in C# is recommended for synchronization within a single app. A lightweight semaphore controls access to a pool of resources that is local to the application. With an instantiate of a semaphore, we can specify the maximum number of threads that can enter the semaphore concurrently. When we want more control over the number of internal threads that can access our application code, then we should use the SemaphoreSlim class in C#. Figure 1 explains the SemaphoreSlm class.
Figure 1: The SimaphoreSlim class
In the case of the application that is trying to download and process multiple documents from the Azure Bob storage, we must use an efficient approach to interact with Blob Storage and also manage the resources internally with synchronization. Earlier I published an article on uploading files to Blob, you can refer to it to create the Azure Resource Group, Storage Account, and Blob Container. You can access it from this link.
Step 1: Open the Visual Studio and create a new Console Application named CS_Azure_Blob_DownloadMultipleFiles targetted to .NET 8.
Step 2: In this project, add the Azure.Storage.Blobs Nuget Package. This package will provide access to Azure Blob Storage.
Step 3: Visit the Microsoft Aure Portal and Create a Resource Group named myresgp (if it is not available then choose a different name). In this resource group create a Storage Account named mytestdatastorageaccount and in this Storage Account create a Container named 'Image'. Upload some files in this container as shown in Figure 2
Figure 2: Uploading Files in Container
To download files from the Azure Blob Storage, our application should connect to the Storage using the Connection String. Copy the Connection String from the Storage Account as shown in Figure 3
Step 4: In the project, add a new class file named BLOBDownloader.cs. In this class file, we will write code for downloading the Blobs from the Storage. Listing 1 shows the code
using Azure.Storage.Blobs;
namespace CS_Azure_Blob_DownloadMultipleFiles
{
public class BLOBDownloader
{
private readonly BlobServiceClient _blobClient;
public BLOBDownloader(BlobServiceClient blobClient)
{
_blobClient = blobClient;
}
public async Task DownloadMultipleFilesAsync(string contName)
{
// Get the Path of the Project Folder
string? startupPath = Directory.GetParent(Directory.GetCurrentDirectory()).Parent?.Parent?.FullName;
// Create a BlobClient Object based on ContainerName
var blobConClient = _blobClient.GetBlobContainerClient(contName);
// Call the listing operation and return pages
var blobItemPages = blobConClient.GetBlobsAsync()
.AsPages();
// SemaphoreSlim with 8 THreads
var semaphoreSlim = new SemaphoreSlim(8);
var tasksBlobDownload = new List<Task>();
// Process each BLOB
await foreach (var blobItemPage in blobItemPages)
{
foreach (var blob in blobItemPage.Values)
{
// blocks the current thread until it can enter the System.Threading.SemaphoreSlim.
await semaphoreSlim.WaitAsync();
tasksBlobDownload.Add(Task.Run(async () =>
{
try
{
// Get the BLOB Name
var client = blobConClient.GetBlobClient(blob.Name);
// Create a BLOB in the DownloadedFiles folder
using var file = File.Create($"{Path.Combine(startupPath, "DownloadedFiles")}\\{blob.Name}");
await client.DownloadToAsync(file);
}
finally
{
semaphoreSlim.Release();
}
}));
}
await Task.WhenAll(tasksBlobDownload);
tasksBlobDownload.Clear();
}
}
}
}
Listing 1: The code for the BLOBDownloader Class
As shown in Listing 1, the DownloadMultipleFilesAsync() method accepts the Blob container name to create an instance of the BlobContainerClient class. Using this instance we can get the sequence of Blobs from the container using the GetBlobsAsync() method. We are returning these obs as pages. Furthermore, the code creates an instance of the SemaphoreSlim class specifies the number of threads used to perform Blob Read, and processes them to download these Blobs to a new folder. The WaitAsync() and Release() methods will manage the synchronization.
Step 5: Add the code shown in Listing 2 in Program.cs to use the BLOBDownloader class.
// See https://aka.ms/new-console-template for more information
using Azure.Storage.Blobs;
using CS_Azure_Blob_DownloadMultipleFiles;
Console.WriteLine("Download Multiple Blobs");
var connectionString = "ConnString";
var blobClient = new BlobServiceClient(connectionString);
var download = new BLOBDownloader(blobClient);
await download.DownloadMultipleFilesAsync("images");
Console.WriteLine("Completed");
Console.ReadLine();
Listing 2: The Program.cs
Make sure that you replace the ConnString with the connection string of the Storage Account copied from the folder.
Run the application. You will see that Blobs that you have uploaded into the container are downloaded in the DownloadedFiles folder which we have created as shown in Listing 1.
The code for this article can be downloaded from this link.
Conclusion: The SemaphoreSlim class is a supercool class that provides an effective mechanism to deal with resource-intensive operations with a great deal with synchronization.
