Uploading File to Azure BLOB using Node.js and Express-FileUpload

 Uploading File to Azure Blob using Node.js and Express-FileUpload 

Recently, I was working on an assignment for storing of the  information captured from the client application on the Azure cloud. This application is developed using JavaScript Full-Stack with Express and Node.js on the server-side. I was supposed to accept images from client and upload them to Azure Blob Storage using Node.js application.  To implement this, I have explored various ways like Multer package and Express-FileUplaod. After doing some research I have implemented the requirement. But then I have decided to put my thoughts in the form of this article.  In fact, I have already written an article on similar requirements which you can read it from this link

In this article, we will see how to use an express-fileupload to upload file to Node.js Server Application  and further uploading it in Azure Blob. 

The figure 1, will explain an idea of the application implementation





Figure 1: The idea of file upload application implementation 

The numbering used in the image explains all the steps for implementation of the application. 

Express-FileUpload

The express-fileupload is a simple middleware that is used to upload files to the server when using Node.js Express applications. This middleware is added into the express HTTP request processing object and it reads the posted  file from the client using request object and processes it. 

Azure Blob Storage

The uploaded file will be further stored in the Azure Blob Storage. The Azure Blob storage is a Microsoft's Object Storage solution for the Cloud. The Azure Blob storage is capable for storing massive amount of unstructured data like images, documents, audio, video files, etc.  To access the Azure Blob storage, we need an Azure Subscription. The Azure subscription can be created using Azure Free account by using this link.  

Creating a Storage Account  to Access Blob Storage

Step 1: To create a storage account, we need to create Resource Group. Open the Azure Portal and login on it.  To create a resource group, click on the Resource Group from the home page as shown in Figure 2

The Resource Group is an logical group of all the resources (App, Database, storage) you create on Azure.

Figure 2: Creating a resource group  

Once the Resource Group link is clicked, the Resource Group page will be displayed. On this page, click on Create link. This action will show Create Resource Group page. On this page enter the resource group information like Subscription, Resource Group Name, Region, etc. and click on Review + create  button as shown in figure 3.



Figure 3: Creating a Resource Group  

Step 2: . Select the Storage accounts from the Home page as shown in figure 4



Figure 4: The Storage Account      

Once the Storage accounts is clicked, the page for creating new storage account is displayed as shown in figure 5



Figure 5: Creating Storage account          

Click on the Create link and this will show the Create a storage account page.  On this page enter details like Subscription, Resource Group  (we have created it in Step 1), Storage account name, Region etc. and then click on the Review + create button. The Storage account will be created.  The Storage account name we are creating is msitart. The details are shown in figure 6



Figure 6: Create storage account   

Once, the storage account is created, click on it, it will show all storage properties, e.g. Containers, Queues, Tables, File Shares, Access Keys, etc. as shown in the figure 7



Figure 7: Storage account properties 

On this page, click on Access Keys, these keys are used to authenticate the client application to access Storage Account. Click on Show Keys link, this will show Connection string as shown in figure 8



Figure 8: The Connection Strings  

Copy the Connection String and paste it in the notepad, we will be using this in the Node.js application.

As shown in Figure 7, click on Containers link and create anew container of name imagecontainer. (Make sure that the container name is in lower case.)

Creating the Node.js application

Step 3: Create a new folder of name expressfileupload. Open this folder in Visual Studio Code. Open the Command Window (or Terminal window in Linux and Mac) and navigate to the expressfileupload folder. We need to create a package.json file and install various package which we will be using for the application development.

Run the following command to create package.json file in the folder

npm init -y

Run the following command to install required packages

npm install --save express express-fileupload into-stream dotenv azure-storage

These packages are used for the following requirements

  1. dotenv, since we will be storing the Azure Storage Account Connection String in Node.js environment file we need this package to configure the Node.js application to read keys from the environment file.
  2. into-stream, This package is used to convert promise, string, array, buffer, arraybuffer, object, etc. into a stream.
  3. azure-storage,  This is a Microsoft's Storage SDK for Node.js to access Azure Storage in Node.js applications.
Step 4: Modify the package.json to enable ES 6 Module syntax to import JS modules in the application code as shown in listing 1:

{
  "name": "expressfileupload",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "nodemon server.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "azure-storage": "^2.10.5",
    "dotenv": "^10.0.0",
    "express": "^4.17.1",
    "express-fileupload": "^1.2.1",
    "into-stream": "^7.0.0"
  },
  "type": "module"
}

 
Listing 1: Modifications in package.json to support ES 6 syntax for Module import

Step 5:  In the project add a folder and name it as files. This folder is used by the express-fileupload to  save the posted file from the client application.

Step 6: In the project add a new file and name it as '.env'. This is an environment file where we will define Name=Value pairs of the environment variables used by the Node.js application. It is recommended that, instead of hard-coding security critical values in the application code we should put them in environment file.

AZURE_STORAGE_CONNECTION_STRING= "YOUR CONNECTION STRING HERE"

Listing 2: An Environment file

Step 7: In the project add a new file and name it as server-blob.js. In this file add the code as shown in the listing 3

import express from "express";
import fileUpload from "express-fileupload";
import path from "path";
import { fileURLToPath } from "url";
import azureStorage from "azure-storage";
import intoStream from "into-stream";
import dotenv from "dotenv";

// 1.
const __filename = fileURLToPath(import.meta.url);
// 2.
const port = process.env.PORT || 7001;
// 3.
const instance = new express();

// 4.
const containerName = "imagecontainer";

// 5.
const __dirname = path.dirname(__filename);
// 6.
dotenv.config();
// 7.
instance.use(
  fileUpload({
    createParentPath: true,
  })
);
// 8.
const blobService = azureStorage.createBlobService(
  process.env.AZURE_STORAGE_CONNECTION_STRING
);
// 9.
instance.get("/", (req, res) => {
  res.sendFile(path.join(__dirname, "index.html"));
});
// 10.
instance.post("/fileupload", (request, response) => {
  // 10.1.
  if (!request.files) {
    return res.status(400).send("No files are received.");
  }

  // 10.2.
  const file = request.files.file;
  // 10.3.
  const path = __dirname + "/files/" + file.name;
  // 10.4.
  file.mv(path, (err) => {
    // 10.5.
    if (err) {
      return response.status(500).send(err);
    }
    // 10.6.
    return response.send({ status: "success", path: path });
  });
});
// 11.
instance.post("/blobupload", (request, response) => {
  if (!request.files) {
    return res.status(400).send("No files are received.");
  }

  // 11.1.
  const blobName = request.files.file.name;
  console.log(`Blob Name ${blobName}`);
  // 11.2.
  const stream = intoStream(request.files.file.data);
  console.log(`stream ${stream}`);
  // 11.3.
  const streamLength = request.files.file.data.length;
  console.log(`Length ${streamLength}`);
  // 11.4.
  blobService.createBlockBlobFromStream(
    containerName,
    blobName,
    stream,
    streamLength,
    (err) => {
      if (err) {
        response.status(500);
        response.send({ message: "Error Occured" });
        return;
      }

      response.status(200).send({message:
'File Uploaded Successfully'});
    }
  );
});
// 12.
instance.listen(port, () => {
  console.log(`Server is listening on port ${port}`);
});

Listing 3: The Application code

The code in listing 3 shoes that it uses ES 6 module import to import modules required for the application e,g, express, express-fileupload, into-stream, azure-storage,dotenv, etc. and further the code  has specifications as explained in following point. (Please note that following numbering matches with the comment applied on the application code)

  1. The support for __dirname is not available in Node.js with ES 6 support, so use the fileURLToPath() function to read the directory path.
  2. The port from which REST API will be accessible
  3. Defining instance of express
  4. Defining the container name where uploaded files will be stored
  5. The directory path, this will be the path for the  directory where the server is running
  6. Environment configuration to read keys from the .env file 
  7. configure the file upload middleware
  8. Connecting the BLOB Service using the Connection String read from the Environment Key from .env file
  9. The get request to respond index.html. This file contains HTML UI that will be posting the image file to server. 
  10. The post request to to accept posted file from the client  
    1. If the file not found then respond error
    2. Read the file
    3. The file path where the file will be stored on the server
    4. Copy the received file in 'files' folder
    5. If error occurred then send error response
    6. Send the success response
  11. Post request for accepting file from the client and uploading it on Blob storage 
    1. Read the file name received from the client 
    2. Convert the file into stream
    3. Read the Length of the file
    4. Upload the file from to the Blob
  12. Start listening the server on the port

Step 8: In the project add a new file and name it as index.html, add the code in this file as shown in Listing 4


<!DOCTYPE html>
<html>
  <head>
    <title>Uploading The File Using Express</title>
  </head>
  <body>
    <h1>Uploading The File Using Express</h1>
    <form method="POST" action="/blobupload" enctype="multipart/form-data">
      <input type="file" name="file" />
      <input type="submit" />
    </form>
  </body>
</html>

Listing 4: index.html

The index.html contains a HTML form with HTML input file element and a submit button. The file that is to be uploaded is selected using the input file element that has name attribute as file. we have used this attribute value in the server code to read uploaded file (see listing 3, code comment 11.1). When the form is submitted, the REST API with endpoint as blobupload will be requested with file posted to it.  

To run the application, run the following command from the command prompt

node server-blob.js

This will start the server on the port 7001. Open the browser and browse the address http://localhost:7001. The index.html will be loaded in the browser as shown in the figure 9



Figure 9: The index.html 

Click on the Choose File button, this will open the Open file dialog, from this dialog select the file to be uploaded, and then click in Submit button. The file will be uploaded, if the file upload is successful, the File Uploaded Successfully message will be displayed.


Visit back to the Azure portal and open the Containers as shown in figure 7. If you open the imagecontainer, you will see the file uploaded to Azure Bob Storage.

The Code for this article can be downloaded from this link.

Conclusion: The Express-FileUpload package is a great use for uploading files in Node.js application. Using Azure-Storage package we can easily upload files on Azure Blob Containers. This helps to build JavaScript Full-Stack apps for cloud easily.     


 

Comments

Popular posts from this blog

gRPC Services using .NET 5: Using Entity Framework Core with gRPC for Performing Database Operations

Using Session State in ASP.NET Core

Understanding Token Based Authentication in ASP.NET Core 3.1 using JSON WEB TOKENS