ASP.NET Core 9: How to implement the Request Compression in .NET 9
In this article, we will implement a request compression in ASP.NET Core from the client application. In ASP.NET Core the built-in support for the response compression is available but there is not support available for the request compression. We can perform the request compression by implementing the custom middleware.
Why is the Request Compression required?
In the modern application development, there exist the frequency of uploading large file or large JSON bodies to the server from the client applications. When the client wants to upload such large files or JSON data then there is possibility that the out-of-memory exception is raised or the client application or even sometimes the server-side application may be crashed or hanged. Naturally, in such case, we need to plan for reducing the data to be transferred from the client application to server application, but this is not always possible. E.g. if the client wants to upload a large file or JSON for AI related work to server-side app then it is challenging to chunk or break such files or data. This is where request compression is required. To a conclusion, we can use the request compression in following situation:
- Reduce the payload size for the large requests containing files or JSON data.
- Improving communication over a low-bandwidth network.
- Saving the processing time for a large data transfer.
Figure 1: The Implementation
How to implement Compression in .NET 9?
In .NET the compression can be performed using the following classes:
- GZipStream:
- This class is a part of System.IO.Compression namespace.
- This class provides compression and decompression functionality using the GZIP data format.
- The GZIP is widely supported, and it compresses files without losing any information.
- Since GZIP is lossless when it is decompressed the file or data is perfectly restructured.
- You should not use the GZipStream for already compressed data.
- For using GZipStream, the stream must be read sequentially.
- This is used in case of compressing files, memory streams, HTTP request/response bodies
- DeflateStream
- This class is a part of System.IO.Compression namespace.
- This is used in case of embedded formats e.g. ZIP
- StringContent: Used in case the contents as string e.g. JSON, Plain Text.
- StreamContent: Used when the contents are stream.
- JsonContent: Used when the contents are to be serialized from object to JSON.
- ByteArrayContent: used when the contents are Binary array.
- MultipartFormDataContent: Used in case of File uploading
- FormEncodedUrlContent: Used in case of form data
The Implementation
Step 1: Open Visual Studio 2022 and create a Blank Solution named Net9Compression. In this solution, add a new ASP.NET Core 9 API project named Core_CompressionAPI. Add the folder named Files in this project. In this API project, add a new folder named Models. In this folder add a new class file named FileData.cs. In this class file, add a class code as shown in Listing 1.
namespace Core_CompressionAPI.Models
{
public class FileData
{
public string? FileName { get; set; }
public string? Content { get; set; }
}
}
Listing 1: The FileData class
This class will be used to Read the FileName and Content received from the client's HTTP Request.
Step 2: In the API project, add a new folder named Middlewares. In this folder, add a new class file named CompressionMiddleware.cs. In this file we will add a CompressionMiddleware class. In this class we will add the middleware to decompress the received data as well as file. Listing 2 shows the code for the middleware.
using System.IO.Compression;
namespace Core_CompressionAPI.Middlewares
{
public class CompressionMiddleware
{
private readonly RequestDelegate _next;
public CompressionMiddleware(RequestDelegate next)
{
_next = next;
}
public async Task InvokeAsync(HttpContext context)
{
var contentEncoding = context.Request.Headers["Content-Encoding"].ToString();
context.Request.Body = contentEncoding switch
{
"gzip" => new GZipStream(context.Request.Body, CompressionMode.Decompress),
"deflate" => new DeflateStream(context.Request.Body, CompressionMode.Decompress),
_ => context.Request.Body
};
await _next(context);
}
}
}
Listing 2: CompressionMiddleware class
The middleware will decompress the received request data based on the received encoding either gzip core deflate.
Step 3: Modify the Program.cs add the code for registering the Middlewares for Static files so that uploaded data as well as files will be stored in Files folder. In the Program.cs, we will also add the code for endpoints for Post request for data as well as files. Listing 3 shows the code for Program.cs.
using System.Text;
using System.Text.Json;
using Core_CompressionAPI.Middlewares;
using Core_CompressionAPI.Models;
using Microsoft.AspNetCore.Http.Features;
using Microsoft.Extensions.FileProviders;
var builder = WebApplication.CreateBuilder(args);
// Add services to the container.
// Learn more about configuring OpenAPI at https://aka.ms/aspnet/openapi
builder.Services.AddOpenApi();
builder.Services.AddCors(options =>
{
options.AddPolicy("cors",
builder => builder.AllowAnyOrigin()
.AllowAnyMethod()
.AllowAnyHeader());
});
builder.WebHost.ConfigureKestrel(options =>
{
options.Limits.MaxRequestBodySize = 1000 * 1024 * 1024; // 1000MB
});
builder.Services.Configure<FormOptions>(options =>
{
options.MultipartBodyLengthLimit = 1000 * 1024 * 1024; // 1000MB
});
var app = builder.Build();
// Configure the HTTP request pipeline.
if (app.Environment.IsDevelopment())
{
app.MapOpenApi();
}
app.UseHttpsRedirection();
app.UseCors("cors");
// Serve default wwwroot
app.UseStaticFiles();
// Serve your custom Files folder
app.UseStaticFiles(new StaticFileOptions
{
FileProvider = new PhysicalFileProvider(
Path.Combine(builder.Environment.ContentRootPath, "Files")),
RequestPath = "/Files"
});
app.UseMiddleware<CompressionMiddleware>();
app.MapPost("/api/upload/data", async (HttpContext context) =>
{
try
{
var bodySizeFeature = context.Features.Get<IHttpMaxRequestBodySizeFeature>();
if (bodySizeFeature != null && !bodySizeFeature.IsReadOnly)
{
bodySizeFeature.MaxRequestBodySize = null; // Disable limit
}
using var reader = new StreamReader(context.Request.Body, Encoding.UTF8);
var json = await reader.ReadToEndAsync();
var fileData = JsonSerializer.Deserialize<FileData>(json);
string fileName = fileData?.FileName;
if (string.IsNullOrEmpty(fileName))
{
throw new InvalidOperationException("FileName is missing from the uploaded data.");
}
var filePath = Path.Combine(builder.Environment.ContentRootPath, "Files", fileName);
// Save the JSON content from FileData instead of copying an empty stream
await File.WriteAllTextAsync(filePath, fileData.Content); // Changed back to save original JSON content
await context.Response.WriteAsync("File uploaded successfully.");
}
catch (Exception ex)
{
await context.Response.WriteAsync($"File uploaded failed: {ex.Message}.");
}
});
app.MapPost("/api/upload/file", async (HttpContext context) =>
{
var random = new Random();
var filePath = Path.Combine(builder.Environment.ContentRootPath,"Files", $"File-{random.Next(100)}.json");
await using var fileStream = File.Create(filePath);
await context.Request.Body.CopyToAsync(fileStream);
await context.Response.WriteAsync("File uploaded successfully.");
});
app.Run();
Listing 3: Program.cs
The endpoint /api/upload/data is used to read the JSON data and write it into the JSON file created in the Files folder. The endpoint /api/upload/file is used to read the received stream from the request and write them into the new file created in the Files folder. The reason behind having the /api/upload/file endpoint is when the client sends the compressed large files e.g. file more than 100MB in size, then instead of reading its contents and write them into new file its better read a stream decompress it and write into a new file. This will save memory and processing cost on the server for large files.
Step 4: In the solution, add a new Blazor Web Assembly project named Blaz_client. In this project, add a new folder named CompressionService. In this folder, add a new class file named CompressionAgent.cs. In this class file, add the code for CompressionAgent class. This class is derived from the HttpContent class. The class overrides SerializeToAsync() method that will compress the data from the JSON file to be uploaded by the Blazor Client application. Listing 4 shows the code for the CompressionAgent class.
using System.IO.Compression;
using System.Net;
namespace Blaz_Client.CompressionService
{
public class CompressionAgent : HttpContent
{
private readonly HttpContent _originalRequestContent;
private readonly string _encodingType;
public CompressionAgent(HttpContent originalContent, string encodingType)
{
_originalRequestContent = originalContent ?? throw new ArgumentNullException(nameof(originalContent));
_encodingType = encodingType ?? throw new ArgumentNullException(nameof(encodingType));
// Read Header Keys and Values
foreach (var header in _originalRequestContent.Headers)
{
Headers.TryAddWithoutValidation(header.Key, header.Value);
}
Headers.ContentEncoding.Add(_encodingType);
}
protected override async Task SerializeToStreamAsync(Stream stream, TransportContext? context)
{
// Compress the original content based on the encoding type
Stream compressesStream = _encodingType switch
{
"gzip" => new GZipStream(stream, CompressionMode.Compress, true),
"deflate" => new DeflateStream(stream, CompressionMode.Compress, true),
_ => throw new NotSupportedException($"Encoding type '{_encodingType}' is not supported.")
};
await _originalRequestContent.CopyToAsync(compressesStream);
await compressesStream.FlushAsync();
}
protected override bool TryComputeLength(out long length)
{
length = -1; // Length is not computable for compressed content
return false;
}
}
}
Listing 4: The CompressionAgent class
The constructor of the CompressionAgent class will read the HTTP Request Body contents using the HttpContent class the encoding type represents the content encoding e.g. gzip or deflate so that the data will be compressed based on the encoding.
Step 5: In the Pages folder, add a new Razor Component named FileUploadComponent.razor. In this component we will have methods as follows:
- PostCompressedJsonAsync:
- This method will read the JSON file contents, these contents are compressed using the CompressionAgent class. These contents will be posted to the API.
- CompressAndSendLargeFileAsync:
- This method will read the file as stream. This stream will be compressed using GZipStream class. This compressed stream will be stored in the StreamContent so that it will be posted to the API using the HTTP POST request.
- UploadSelectedFileData:
- This method will be bind with the InputFile component. This method will be executed when the file for uploading is selected. This method will read file contents. so that they can be compressed. One important thing is that for the sake of performance optimization, we are limiting the file contents to be uploaded as 10 MB. This method invokes the PostCompressedJsonAsync method to post the compressed contents to the API.
- UploadLargeFile:
- Like UploadSelectedFileData method explained above, this method is also bound to the InputFile component to select the file. The only difference is, in this method we are reading the file as stream and posting to the API using the CompressAndSendLargeFileAsync method. An important point here is that we are setting the file size as 200 MB.
@page "/fileuploadcomponent"
@using System.Text.Json
@using System.Text
@using Blaz_Client.CompressionService
@using System.IO.Compression
@using System.Net.Http.Headers
<h3>File Upload Component</h3>
<div class="container">
<div class="form-group">
<label for="encodingType">Select Encoding Type:</label>
<InputSelect @bind-Value="selectedEncoding" class="form-control">
@foreach (var encoding in encodingType)
{
<option value="@encoding">@encoding</option>
}
</InputSelect>
</div>
<table class="table table-bordered table-striped">
<tbody>
<tr>
<td>File Data Upload</td>
<td>
<InputFile OnChange="UploadSelectedFileData"></InputFile>
</td>
</tr>
<tr>
<td colspan="2"></td>
</tr>
<tr>
<td colspan="2"></td>
</tr>
<tr>
<td colspan="2"></td>
</tr>
<tr>
<td>Large File Upload</td>
<td>
<InputFile OnChange="UploadLargeFile"></InputFile>
</td>
</tr>
</tbody>
<tfoot>
<tr>
<td>
@statusMessage
</td>
</tr>
</tfoot>
</table>
</div>
@code {
List<string> encodingType = new List<string>
{
"gzip",
"deflate"
};
private string selectedEncoding = "gzip";
string statusMessage = string.Empty;
private async Task UploadSelectedFileData(InputFileChangeEventArgs e)
{
try
{
var file = e.File;
var fileName = file.Name;
// Read te File Content into JSON
using var stream = file.OpenReadStream(maxAllowedSize: 10 * 1024 * 1024); // 10 MB limit
using var reader = new StreamReader(stream);
var jsonContent = await reader.ReadToEndAsync();
var payload = new { FileName = fileName, Content = jsonContent };
// Send the JSON content to the server
var fileUploadResult = await PostCompressedJsonAsync("https://localhost:7287/api/upload/data", payload);
if (fileUploadResult)
{
statusMessage = "File uploaded successfully!";
}
else
{
statusMessage = "File upload failed.";
}
}
catch (Exception ex)
{
statusMessage = $"Error: {ex.Message}";
}
}
/// <summary>
///
/// </summary>
/// <param name="url"></param>
/// <param name="data"></param>
/// <returns></returns>
async Task<bool> PostCompressedJsonAsync(string url, object data)
{
bool isFileUploaded = false;
var json = JsonSerializer.Serialize(data);
var content = new StringContent(json, Encoding.UTF8, "application/json");
var compressed = new CompressionAgent(content, selectedEncoding);
using var client = new HttpClient();
var response = await client.PostAsync(url, compressed);
var responseCode = response.EnsureSuccessStatusCode();
return isFileUploaded = responseCode.IsSuccessStatusCode;
}
private async Task UploadLargeFile(InputFileChangeEventArgs e)
{
try
{
var file = e.File;
var fileName = file.Name;
// Read te File Content into JSON
using var stream = file.OpenReadStream(maxAllowedSize: 200 * 1024 * 1024); // 200 MB limit
// Send the JSON content to the server
var fileUploadResult = await CompressAndSendLargeFileAsync("https://localhost:7287/api/upload/file", stream, fileName);
if (fileUploadResult)
{
statusMessage = "File uploaded successfully!";
}
else
{
statusMessage = "File upload failed.";
}
}
catch (Exception ex)
{
throw;
}
}
/// <summary>
/// USe This method to send a compressed file to the server.
/// </summary>
/// <param name="url"></param>
/// <param name="filePath"></param>
/// <returns></returns>
public async Task<bool> CompressAndSendLargeFileAsync(string url, Stream stream, string fileName)
{
bool isFileUploaded = false;
using var compressedStream = new MemoryStream();
using var gzip = new GZipStream(compressedStream, CompressionMode.Compress, leaveOpen: true);
await stream.CopyToAsync(gzip);
await gzip.FlushAsync();
compressedStream.Position = 0;
var streamContent = new StreamContent(compressedStream);
streamContent.Headers.ContentEncoding.Add(selectedEncoding);
streamContent.Headers.ContentType = new MediaTypeHeaderValue("application/json");
using var client = new HttpClient();
var response = await client.PostAsync(url, streamContent);
var responseCode = response.EnsureSuccessStatusCode();
return isFileUploaded = responseCode.IsSuccessStatusCode;
}
}
Listing 5: The FileUploadComponent
We are ready with the Blazor Client. Modify the NavMenu.razor file to define navigation for the fileuploadcomponent as shown in Listing 6.
<div class="nav-item px-3">
<NavLink class="nav-link" href="fileuploadcomponent">
<span class="bi bi-list-nested-nav-menu" aria-hidden="true"></span> File Upload
</NavLink>
</div>
Listing 6: The Navigation for File Upload Component
Run the application and upload JSON file, you will see that the Files folder in the API project will be having the files created.
The code for this article can be downloaded from this link.
Conclusion: The compression is one of the most useful and beneficial features to manage the performance while handling Large data.
