Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData
1.1.0
dotnet add package Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData --version 1.1.0
NuGet\Install-Package Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData -Version 1.1.0
<PackageReference Include="Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData" Version="1.1.0" />
paket add Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData --version 1.1.0
#r "nuget: Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData, 1.1.0"
// Install Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData as a Cake Addin #addin nuget:?package=Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData&version=1.1.0 // Install Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData as a Cake Tool #tool nuget:?package=Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData&version=1.1.0
Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData ![Nuget](https://img.shields.io/nuget/v/Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData.svg)
![Downloads](https://img.shields.io/nuget/dt/Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData.svg)
This package allows you to read Form Data values and files. Files streams are not read in binding model process and are provided to be used in user code.
How it works
Default ASP .NET Core behaviour reads all form contents including files during model binding process. File streams are fully drained to the end and their contents are saved in memory and/or disk before any user custom code.
This behaviour is not convenient when we want to treat files as streams. Possible cases:
- You process file streams consequentially. For example, you want to calculate file hash only.
- You want to read form string values first and validate its data before reading any files' content.
This package allows you to use default binding model except for files that will be available to user code later after model binding process.
There are some drawbacks:
- If you read IFormStreamedFile object from FormStreamedFileCollection then previous files' streams will become empty either you have read them earlier or not. It is not obvious behaviour.
- Swagger is not supported.
- FormOptions settings are not supported yet.
- RequestFormLimitsAttribute is not supported yet.
It is recommended to use standard behaviour if both conditions are met:
- You don't need streaming.
- You have enough resources to store incoming files' content of all simultaneous requests in memory and on disk.
In Benchmarking section you can see that new behaviour is faster in high performance stream processing use cases. It is strongly recommended to measure performance for your use case before using it in production environment.
Implementation
The implementation is based on Microsoft suggestion with addition:
- DisableFormValueModelBindingAttribute is presented as SetFormStreamedDataValueProviderAttribute which actually replaces value provider with new provider.
- Model binding is enabled. It means that all standard and custom binding to your properties of specific types from string values is supported.
- Model binding of property or parameter of type FormStreamedFileCollection is introduced. This type allows reading file contents consequentially by user code after model binding process.
- You can read Form Data with ReadFormStreamedDataAsync extensions method of HttpRequest.
New behaviour usage requirements
- Always provide form string values first.
- Always provide form file values last.
- Read file streams consequentially, that is at the beginning get first IFormStreamedFile object then read its stream to the end. Then get next IFormStreamedFile object and then read its stream to the end. When you get next IFormStreamedFile object the first file stream will be read automatically to the end and you will not be able to read it.
- Use only one property or one parameter of type FormStreamedFileCollection because it will always be one stream.
- Do not combine standard behaviour and this behaviour in one controller action. For example, do not use IFormFile and FormStreamedFileCollection in one action.
Installing
dotnet add package Byndyusoft.AspNetCore.Mvc.ModelBinding.FormStreamedData
Usage
Register model binder for FormStreamedFileCollection type.
services.AddControllers(o => o.AddFormStreamedFileCollectionBinder());
Set SetFormStreamedDataValueProviderAttribute for controller actions where file streams should be read. This will replace default value providers for Form Data with FormStreamedDataValueProvider.
Set FromFormStreamedDataAttribute for action parameters or request object properties that should be bound to form values. To read files' streams use only one property or one parameter of type FormStreamedFileCollection.
Examples:
public class NewRequestDto
{
public string Name { get; set; }
public int Age { get; set; }
public FormStreamedFileCollection StreamedFiles { get; set; }
}
[ApiController]
[Route("[controller]")]
public class FilesController : ControllerBase
{
private readonly FileService _fileService;
public FilesController(FileService fileService)
{
_fileService = fileService;
}
[HttpPost("SaveNew")]
[RequestSizeLimit(long.MaxValue)]
[SetFormStreamedDataValueProvider]
public async Task<ActionResult> SaveNewWay(
[FromFormStreamedData] NewRequestDto requestDto,
CancellationToken cancellationToken)
{
await foreach (var file in requestDto.StreamedFiles.WithCancellation(cancellationToken))
{
await using var stream = file.OpenReadStream();
await _fileService.SaveFileAsync(stream, file.FileName, cancellationToken);
}
return Ok();
}
[HttpPost("SaveNewByParameter")]
[RequestSizeLimit(long.MaxValue)]
[SetFormStreamedDataValueProvider]
public async Task<ActionResult> SaveNewWayByParameter(
FormStreamedFileCollection files,
CancellationToken cancellationToken)
{
await foreach (var file in files.WithCancellation(cancellationToken))
{
await using var stream = file.OpenReadStream();
var filePath = await _fileService.SaveFileAsync(stream, file.FileName, cancellationToken);
}
return Ok();
}
}
Benchmarking
Use cases and implementation
BenchmarkNet was used to measure new behaviour performance. 3 use cases were tested:
- File content hashing.
- Saving to disk.
- Uploading to localhost Minio S3 storage.
All three cases were implemented in TestApi project. Performance tests were implemented in (PerformanceTests)[tests/PerformanceTests] project.
There were used three values for TestFileSize parameter:
- Small - API receives 5 1Mb-sized generated files.
- Big - API receives 2 100Mb-sized generated files.
- Large - API receives 1 1Gb-sized generated files.
All these values can be changed in FileGeneratorSetting class.
Results
File content hashing use case
Method | TestFileSize | Mean | Error | StdDev | StdErr |
---|---|---|---|---|---|
HashOld | Small | 41.65 ms | 0.822 ms | 1.698 ms | 0.236 ms |
HashNew | Small | 15.39 ms | 0.058 ms | 0.045 ms | 0.013 ms |
HashOld | Big | 1,259.90 ms | 24.976 ms | 26.724 ms | 6.299 ms |
HashNew | Big | 485.09 ms | 10.955 ms | 30.899 ms | 3.221 ms |
HashOld | Large | 10,082.17 ms | 176.127 ms | 164.749 ms | 42.538 ms |
HashNew | Large | 2,469.44 ms | 44.506 ms | 110.007 ms | 12.964 ms |
Saving to disk use case
Method | TestFileSize | Mean | Error | StdDev | StdErr |
---|---|---|---|---|---|
SaveOld | Small | 24.87 ms | 0.493 ms | 1.102 ms | 0.142 ms |
SaveNew | Small | 14.91 ms | 0.107 ms | 0.083 ms | 0.024 ms |
SaveOld | Big | 836.72 ms | 39.259 ms | 113.272 ms | 11.561 ms |
SaveNew | Big | 501.68 ms | 11.450 ms | 32.669 ms | 3.370 ms |
SaveOld | Large | 8,952.73 ms | 178.932 ms | 513.388 ms | 52.673 ms |
SaveNew | Large | 7,913.74 ms | 139.790 ms | 149.574 ms | 35.255 ms |
Uploading to localhost Minio S3 storage use case
Method | TestFileSize | Mean | Error | StdDev | StdErr |
---|---|---|---|---|---|
UploadOld | Small | 184.0 ms | 62.66 ms | 41.45 ms | 13.11 ms |
UploadNew | Small | 446.3 ms | 140.33 ms | 92.82 ms | 29.35 ms |
UploadOld | Big | 5,068.9 ms | 653.25 ms | 432.08 ms | 136.64 ms |
UploadNew | Big | 51,659.3 ms | 1,413.90 ms | 935.21 ms | 295.74 ms |
UploadOld | Large | 26,848.8 ms | 1,585.32 ms | 1,048.59 ms | 331.59 ms |
UploadNew | Large | 259,196.0 ms | 7,761.91 ms | 5,134.02 ms | 1,623.52 ms |
Summary
- New behaviour is faster if file contents are being read rapidly. It can be observed in first two use cases.
- Standard (old) behaviour is faster if files streams are sent to S3. It is much faster to read all content before sending it to next service as another stream. BufferedStream can boost new behaviour speed but it will not be faster anyway.
- Use new behaviour only if you need to process files` content in high performance method or if you do not have enough resources to store incoming files' content of all simultaneous requests in memory and on disk.
Api site, performance test console application and Mino S3 server storage were launched on one machine. So these results may not reflect production environment performance. You are welcome to update them if performance is measured in production (or at least preproduction) environment.
Contributing
To contribute, you will need to setup your local environment, see prerequisites. For the contribution and workflow guide, see package development lifecycle.
Prerequisites
Make sure you have installed all of the following prerequisites on your development machine:
- Git - Download & Install Git. OSX and Linux machines typically have this already installed.
- .NET Core 3.1 or higher - Download & Install .NET Core 3.1.
Package development lifecycle
- Implement package logic in
src
- Add or adapt integration tests (prefer before and simultaneously with coding) in
tests
- Add or change the documentation as needed
- Open pull request in the correct branch. Target the project's
master
branch
Maintainers
Product | Versions Compatible and additional computed target framework versions. |
---|---|
.NET | net5.0 was computed. net5.0-windows was computed. net6.0 was computed. net6.0-android was computed. net6.0-ios was computed. net6.0-maccatalyst was computed. net6.0-macos was computed. net6.0-tvos was computed. net6.0-windows was computed. net7.0 was computed. net7.0-android was computed. net7.0-ios was computed. net7.0-maccatalyst was computed. net7.0-macos was computed. net7.0-tvos was computed. net7.0-windows was computed. net8.0 was computed. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. |
.NET Core | netcoreapp3.1 is compatible. |
-
.NETCoreApp 3.1
- No dependencies.
NuGet packages
This package is not used by any NuGet packages.
GitHub repositories
This package is not used by any popular GitHub repositories.
Version | Downloads | Last updated |
---|---|---|
1.1.0 | 182 | 3/20/2024 |
1.0.0 | 228 | 11/3/2023 |
1.0.0-tags-1.0.0-alpha2 | 65 | 11/3/2023 |
1.0.0-tags-1.0.0-alpha | 61 | 11/3/2023 |