DrewNoakes.Cassette 1.3.2

dotnet add package DrewNoakes.Cassette --version 1.3.2                
NuGet\Install-Package DrewNoakes.Cassette -Version 1.3.2                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="DrewNoakes.Cassette" Version="1.3.2" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add DrewNoakes.Cassette --version 1.3.2                
#r "nuget: DrewNoakes.Cassette, 1.3.2"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install DrewNoakes.Cassette as a Cake Addin
#addin nuget:?package=DrewNoakes.Cassette&version=1.3.2

// Install DrewNoakes.Cassette as a Cake Tool
#tool nuget:?package=DrewNoakes.Cassette&version=1.3.2                

Cassette

Build status Cassette NuGet version

Cassette is a simple and efficient content-addressable storage system for .NET 4.5 and .NET Core (netstandard1.3).

// Create a store, backed by the specified file system location
var cassette = new ContentAddressableStore(@"c:\cassette-data\");

// Store some content, obtaining its hash (content address)
Hash hash = await cassette.WriteAsync(writeStream);

// Later, use the hash to look up the content
Stream stream;
if (cassette.TryRead(hash, out stream, ReadOptions.Asynchronous | ReadOptions.SequentialScan))
{
    using (stream)
    {
        // Read the stored content via the returned read-only stream
        var buffer = new byte[4096];
        var bytesRead = await stream.ReadAsync(buffer, 0, buffer.Length);
        // ...
    }
}

Content-addressable storage (CAS) is a fast and efficient mechanism for storing and retrieving fixed data on disk.

Information is uniquely and unambiguously identified by the SHA-1 hash of its contents.

A significant advantage of CAS is its efficient use of storage media for data backups where a majority of files are identical, and so separate storage would be redundant.

For more information, read Wikipedia's CAS article.


Types

  • IContentAddressableStore exposes functionality of a cassette store
  • ContentAddressableStore is the concrete implementation
  • Hash holds the identity of a piece of content

Writing content

using (var stream = File.OpenRead(@"c:\content.jpg"))
{
    var hash = await store.WriteAsync(dataStream);
    Console.WriteLine("Stored with address: {0}", hash);
}

The hash is computed efficiently by observing content during buffered writes to a temporary location of the file system. Then:

  • if the content does not already exist, the file is atomically moved into location and marked as read-only
  • if the content does already exist, the temporary file is deleted and the store is unchanged

This means writes are idempotent. Repeated writes of the same content do not increase the size of the store on disk, though do incur disk IO.

Reading

Read operations atomically test for the availability of a file and open it for reading if present.

Multiple clients may read a file concurrently. Content may not be deleted via IContentAddressableStore.Delete while a read stream is open.

When TryRead returns true, client code must dispose the returned Stream.

Hash hash = ...;
Stream stream;
if (store.TryRead(hash, out stream))
{
    using (stream)
    {
        // Use the stream
    }
}

The performance of reads can be improved by specifying ReadOptions as described in sections async IO and access patterns and performance.

  • None indicates no special options. This is the default.
  • SequentialScan indicates data will be read sequentially.
  • RandomAccess indicates seek operations will be performed during reading.
  • Asynchronous indicates Stream.ReadAsync will be used.
store.TryRead(hash, out stream, ReadOptions.SequentialScan | ReadOptions.ReadAsync)

Asynchronous IO

Write operations happen using asynchronous IO and return awaitable tasks to prevent blocking calling code. Use of a CancellationToken allows immediate cancellation of the write operation.

Read operations occur using Streams which support asynchronous IO themselves via ReadAsync and CopyToAsync. When using these asynchronous methods you can improve performance by passing ReadOptions.Asynchronous to IContentAddressableStore.TryRead.

On Windows these asynchronous operations use IO Completion ports (MSDN, Dr. Dobbs). Other platforms may have implementations using suitable alternatives.

Access patterns and performance

When writing content to disk, cassette notifies the file system that data will be written sequentially and that no random-access seeking will occur. This allows the caching system to prepare pages of data efficiently which can significantly reduce latency.

When reading content from disk, users can get the same caching benefits by specifying ReadOptions.SequentialScan or ReadOptions.RandomAccess. This is optional however.

Deleting content

Content may be deleted from the store by calling Delete with the relevant hash.

Measuring content length

If the length of stored content is to be retrieved, it is most efficient to use TryGetContentLength which provides the length in bytes.

Enumerating content

The complete set of hashes is returned via GetHashes. This method computes the enumerable lazily by walking the file system so is thread-safe with respect to reads and writes. However it cannot be relied upon to behave deterministically if enumerating while content is being written or deleted. Whether new or deleted content is included in an enumeration whose processing spans the write/delete may or may not contain the affected content.

Encoding

Cassette supports optionally storing content using an encoding. The primary use case for this is to store pre-compressed data whereby the cost of compressing content is taken upfront once at write time, rather than for each read.

Content may be stored in multiple encodings. For example, an HTTP server may support both no encoding, or GZIP content/transfer encoding. Such an HTTP handler could request either encoding from the store depending upon request headers.

// Instantiate a content encoding
var gzipEncoding = new GZipContentEncoding();

// Write some (unencoded) content to the store, and request an encoded copy be stored
var hash = await store.WriteAsync(stream, encodings: new[] { gzipEncoding });

// Read the encoded content out directly
Stream gzipStream;
if (store.TryRead(hash, out gzipStream, encodingName: gzipEncoding.Name))
{
    using (gzipStream)
    {
        // Use the GZipped data directly
    }
}
Product Compatible and additional computed target framework versions.
.NET net5.0 was computed.  net5.0-windows was computed.  net6.0 was computed.  net6.0-android was computed.  net6.0-ios was computed.  net6.0-maccatalyst was computed.  net6.0-macos was computed.  net6.0-tvos was computed.  net6.0-windows was computed.  net7.0 was computed.  net7.0-android was computed.  net7.0-ios was computed.  net7.0-maccatalyst was computed.  net7.0-macos was computed.  net7.0-tvos was computed.  net7.0-windows was computed.  net8.0 was computed.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
.NET Core netcoreapp1.0 was computed.  netcoreapp1.1 was computed.  netcoreapp2.0 was computed.  netcoreapp2.1 was computed.  netcoreapp2.2 was computed.  netcoreapp3.0 was computed.  netcoreapp3.1 was computed. 
.NET Standard netstandard1.3 is compatible.  netstandard1.4 was computed.  netstandard1.5 was computed.  netstandard1.6 was computed.  netstandard2.0 was computed.  netstandard2.1 was computed. 
.NET Framework net45 is compatible.  net451 was computed.  net452 was computed.  net46 was computed.  net461 was computed.  net462 was computed.  net463 was computed.  net47 was computed.  net471 was computed.  net472 was computed.  net48 was computed.  net481 was computed. 
MonoAndroid monoandroid was computed. 
MonoMac monomac was computed. 
MonoTouch monotouch was computed. 
Tizen tizen30 was computed.  tizen40 was computed.  tizen60 was computed. 
Universal Windows Platform uap was computed.  uap10.0 was computed. 
Xamarin.iOS xamarinios was computed. 
Xamarin.Mac xamarinmac was computed. 
Xamarin.TVOS xamarintvos was computed. 
Xamarin.WatchOS xamarinwatchos was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
1.3.2 145 2/1/2024
1.2.0 2,291 1/5/2018
1.0.0 14,278 1/6/2017
1.0.0-rc8 836 11/25/2016
1.0.0-rc7 843 11/18/2016
1.0.0-rc6 799 10/12/2016
1.0.0-rc5 788 10/12/2016
1.0.0-rc4 795 10/12/2016
1.0.0-rc3 1,105 2/15/2016
1.0.0-rc2 1,087 2/13/2016
1.0.0-rc1 1,181 2/13/2016
0.3.0 1,377 7/15/2015
0.2.1 1,209 7/15/2015
0.2.0 1,075 5/19/2015
0.1.0 1,106 3/29/2015