PocketCsvReader 2.45.1

PocketCsvReader

PocketCsvReader is a highly efficient and lightweight library tailored for parsing delimited flat files. Optional extension packages add support for formats such as JSON, NDJSON, fixed-width, LTSV, logfmt, Common Log Format, and W3C Extended Log Format. With a focus on simplicity and performance, it offers seamless file reading and supports versatile outputs, including DataTables, string arrays, strongly-typed object mapping and an IDataReader interface.

About | Install | Quick-start

About

Social media:

Releases:

Dev. activity:

Continuous integration builds:

Status:

Install

Replace <VersionNumber> with the desired version in each of the following solutions. If no version is specified, the latest version will be installed.

NuGet CLI

1. Open a command prompt or terminal.

2. Run the following command:

   nuget install PocketCsvReader -Version <VersionNumber>
   

Visual Studio Package Manager Console

1. Open the Package Manager Console from Tools > NuGet Package Manager > Package Manager Console.

2. Run the following command:

   Install-Package PocketCsvReader -Version <VersionNumber>
   

Dotnet-CLI

1. Open a terminal or command prompt.

2. Navigate to the directory of your project.

3. Run the following command:

   dotnet add package PocketCsvReader --version <VersionNumber>
   

   For standard JSON documents, install the JSON extension package:

   dotnet add package PocketCsvReader.Json --version <VersionNumber>
   

   For newline-delimited JSON streams, install the NDJSON extension package:

   dotnet add package PocketCsvReader.Ndjson --version <VersionNumber>
   

   For fixed-width files, install the extension package as well:

   dotnet add package PocketCsvReader.FixedWidth --version <VersionNumber>
   

   For LTSV and logfmt records, install the key-value extension package:

   dotnet add package PocketCsvReader.KeyValue --version <VersionNumber>
   

   For Common Log Format and W3C Extended Log Format files, install the web-log extension package:

   dotnet add package PocketCsvReader.WebLogs --version <VersionNumber>
   

Quick-start

The CsvReader class is a flexible and efficient tool for reading and parsing CSV files or streams into various formats, such as DataTable, IDataReader, or strongly-typed objects. This documentation explains the basics of how to use the class, including common use cases and examples.

Features

• Read CSV files or streams into a DataTable.
• Access CSV data in a forward-only, read-only manner using IDataReader.
• Map CSV records to strongly-typed objects.
• Map CSV records to array of strings.
• Customizable CSV parsing profiles for delimiters, quote handling, and more.
• Supports encoding detection through the IEncodingDetector interface.
• Stream standard JSON objects and top-level arrays without materializing the complete document.
• Read NDJSON as a sequence of independently framed JSON values.
• Read fixed-width files in one forward-only pass with field conversion deferred until a Get* accessor requests that field.
• Read labeled LTSV and logfmt records through one shared reader and format-specific builders.
• Read Common Log Format and W3C Extended Log Format files with format-aware streaming readers.

Fixed-width support is distributed separately as PocketCsvReader.FixedWidth.

Standard JSON and NDJSON use separate extension packages. PocketCsvReader.Json accepts one JSON document and yields each element of a top-level array as a row. PocketCsvReader.Ndjson expects a sequence of JSON values separated by line terminators and supports NDJSON-specific comments and custom terminators.

using PocketCsvReader.Json;
using PocketCsvReader.Ndjson;

using var json = new JsonReader().ToDataReader("people.json");
while (json.Read())
{
    var id = json.GetInt32(json.GetOrdinal("id"));
    var name = json.GetString(json.GetOrdinal("name"));
}

using var ndjson = new NdjsonReader().ToDataReader("events.ndjson");
while (ndjson.Read())
{
    var eventName = ndjson.GetString(ndjson.GetOrdinal("event"));
}

LTSV and logfmt support is distributed separately as PocketCsvReader.KeyValue:

using PocketCsvReader.KeyValue.Configuration;

var ltsvReader = new LtsvReaderBuilder().Build();
var logfmtReader = new LogfmtReaderBuilder().Build();

using var dataReader = logfmtReader.ToDataReader("application.log");
while (dataReader.Read())
{
    var level = dataReader.GetString(dataReader.GetOrdinal("level"));
    var message = dataReader.GetString(dataReader.GetOrdinal("message"));
}

Common and W3C Extended web-log support is distributed separately as PocketCsvReader.WebLogs:

using PocketCsvReader.WebLogs;

using var common = new CommonLogReader().ToDataReader("access.log");
while (common.Read())
{
    var request = common.GetString(common.GetOrdinal("Request"));
    var status = common.GetInt32(common.GetOrdinal("StatusCode"));
}

using var w3c = new W3cExtendedLogReader().ToDataReader("u_ex.log");
while (w3c.Read())
{
    var method = w3c.GetString(w3c.GetOrdinal("cs-method"));
    var path = w3c.GetString(w3c.GetOrdinal("cs-uri-stem"));
}

Initialization

You can create an instance of CsvReader with various configurations:

// Default configuration: comma-delimited, double quotes for escaping, 4 KB buffer size.
var csvReader = new CsvReader();

// Custom CSV profile (e.g., semicolon-delimited, double quotes for escaping).
var csvReaderWithProfile = new CsvReader(CsvProfile.SemiColumnDoubleQuote);

// Custom buffer size for large files.
var csvReaderWithBuffer = new CsvReader(bufferSize: 64 * 1024);

// Both custom profile and buffer size.
var csvReaderCustom = new CsvReader(CsvProfile.SemiColumnDoubleQuote, bufferSize: 16 * 1024);

Reading CSV Data

Reading Into a DataTable

The ToDataTable method reads CSV data and returns a DataTable containing all rows and fields.

DataTable dataTable = csvReader.ToDataTable("example.csv");

or to read from a stream,

using var stream = new FileStream("example.csv", FileMode.Open, FileAccess.Read);
DataTable dataTable = csvReader.ToDataTable(stream);

Accessing Data with IDataReader

The ToDataReader method provides a forward-only, read-only CsvDataReader implementing IDataReader for processing large files efficiently. IDataReader itself is synchronous; PocketCsvReader readers also implement IAsyncDataReader when asynchronous stream I/O is needed.

using var stream = new FileStream("example.csv", FileMode.Open, FileAccess.Read);
using var reader = csvReader.ToDataReader(stream);
while (reader.Read())
{
    Console.WriteLine(reader[0]); // Access the first column of the current row.
    Console.WriteLine(reader.GetDateTime(1)); // Access the second column of the current row as an object boxing a DateTime.
    Console.WriteLine(reader.GetFieldValue<DateOnly>(2)); // Access the third column of the current row as DateOnly.
}

For consumers built around the standard DbDataReader abstraction, use ToDbDataReader. It is supported by every PocketCsvReader format and by CSV batch readers. Asynchronous advancement uses the underlying stream's asynchronous operations and accepts a cancellation token. Field getters remain synchronous because the current record is already buffered after ReadAsync completes.

await using var stream = new FileStream("example.csv", FileMode.Open, FileAccess.Read, FileShare.Read, 4096, useAsync: true);
await using DbDataReader reader = csvReader.ToDbDataReader(stream);
while (await reader.ReadAsync(cancellationToken))
{
    Console.WriteLine(reader.GetInt32(0));
    Console.WriteLine(reader.GetString(1));
}

Reading as Arrays

using var stream = new FileStream("example.csv", FileMode.Open, FileAccess.Read);
foreach (var record in csvReader.ToArrayString(stream))
{
    Console.WriteLine(string.Join(", ", record));
}

Mapping Records to Strongly-Typed Objects

The To<T> method maps CSV records to objects of a specified type.

public class Person
{
    public string FirstName { get; set; }
    public string LastName { get; set; }
    public int Age { get; set; }
}

using var stream = new FileStream("example.csv", FileMode.Open, FileAccess.Read);
IEnumerable<Person> people = csvReader.To<Person>(stream);

foreach (var person in people)
{
    Console.WriteLine($"{person.FirstName} {person.LastName}, Age: {person.Age}");
}

Use ToAsync<T> to stream the same strongly typed records with asynchronous input reads. It is lazy and forward-only, so records are mapped one at a time rather than buffering the complete input.

await foreach (var person in csvReader.ToAsync<Person>("example.csv", cancellationToken: cancellationToken))
{
    Console.WriteLine($"{person.FirstName} {person.LastName}, Age: {person.Age}");
}

ToAsync<T> uses the same asynchronous reader and parser infrastructure as ReadAsync, including profiles, headers, bad-data handling, buffer-boundary handling, and cancellation. A file passed by name is opened on enumeration and closed with the enumerator. A caller-provided stream remains owned by the caller and is left open.