LanguageExt.Pipes 5.0.0-beta-43

This is a prerelease version of LanguageExt.Pipes.
There is a newer prerelease version of this package available.
See the version list below for details.
dotnet add package LanguageExt.Pipes --version 5.0.0-beta-43                
NuGet\Install-Package LanguageExt.Pipes -Version 5.0.0-beta-43                
This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.
<PackageReference Include="LanguageExt.Pipes" Version="5.0.0-beta-43" />                
For projects that support PackageReference, copy this XML node into the project file to reference the package.
paket add LanguageExt.Pipes --version 5.0.0-beta-43                
#r "nuget: LanguageExt.Pipes, 5.0.0-beta-43"                
#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.
// Install LanguageExt.Pipes as a Cake Addin
#addin nuget:?package=LanguageExt.Pipes&version=5.0.0-beta-43&prerelease

// Install LanguageExt.Pipes as a Cake Tool
#tool nuget:?package=LanguageExt.Pipes&version=5.0.0-beta-43&prerelease                

LanguageExt.Pipes

This feature of language-ext is based on the wonderful work of Gabriella Gonzalez on the Haskell Pipes library. I have had to make some significant changes to make it work in C#, but the essence is the same, and the core types and composition of the components is exactly the same.

  • If you find this feature confusing at first, and it wouldn't be surprising as it's quite a complex idea, there are some examples in the EffectsExample sample in the repo

Conventional stream programming forces you to choose only two of the following three features:

  1. Effects
  2. Streaming
  3. Composability

If you sacrifice Effects you get IEnumerable, which you can transform using composable functions in constant space, but without interleaving effects (other than of the imperative kind).

If you sacrifice Streaming you get 'Traverse' and 'Sequence', which are composable and effectful, but do not return a single result until the whole list has first been processed and loaded into memory.

If you sacrifice Composability you write a tightly coupled for loops, and fire off imperative side-effects like they're going out of style. Which is streaming and effectful, but is not modular or separable.

Pipes gives you all three features: effectful, streaming, and composable programming. Pipes also provides a wide variety of stream programming abstractions which are all subsets of a single unified machinery:

On top of that, Pipes has more advanced features, including bi-directional streaming. This comes into play when fusing clients and servers:

All of these are connectable and you can combine them together in clever and unexpected ways because they all share the same underlying type.

The pipes ecosystem decouples stream processing stages from each other so that you can mix and match diverse stages to produce useful streaming programs. If you are a library writer, pipes lets you package up streaming components into a reusable interface. If you are an application writer, pipes lets you connect pre-made streaming components with minimal effort to produce a highly-efficient program that streams data in constant memory.

To enforce loose coupling, components can only communicate using two commands:

Pipes has four types of components built around these two commands:

Pipes uses parametric polymorphism (i.e. generics) to overload all operations.

You've probably noticed this overloading already:

This overloading is great when it works, but when connections fail they produce type errors that appear intimidating at first. This section explains the underlying types so that you can work through type errors intelligently.

Producer, Consumer, Pipe, and Effect are all special cases of a single underlying type: Proxy. This overarching type permits fully bidirectional communication on both an upstream and downstream interface.

You can think of it as having the following shape:

Proxy<RT, UOut, UIn, DIn, DOut, A>

      Upstream | Downstream
          +---------+
          |         |
    UOut ◄--       ◄-- DIn   -- Information flowing upstream
          |         |
    UIn --►        --► DOut  -- Information flowing downstream
          |    |    |
          +----|----+
               |
               A

The four core types do not use the upstream flow of information. This means that the UOut and DIn in the above diagram go unused unless you use the more advanced features.

Pipes uses type synonyms to hide unused inputs or outputs and clean up type signatures. These type synonyms come in two flavors:

  • Concrete type synonyms that explicitly close unused inputs and outputs of the Proxy type

  • Polymorphic type synonyms that don't explicitly close unused inputs or outputs

The concrete type synonyms use Unit to close unused inputs and Void (the uninhabited type) to close unused outputs:

  • Effect: explicitly closes both ends, forbidding awaiting and yield

     Effect<RT, A> = Proxy<RT, Void, Unit, Unit, Void, A>
    
              Upstream | Downstream
                  +---------+
                  |         |
            Void ◄--       ◄-- Unit
                  |         |
            Unit --►       --► Void
                  |    |    |
                  +----|----+
                       |
                       A
    
  • Producer: explicitly closes the upstream end, forbidding awaiting

     Producer<RT, OUT, A> = Proxy<RT, Void, Unit, Unit, OUT, A>
    
              Upstream | Downstream
                  +---------+
                  |         |
            Void ◄--       ◄-- Unit
                  |         |
            Unit --►       --► OUT
                  |    |    |
                  +----|----+
                       |
                       A
    
  • Consumer: explicitly closes the downstream end, forbidding yield

      Consumer<RT, IN, A> = Proxy<RT, Unit, IN, Unit, Void, A>
    
              Upstream | Downstream
                  +---------+
                  |         |
            Unit ◄--       ◄-- Unit
                  |         |
              IN --►       --► Void
                  |    |    |
                  +----|----+
                       |
                       A
    
  • Pipe: marks both ends open, allowing both awaiting and yield

      Pipe<RT, IN, OUT, A> = Proxy<RT, Unit, IN, Unit, OUT, A>
    
              Upstream | Downstream
                  +---------+
                  |         |
            Unit ◄--       ◄-- Unit
                  |         |
              IN --►       --► OUT
                  |    |    |
                  +----|----+
                       |
                       A
    

When you compose Proxy using | all you are doing is placing them side by side and fusing them laterally. For example, when you compose a Producer, Pipe, and a Consumer, you can think of information flowing like this:

            Producer                Pipe                 Consumer
         +------------+          +------------+          +-------------+
         |            |          |            |          |             |
   Void ◄--          ◄--  Unit   ◄--         ◄--  Unit  ◄--           ◄-- Unit
         |  readLine  |          |  parseInt  |          |  writeLine  |
   Unit --►         --► string  --►          --► string --►           --► Void
         |     |      |          |    |       |          |      |      |
         +-----|------+          +----|-------+          +------|------+
               v                     v                       v
               ()                    ()                      ()

Composition fuses away the intermediate interfaces, leaving behind an Effect:

                       Effect
        +-----------------------------------+
        |                                   |
  Void ◄--                                 ◄-- Unit
        |  readLine | parseInt | writeLine  |
  Unit --►                                 --► Void
        |                                   |
        +----------------|------------------+
                        Unit
Product Compatible and additional computed target framework versions.
.NET net8.0 is compatible.  net8.0-android was computed.  net8.0-browser was computed.  net8.0-ios was computed.  net8.0-maccatalyst was computed.  net8.0-macos was computed.  net8.0-tvos was computed.  net8.0-windows was computed. 
Compatible target framework(s)
Included target framework(s) (in package)
Learn more about Target Frameworks and .NET Standard.

NuGet packages (2)

Showing the top 2 NuGet packages that depend on LanguageExt.Pipes:

Package Downloads
LanguageExt.Parsec

Parser combinators library based on Haskell Parsec. This is part of the LanguageExt functional framework and requires LanguageExt.Core

LanguageExt.Sys

Extensions to language-ext framework effects system that wraps the IO behaviours from the .NET BCL

GitHub repositories

This package is not used by any popular GitHub repositories.

Version Downloads Last updated
5.0.0-beta-44 0 12/25/2024
5.0.0-beta-43 49 12/23/2024
5.0.0-beta-42 57 12/19/2024
5.0.0-beta-41 51 12/19/2024
5.0.0-beta-40 65 12/17/2024
5.0.0-beta-39 112 11/27/2024
5.0.0-beta-38 95 11/18/2024
5.0.0-beta-36 77 11/6/2024
5.0.0-beta-35 55 11/6/2024
5.0.0-beta-34 136 10/28/2024