Faborite.Core 0.1.1

Faborite 🎯

Sync Microsoft Fabric lakehouse data locally for faster development.

Faborite lets you pull sample data from your Fabric Lakehouses to your local machine, so you can develop and test notebooks/scripts without waiting for cloud compute.

Why Faborite?

When working with Microsoft Fabric, you often need to:

• 🐢 Wait for cloud compute to spin up just to test a simple query
• 💸 Pay for compute time during development iterations
• 🔄 Context-switch between local and cloud environments

Faborite solves this by bringing a representative sample of your data locally, enabling:

• ⚡ Instant iteration - No cold start, no waiting
• 💰 Cost savings - Develop locally, deploy to cloud
• 🧪 Better testing - Test with real data patterns locally

Features

• 🎲 Smart Sampling - Random, recent, stratified, or custom SQL sampling
• 📦 Multiple Formats - Export to Parquet, CSV, JSON, or DuckDB
• ⚡ Fast - Parallel downloads, DuckDB-powered sampling
• 🔧 Configurable - Sensible defaults, fully customizable per-table
• 🔐 Secure - Uses Azure authentication (CLI, Service Principal, Managed Identity)
• 🚀 Single Executable - Built with .NET 10 for fast startup and easy deployment
• 🛡️ Production Ready - Comprehensive validation, logging, and retry policies

Installation

Download Binary

Download the latest release from GitHub Releases:

As .NET Global Tool

dotnet tool install -g faborite

From Source

git clone https://github.com/mjtpena/faborite.git
cd faborite
dotnet build

Quick Start

1. Login to Azure

az login

2. Initialize Configuration

faborite init

Edit faborite.json with your workspace and lakehouse IDs.

3. Sync Data

# Sync all tables with defaults (10,000 random rows each)
faborite sync --workspace <workspace-id> --lakehouse <lakehouse-id>

# Or use the config file
faborite sync

4. Use Your Data

import duckdb
df = duckdb.read_parquet('./local_lakehouse/customers/customers.parquet').df()

CLI Reference

sync

Sync data from OneLake to local machine.

faborite sync [options]

Examples:

# Sync specific tables
faborite sync -w <id> -l <id> --table customers --table orders

# Custom sampling
faborite sync -w <id> -l <id> --rows 5000 --strategy recent

# Export as DuckDB database
faborite sync -w <id> -l <id> --format duckdb

# Export as CSV
faborite sync -w <id> -l <id> --format csv

list-tables (alias: ls)

List available tables in a lakehouse.

faborite list-tables -w <workspace-id> -l <lakehouse-id>

init

Generate a sample configuration file.

faborite init [options]

status

Show status of locally synced data.

faborite status [options]

Sampling Strategies

Configuration

Config File

Create a faborite.json file in your project root:

{
  "workspaceId": "your-workspace-guid",
  "lakehouseId": "your-lakehouse-guid",
  "sample": {
    "rows": 10000,
    "strategy": "random"
  },
  "format": {
    "output": "parquet",
    "compression": "snappy"
  },
  "sync": {
    "localPath": "./local_lakehouse",
    "parallelTables": 4,
    "includeSchema": true
  },
  "auth": {
    "method": "cli"
  },
  "tableOverrides": {
    "large_table": {
      "rows": 1000,
      "strategy": "recent",
      "dateColumn": "created_at"
    },
    "lookup_table": {
      "strategy": "full"
    }
  }
}

Environment Variables

All configuration can be overridden with environment variables:

Output Structure

./local_lakehouse/
├── customers/
│   ├── customers.parquet
│   └── _schema.json
├── orders/
│   ├── orders.parquet
│   └── _schema.json
├── products/
│   ├── products.parquet
│   └── _schema.json
└── lakehouse.duckdb     # When using --format duckdb

Authentication

Faborite uses Azure Identity for authentication:

Service Principal Setup

# Set environment variables
export AZURE_TENANT_ID="your-tenant-id"
export AZURE_CLIENT_ID="your-client-id"
export AZURE_CLIENT_SECRET="your-client-secret"

# Update config
{
  "auth": {
    "method": "serviceprincipal",
    "tenantId": "your-tenant-id",
    "clientId": "your-client-id"
  }
}

Using with Notebooks

After syncing, load data in your local notebooks:

Python with DuckDB

import duckdb

# If exported as DuckDB
conn = duckdb.connect('./local_lakehouse/lakehouse.duckdb')
df = conn.execute("SELECT * FROM customers").df()

# If exported as Parquet
df = duckdb.read_parquet('./local_lakehouse/customers/customers.parquet').df()

Python with Pandas

import pandas as pd

df = pd.read_parquet('./local_lakehouse/customers/customers.parquet')

Python with Polars

import polars as pl

df = pl.read_parquet('./local_lakehouse/customers/customers.parquet')

.NET

using DuckDB.NET.Data;

using var connection = new DuckDBConnection("Data Source=./local_lakehouse/lakehouse.duckdb");
connection.Open();
// Query your data...

Requirements

• Runtime: .NET 10.0 (or use self-contained builds)
• Azure: Account with access to Microsoft Fabric
• Permissions: Read access to the target Lakehouse via OneLake

Finding Your IDs

1. Workspace ID: Go to your Fabric workspace → Settings → Copy the Workspace ID
2. Lakehouse ID: Open your Lakehouse → Settings → Copy the Lakehouse ID

Development

Prerequisites

• .NET 10.0 SDK
• Azure CLI (for authentication)

Building

# Clone the repository
git clone https://github.com/mjtpena/faborite.git
cd faborite

# Build
dotnet build

# Run tests
dotnet test

# Run with coverage
dotnet test --collect:"XPlat Code Coverage"

# Run the CLI
dotnet run --project src/Faborite.Cli -- sync -w <workspace-id> -l <lakehouse-id>

Publishing

# Publish self-contained for Windows
dotnet publish src/Faborite.Cli -c Release -r win-x64 --self-contained -o publish/win-x64

# Publish self-contained for Linux
dotnet publish src/Faborite.Cli -c Release -r linux-x64 --self-contained -o publish/linux-x64

# Publish self-contained for macOS
dotnet publish src/Faborite.Cli -c Release -r osx-x64 --self-contained -o publish/osx-x64
dotnet publish src/Faborite.Cli -c Release -r osx-arm64 --self-contained -o publish/osx-arm64

Project Structure

faborite/
├── src/
│   ├── Faborite.Core/           # Core library
│   │   ├── Configuration/       # Config loading & validation
│   │   ├── OneLake/             # OneLake ADLS Gen2 client
│   │   ├── Sampling/            # DuckDB-powered sampling
│   │   ├── Export/              # Format exporters
│   │   ├── Logging/             # Logging infrastructure
│   │   ├── Resilience/          # Retry policies (Polly)
│   │   └── FaboriteService.cs   # Main orchestrator
│   └── Faborite.Cli/            # CLI application
│       ├── Commands/            # CLI commands
│       └── Program.cs           # Entry point
├── tests/
│   ├── Faborite.Core.Tests/     # Core library tests
│   └── Faborite.Cli.Tests/      # CLI tests
├── .github/
│   └── workflows/
│       └── ci.yml               # CI/CD pipeline
└── Faborite.sln

Roadmap

• Delta Lake time travel support
• Incremental sync (only changed data)
• Schema drift detection
• VS Code extension
• GitHub Action for CI/CD pipelines
• Support for Fabric Warehouses

Contributing

Contributions are welcome! Please read our Contributing Guide for details on our code of conduct and the process for submitting pull requests.

Quick Contribution Steps

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

Security

Please see our Security Policy for reporting vulnerabilities.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• DuckDB - For blazing fast local analytics
• Azure SDK for .NET - For Azure integration
• Spectre.Console - For beautiful CLI output
• Polly - For resilience policies

Made with ❤️ by Michael John Peña