AaTurpin.NetworkFileManager 1.1.0

AaTurpin.NetworkFileManager

A robust, high-performance file management library for .NET Framework 4.7.2 applications, specifically designed for reliable file operations across network locations and local storage.

Features

• Resilient Network Operations: Handles network interruptions with configurable retry logic
• Parallel Processing: Optimized multi-threaded file operations with throttling
• Comprehensive Error Handling: Detailed logging and error recovery
• Directory Structure Preservation: Maintains folder hierarchies during file transfers
• File Operation Batching: Process multiple files efficiently with a single call
• Configurable Options: Customize behavior with extensive configuration options
• Detailed Progress Tracking: Monitor file operation success, failure, and skip counts
• File Restoration: Restore files from preserved structures back to their original network locations

Dependencies

This package depends on:

• RunLog: Provides structured logging capabilities

Installation

Install-Package AaTurpin.NetworkFileManager

Note: The required dependency (RunLog) will be installed automatically.

Quick Start

using AaTurpin.NetworkFileManager;
using RunLog;
using System;
using System.Collections.Generic;
using System.Threading;
using System.Threading.Tasks;

// Configure logging first
var logger = new LoggerConfiguration()
    .WriteTo.Console()
    .WriteTo.File("logs/file_operations.log", rollingInterval: RollingInterval.Day)
    .CreateLogger();

Log.Logger = logger;

// Example: Copy files with custom options
async Task CopyFilesExample()
{
    // Create file items to process
    var files = new List<FileItem>
    {
        new FileItem(@"C:\SourceFolder\file1.txt"),
        new FileItem(@"C:\SourceFolder\file2.txt"),
        new FileItem(@"C:\SourceFolder\SubFolder\file3.txt", "SubFolder\file3.txt") // With relative path
    };
    
    // Configure options
    var options = new FileOperationOptions
    {
        MaxParallelOperations = 4,
        Overwrite = true,
        PreserveDirectoryStructure = true,
        SourceBaseDir = @"C:\SourceFolder",
        DeleteEmptySourceDirectories = false,
        MaxRetries = 3,
        RetryDelayMilliseconds = 500
    };
    
    // Execute the copy operation
    var result = await NetworkFileManager.ProcessFilesAsync(
        files,
        @"D:\DestinationFolder",
        FileOperation.Copy,
        options,
        CancellationToken.None);
    
    // Check the results
    Console.WriteLine($"Operation completed. " +
        $"Successful: {result.SuccessCount}, " +
        $"Failed: {result.FailedCount}, " +
        $"Skipped: {result.SkippedCount}");
    
    if (result.FailedCount > 0)
    {
        Console.WriteLine("Failed files:");
        foreach (var failedFile in result.FailedFiles)
        {
            Console.WriteLine($"- {failedFile}");
        }
    }
}

Moving Files

// Example: Move files while preserving directory structure
async Task MoveFilesExample()
{
    // Simple API for moving files from one directory to another
    var result = await NetworkFileManager.MoveFilesAsync(
        @"C:\SourceFolder",
        @"D:\DestinationFolder",
        "*.txt",                  // File pattern
        true,                     // Include subdirectories
        true);                    // Delete empty directories after move
    
    Console.WriteLine($"Move operation completed. " +
        $"Moved: {result.Item1}, " +
        $"Failed: {result.Item2}, " +
        $"Skipped: {result.Item3}");
}

Restoring Files to Original Locations

// Example: Restore files from preserved structure back to original locations
async Task RestoreFilesExample()
{
    // Option 1: Using a mapping dictionary
    var fileMappings = new Dictionary<string, string>
    {
        { "C/Users/file1.txt", @"C:\Users\file1.txt" },
        { "D/Projects/app.config", @"D:\Projects\app.config" }
    };
    
    var result = await NetworkFileManager.MoveFilesToOriginalLocationsAsync(
        @"E:\Backup",           // Source backup directory
        fileMappings,           // Mapping of preserved paths to original paths
        4,                      // Max parallel operations
        true,                   // Overwrite existing files
        CancellationToken.None);
        
    // Option 2: Using a list of original paths
    var originalPaths = new List<string>
    {
        @"C:\Users\file1.txt",
        @"D:\Projects\app.config"
    };
    
    result = await NetworkFileManager.MoveFilesToOriginalLocationsAsync(
        @"E:\Backup",           // Source backup directory
        originalPaths,          // List of original full paths
        4,                      // Max parallel operations
        true,                   // Overwrite existing files
        CancellationToken.None);
        
    Console.WriteLine($"Restore operation completed. " +
        $"Successful: {result.SuccessCount}, " +
        $"Failed: {result.FailedCount}, " +
        $"Skipped: {result.SkippedCount}");
}

Advanced Usage

Custom File Attributes

// Set custom file attributes during transfer
var files = new List<FileItem>
{
    new FileItem(@"C:\SourceFolder\file1.txt") 
    { 
        Attributes = FileAttributes.ReadOnly | FileAttributes.Archive 
    }
};

Custom Destination Paths

// Specify a custom destination path for a specific file
var files = new List<FileItem>
{
    new FileItem(@"C:\SourceFolder\file1.txt") 
    { 
        CustomDestinationPath = @"D:\SpecialFolder\renamed_file1.txt" 
    }
};

Buffered File Operations with Progress Tracking

// Setup a CancellationTokenSource for cancellation support
using (var cts = new CancellationTokenSource())
{
    // For long-running operations, you can use a timeout
    cts.CancelAfter(TimeSpan.FromMinutes(30));
    
    // Create file items with various configurations
    var files = GetFilesToProcess(); // Your method to determine files to process
    
    var options = new FileOperationOptions
    {
        MaxParallelOperations = Environment.ProcessorCount,
        PreCreateDirectories = true,
        MaxRetries = 5
    };
    
    try
    {
        var result = await NetworkFileManager.ProcessFilesAsync(
            files,
            destinationDir: @"\\NetworkShare\Destination",
            operation: FileOperation.Move,
            options: options,
            token: cts.Token);
            
        Log.Information("Operation complete. Success: {SuccessCount}, Failed: {FailedCount}",
            result.SuccessCount, result.FailedCount);
    }
    catch (OperationCanceledException)
    {
        Log.Warning("Operation was cancelled");
    }
}

Logging Integration

The library integrates with the RunLog logging package, which provides a structured logging system. Configure the logger as shown in the Quick Start example to capture detailed information about file operations.

You can also provide a custom logger instance:

// Custom logger setup
var customLogger = new LoggerConfiguration()
    .WriteTo.Console(restrictedToMinimumLevel: LogLevel.Warning)
    .WriteTo.File("logs/network_operations.log", 
        rollingInterval: RollingInterval.Day,
        restrictedToMinimumLevel: LogLevel.Debug)
    .CreateLogger();

// Set the custom logger for the NetworkFileManager
NetworkFileManager.SetLogger(customLogger);

PathUtils

The NetworkFileManager includes built-in utilities for path manipulation:

// Ensure directory exists (handled internally by NetworkFileManager)
string destinationDir = @"D:\NewFolder\SubFolder";
Directory.CreateDirectory(destinationDir);

// Get relative path from a base directory (used internally for directory structure preservation)
// Similar functionality to Path.GetRelativePath() in .NET Core

Thread Safety

All operations in the NetworkFileManager are thread-safe and can be safely used in multi-threaded applications.

Error Handling

The library implements comprehensive error handling:

• Transient errors are automatically retried with configurable retry logic
• Network disconnections are handled gracefully
• Timeouts can be specified via cancellation tokens
• Detailed error information is available through the returned result objects
• All operations log detailed information for troubleshooting

Requirements

• .NET Framework 4.7.2 or higher
• Windows operating system

License

This library is licensed under the MIT License - see the LICENSE file for details.