StableDiffusion.ML.OnnxRuntime 1.1.3

Experimental Package to Inference Stable Diffusion with C# and ONNX Runtime

This package contains the logic to do inferencing for the popular Stable Diffusion deep learning model in C#. Stable Diffusion models take a text prompt and create an image that represents the text.

How to use this NuGet package

• Download the ONNX Stable Diffusion models from Hugging Face

  • Stable Diffusion Models v1.4
  • Stable Diffusion Models v1.5

• Once you have selected a model version repo, click Files and Versions, then select the ONNX branch. If there isn't an ONNX model branch available, use the main branch and convert it to ONNX. See the ONNX conversion tutorial for PyTorch for more information.

• Clone the model repo:

git lfs install
git clone https://huggingface.co/CompVis/stable-diffusion-v1-4 -b onnx

• Copy the folders with the ONNX files to the C# project folder models. The folders to copy are: unet, vae_decoder, text_encoder, safety_checker.

• Install the following NuGets for DirectML

<PackageReference Include="Microsoft.ML" Version="2.0.1" />
<PackageReference Include="Microsoft.ML.OnnxRuntime.DirectML" Version="1.14.1" />

• Cuda support coming soon.

• Sample logic for implementing in your project

    //Default args
    var prompt = "a fireplace in an old cabin in the woods";
    Console.WriteLine(prompt);

    var config = new StableDiffusionConfig
    {
        // Number of denoising steps
        NumInferenceSteps = 15,
        // Scale for classifier-free guidance
        GuidanceScale = 7.5,
        // Set your preferred Execution Provider. Currently DirectML and CPU are supported.
        ExecutionProviderTarget = StableDiffusionConfig.ExecutionProvider.DirectML,
        // Set GPU Device ID.
        DeviceId = 1,
        // Update paths to your models
        TextEncoderOnnxPath = @".\models\text_encoder\model.onnx",
        UnetOnnxPath = @".\models\unet\model.onnx",
        VaeDecoderOnnxPath = @".\models\vae_decoder\model.onnx",
        SafetyModelPath = @".\models\safety_checker\model.onnx",
    };

    // Inference Stable Diff
    var image = UNet.Inference(prompt, config);

    // If image failed or was unsafe it will return null.
    if (image == null)
    {
        Console.WriteLine("Unable to create image, please try again.");
    }

• Set Build for x64

• Hit F5 to run the project in Visual Studio or dotnet run in the terminal to run the project in VS Code.

Resources

• ONNX Runtime C# API Doc
• Get Started with C# in ONNX Runtime
• Hugging Face Stable Diffusion Blog
• Stable Diffusion C# Tutorial for this Repo