Lyo.Privacy 2.0.0

Lyo.Privacy

Masks and sanitizes free text, JSON, and XML. Built-in rules cover emails, phones, Luhn payment-card numbers with optional BIN allow/block lists, IBAN (MOD-97), heuristic bank-account digit blocks, API key and secret patterns (AWS access keys, GitHub PATs, KEY=value assignments) with optional entropy gating, opt-in national / tax ID packs (US SSN shape, UK NINO, German Steuer-ID heuristic), URL query strings, IP addresses, and best-effort US street lines. You can add regex or literal rules, allowlist literals that must never be masked, load policy JSON for ops-owned tuning, and record SHA-256 policy fingerprints for audit (the hash omits secret material). Named presets and optional Lyo.Metrics.IMetrics instrumentation are included.

Targets netstandard2.0 and net10.0. ASP.NET Core helpers live in Lyo.Privacy.AspNetCore.

Use this for logs, exports, and support tooling. It does not replace legal or compliance review.

Examples

Manual sample

using Lyo.Metrics;
using Lyo.Privacy.Enums;
using Lyo.Privacy.Policy;
using Lyo.Privacy.Rules;
using Lyo.Privacy.Text;

var policy = new RedactionPolicyBuilder()
    .AddEmailRule(EmailMaskStyle.PartialLocalPreserveDomain, visibleLocalPrefixLength: 1)
    .AddPhoneRule(PhoneMaskOptions.LastFourDigits(digitsOnly: true))
    .Build();

var redactor = new TextRedactor(policy, NullMetrics.Instance);
var result = redactor.Redact("Call +1-555-123-4567 or email alice@example.com");

ASP.NET Core

using Lyo.Privacy;
using Lyo.Privacy.AspNetCore;

services.AddLyoPrivacy(configuration);

ASP.NET Core

services.AddLyoPrivacyPolicy("support", b => b.AddPolicy(PrivacyPolicies.SupportExport()));

How the project is laid out

Like Lyo.Diagnostic, sources are grouped by area. Namespaces are split (for example Lyo.Privacy.Abstractions, Lyo.Privacy.Rules, Lyo.Privacy.Json) so feature boundaries stay explicit.

PhoneMaskOptions, EmailMaskOptions, XmlRedactorOptions, and JsonRedactorOptions are sealed classes with { get; set; } for configuration binding and in-place tuning, the same pattern as Lyo.Cache.

Compliance stance

• Not a DLP or lawful-basis tool. Presets and rules are pattern-based heuristics. They do not prove removal of personal data, determine lawful grounds, or satisfy HIPAA Safe Harbor, GDPR pseudonymisation, or sector-specific definitions on their own.
• Prefer structured fields. Redact columns or JSON keys you classify as sensitive. Use free-text rules only where unstructured data is unavoidable.
• Placeholders vs partial masks. Placeholders minimise residual disclosure. Partial masks trade risk for supportability. Choose explicitly per sink and retention.
• Stable hashes. HashStable in JSON is not encryption. Anyone with the salt and payload can correlate values. Treat salts as secrets and rotation as a product decision.

Performance

• Text. Per-code-unit work is O(input length × rules) in the worst case (each rule scans the string). Presets keep rule counts small. Trim rules on hot paths.
• Text. Winning-rule tracking uses parallel int[] / RedactionKind[] windows (no per-position nullable types) to reduce overhead vs int?.
• JSON. MemoryStream is pre-sized from input length to cut reallocations. RedactJsonUtf8 parses ReadOnlyMemory<byte> without building a Unicode string first (rules on string values still materialize strings when applied). RedactJsonStream buffers the input stream, redacts, then writes UTF-8 output.
• Text span. ITextRedactor.Redact(ReadOnlySpan<char>) routes through a single buffer copy, then the same engine as string (rules still operate on string today).
• XML. XmlRedactor walks XDocument by sensitive element local name. Invalid XML can fall back to ITextRedactor like JSON.
• Hashes. Short hex prefixes for HashStable use a fixed small char[], not per-nibble string allocations.

Internationalization

• Email. The detector targets ASCII-looking addresses (common in logs). IDN / punycode domains (e.g. xn--) and internationalised local-parts may not match. Consider normalising or using structured email fields.
• Phone. Patterns skew toward NANP-style grouping. International formats (short codes, different punctuation) may be missed or partially matched. Stricter scenarios may need locale- or country-specific rules (or validation libraries) in addition to this package.

Core ideas

RedactionPolicyBuilder helpers

On top of AddRule(IRedactionRule), the builder exposes AddRule<T>(T rule) where T : class, IRedactionRule and shorthand methods such as AddPhoneRule(PhoneMaskMode, …), AddPhoneRule(PhoneMaskOptions, …), AddEmailRule(…), AddPaymentCardRule(allowedBins6, blockedBins6), AddUrlRule(), AddIpRule(…), AddAddressRule(), AddIbanRule(), AddBankAccountRule(…), AddNationalIdRule(…), AddApiSecretRule(…), AddLiteralRule(…), and AddRegexRule(…). Extension AppendPreset lives in Lyo.Privacy.Configuration.

Email masking (EmailRedactionRule, EmailMaskOptions)

EmailMaskOptions is the preferred entry (the legacy EmailMaskStyle ctor maps to the same shapes). When UsePolicyPlaceholder is true, the whole address becomes the policy Placeholder.

Fields that usually matter: VisibleLocalSuffixLength, LocalMaskLiteral, PreserveLocalSeparators / SeparatorMaskChar (keeps . / - / _ next to masked segments), VisibleDomainSuffixLength, DomainMaskLiteral, PreserveEntireDomainHost, PreserveDomainFromFirstDot, AtReplacement (honored only when PreserveAtSign is false. When AtReplacement is unset, LocalMaskLiteral is used).

Phone masking (PhoneRedactionRule, PhoneMaskOptions)

Through the match, digits are counted in order (country code and groups collapse to one digit stream). TrailingDigitsVisible and LeadingDigitsVisible are a union (if they overlap, every digit can become visible). Set OnlyFirstDigitAmongLastN to ignore those two counts and show a single digit in the last N window.

PhoneMaskMode still has a working legacy ctor: Full becomes placeholder, LastDigits becomes TrailingDigitsVisible (separators kept unless you change that), FirstDigitOfLastGroup becomes OnlyFirstDigitAmongLastN with DigitsOnlyOutput and no separators.

Other built-in text rules (quick samples)

Optional Func<string, RedactionSpan, string?>? formatters are accepted by DelegateRedactionRule and RegexRedactionRule.

CompositeRedactionRule does not forward inner formatters; composite matches use the policy placeholder unless you add a custom composite formatter later.

Allowlisted literals (RedactionPolicy.NeverRedactSubstrings)

Winning redaction spans are cleared wherever a substring listed on the policy (or RedactionPolicyBuilder.WithNeverRedactSubstrings) appears. Use this for staging markers, known-safe tokens, or fixed test data you must not destroy in logs.

Policy as data (PolicyJson)

PolicyJson.Build(json, configure?) loads PolicyDefinitionDto-shaped JSON. Rule kinds include email, phone, paymentCard (with allowedBins / blockedBins), iban, bankAccount (bankMinNumeric), nationalId (nationalIdPacks), apiSecret (apiPatterns, apiMinEntropy), literal, regex, plus URL / IP / address. Top-level neverRedactSubstrings maps to the allowlist above.

Audit fingerprint (RedactionPolicyFingerprint)

RedactionPolicyFingerprint.ComputeSha256HexPrefix(policy) hashes policy name, placeholder, merge flag, never-redact strings, rule order/types, and coarse option summaries (BIN sets, API pattern flags plus entropy threshold, national ID packs, phone/email option shape). Raw secrets are not embedded. Regex rules contribute a hash code of the pattern and options, not the pattern text.

JsonRedactor contracts

• If ApplyTextRulesToAllStringValues is true, an ITextRedactor must be passed to the constructor (otherwise ArgumentException).
• If JSON parse fails and no ITextRedactor was supplied for fallback, InvalidFormatException (from Lyo.Exceptions) is thrown (inner exception is a JsonException). With a text redactor, invalid JSON is redacted as raw text.
• RedactJsonUtf8(ReadOnlyMemory<byte>) and RedactJsonStream(Stream, Stream) use the same rewrite pipeline as RedactJson (stream helper buffers the entire input first).

XML (XmlRedactor)

Set XmlRedactorOptions.SensitiveElementLocalNames (local name maps to XmlScalarStrategy.Placeholder or RemoveElement). Counts use RedactionKind.XmlSensitive. Set ApplyTextRedactorToNonSensitiveText and pass an ITextRedactor to run text rules inside non-listed elements. Invalid XML can fall back to the text redactor, recorded as lyo.privacy.xml.fallback_to_text.

Presets (PrivacyPolicies, PrivacyPresetNames)

• Minimal. No built-in rules unless you add them.
• Logging. Email (placeholder), partial phone (last 4 digits visible), cards, URL queries, truncated IPv4.
• SupportExport. Logging core plus Bearer and JWT-shaped tokens.
• PublicSurface. Placeholder phone, cards, URLs, full IP, email placeholder, address rule.
• RegressionTesting. Same rule mix as logging with placeholder [REDACTED] for snapshots.

Metrics (IMetrics)

Hand an Lyo.Metrics.IMetrics into JsonRedactor / TextRedactor, or register IMetrics in DI when using AddLyoPrivacy (resolved via GetService<IMetrics>() ?? NullMetrics.Instance).

Counters use tag kind for RedactionKind. When a policy has RedactionPolicy.Name set (built-in presets do this automatically; use RedactionPolicyBuilder.WithPolicyName, PrivacyRedactorOptions.PolicyName, or JsonRedactorOptions.PolicyName), tag policy is added so dashboards can split Logging vs PublicSurface. Keyed DI registration sets Name from the service key when still null.

Manual sample

For finer control, use the same rule types with PhoneMaskOptions / EmailMaskOptions. See the tables above.

ASP.NET Core

Section name: Privacy (PrivacyRedactorOptions.SectionName). Redaction metrics are recorded only when IMetrics is registered (for example MetricsService).

Keyed policy:

Related

• Lyo.Diagnostic. Stack trace and path sanitisation for observability, separate from field-level PII rules.

Dependencies

Generated from ProjectReference / PackageReference (same model as docs/Lyo.ProjectGraph.html).

• Lyo.Common.Core (direct, lyo)
• Lyo.Hashing (direct, lyo)
• Lyo.Metrics (direct, lyo)
• System.Collections.Immutable 10.0.5 (direct, microsoft, netstandard2.0)
• System.Text.Json 10.0.5 (direct, microsoft, netstandard2.0)
• Lyo.Common.Metadata (transitive, lyo)
• Lyo.Exceptions (transitive, lyo)
• Microsoft.Bcl.AsyncInterfaces 10.0.5 (transitive, microsoft, netstandard2.0)
• Microsoft.Extensions.DependencyInjection.Abstractions 10.0.5 (transitive, microsoft)
• System.IO.Hashing 10.0.5 (transitive, microsoft, net10.0)
• System.Memory 4.6.3 (transitive, microsoft, netstandard2.0)