Caveman.PrivacyGuard 1.4.0

Caveman.PrivacyGuard 🛡️🇪🇺

<img width="638" height="408" alt="Gemini_Generated_Image_tnxoi9tnxoi9tnxo" src="https://github.com/user-attachments/assets/5a8e3193-b7d3-4a41-90e6-e672467ce6cb" />

Enterprise-grade PII & Privacy Analyzer for AI/LLM workflows Detects, scores, and automatically masks sensitive data across 32 countries (27 EU + UK, Switzerland, China, Russia, Ukraine), with risk scoring, GDPR/AI Act/NIS2/PCI-DSS/NIST compliance mapping, and seamless integration with any AI pipeline — including multi-session scenarios (multiple chatbots/conversations running in parallel).

✨ Features

📦 Installation

dotnet add package Caveman.PrivacyGuard

🚀 Quick Start

Warning Mode (Default)

var analyzer = new PrivacyAnalyzer { EnableAutoMasking = true };

// 🔹 Default: English
var resEn = analyzer.Analyze("CF: RSSMRA80A01H501U, email: test@x.com");
Console.WriteLine($"[EN] Risk: {resEn.RiskLevel}\n{resEn.WarningMessage}");

// 🔹 Overload: Italian
var resIt = analyzer.Analyze("CF: RSSMRA80A01H501U, email: test@x.com", "it");
Console.WriteLine($"[IT] Risk: {resIt.RiskLevel}\n{resIt.WarningMessage}");

// 🔹 Overload: German
var resDe = analyzer.Analyze("Steuer-ID: 12345678901, IBAN: DE89370400440532013000", "de");
Console.WriteLine($"[DE] Risk: {resDe.RiskLevel}\n{resDe.WarningMessage}");

Output:

🔒 Score: 26/100
⚠️  Warning: ⛔ Data detected: Italian Tax Code (CF), Email. Pseudonymization mandatory.
✅ Safe for AI: False

var analyzer = new PrivacyAnalyzer { EnableAutoMasking = true };

var text = "Customer: Mario Rossi, email mario@company.it, IBAN: IT60X0542811101000000123456";
var result = analyzer.Analyze(text);

Console.WriteLine($"🛡️  Text safe for LLM:\n{result.MaskedText}");

Output:

🛡️  Text safe for LLM:
Customer: Mario Rossi, email [EMAIL], IBAN: [IBAN]

🪄 Session Restore — Zero Sensitive Data to the LLM

The core value proposition: sensitive data never leaves the client. The flow is:

1. Mask — PrivacyAnalyzer detects PII and replaces it with unique placeholders [PG_1], [PG_2], etc.
2. Send — The masked text (no real data) is sent to the LLM
3. Restore — The LLM's response is processed by PrivacySession.Restore(), which restores the original values client-side

var session = new PrivacySession();
var analyzer = new PrivacyAnalyzer { EnableAutoMasking = true };

// 1. Analyze and mask (the real data only lives in the session)
var result = analyzer.Analyze(
    "Mario Rossi, email mario@company.it, IBAN: IT60X0542811101000000123456",
    "en", session);

Console.WriteLine(result.MaskedText);
// Output: Mario Rossi, email [PG_1], IBAN: [PG_2]

// 2. Send result.MaskedText to the LLM...
var aiResponse = "Contact the customer via [PG_1] regarding the charge on [PG_2]";

// 3. Restore client-side (no sensitive data ever left the client)
var safeResponse = session.Restore(aiResponse);
Console.WriteLine(safeResponse);
// Output: Contact the customer via mario@company.it regarding the charge on IT60X0542811101000000123456

API Reference

// Create a new session
var session = new PrivacySession();

// Analyze with a session (unique [PG_N] placeholders)
var result = analyzer.Analyze(input, "en", session);

// Or via the CurrentSession property
analyzer.CurrentSession = new PrivacySession();
var result = analyzer.Analyze(input);

// Restore an AI response
string restored = session.Restore(aiResponseText);

// Inspect the generated placeholders
foreach (var entry in session.GetAll())
    Console.WriteLine($"{entry.Key} → {entry.Value.OriginalValue} ({entry.Value.Category})");

// Get a single mapping
var entry = session.GetEntry("[PG_1]");

// Manually add a placeholder (public)
session.AddOrGet("Email", "manual@test.com");

// Merge another session (skips duplicates by value)
session.MergeFrom(anotherSession);

// Reset the session
session.Clear();

// Static method on PrivacyAnalyzer
string restored = PrivacyAnalyzer.RestoreText(aiResponse, session);

// Instance method (uses CurrentSession)
analyzer.CurrentSession = session;
string restored2 = analyzer.RestoreText(aiResponse);

Note: Without a session (Analyze(input) or Analyze(input, lang)), masking uses the textual placeholders [EMAIL], [IBAN], etc. as before.

Countries Covered

🧑‍🤝‍🧑 Multi-Session: Multiple Chatbots / AI Providers

If your service juggles many concurrent conversations — several chatbot instances, or an AI-provider-facing gateway serving many users — PrivacySessionManager keeps each conversation's placeholder mapping isolated, so one conversation can never leak another's data.

var sessions = new PrivacySessionManager();
var analyzer = new PrivacyAnalyzer { EnableAutoMasking = true };

// Each conversation gets (and reuses) its own isolated session
var chat1 = sessions.GetOrCreate("conversation-123");
var chat2 = sessions.GetOrCreate("conversation-456");

var r1 = analyzer.Analyze("email alice@example.com", "en", chat1);
var r2 = analyzer.Analyze("email bob@example.com", "en", chat2);

// chat1 can only restore its own placeholders — chat2's data never crosses over
var restored = chat1.Restore(r1.MaskedText);

// Periodically bound memory usage in long-running services:
int removed = sessions.PruneIdleSessions(TimeSpan.FromMinutes(30));

💬 AI Transparency Notice & 🧭 Chat Intent Router

For chatbot/assistant use cases, AiTransparencyNotice gives you a ready-to-display, configurable disclosure that the user is talking to an AI system and that their data is screened/masked — a technical aid toward the transparency expectations of EU AI Act Art.50 for limited-risk AI systems (whether it fully satisfies your specific obligations depends on your use case and should be confirmed with legal counsel).

var notice = new AiTransparencyNotice { Language = "en" }; // or "it", "de", "fr", "es"
Console.WriteLine(notice.GetMessage());
// "You are interacting with an AI system. Your messages are screened locally and any
//  personal or sensitive data is masked before being sent to the AI model (EU AI Act
//  Art.50 transparency notice)."

// Disable it, or supply your own wording:
notice.Enabled = false;
notice.CustomMessage = "Custom disclosure text for your organization.";

ChatIntentRouter lets you register bot commands (e.g. /reset, /help) and route them before the text ever reaches the analyzer:

var router = new ChatIntentRouter();
router.RegisterCommand("/reset", _ => session.Clear());
router.RegisterCommand("/help", _ => ShowHelp());

var matchedIntent = router.TryRoute(userInput);
if (matchedIntent is null)
{
    // No command matched: treat userInput as regular content to analyze/mask.
    var result = analyzer.Analyze(userInput, "en", session);
}

Streaming & Document Masking (v1.3.0)

Boundary-Safe Stream Masking

PrivacyStreamMasker masks PII in incrementally-arriving text (a chunked file read, or a live LLM stream) without loading the whole input. It keeps a tail buffer of overlapChars (default 512) so a match that spans a chunk boundary is still analyzed whole.

var masker = new PrivacyStreamMasker(session: session, language: "en", overlapChars: 512);
foreach (var chunk in ReadInChunks(path))
    Console.Write(masker.Feed(chunk));   // masked-safe prefix as it becomes safe
Console.Write(masker.Flush());           // final remainder

// Or, for an in-memory iterable:
var masked = string.Concat(PrivacyStreamMasker.MaskStream(chunks, session));

Redacted Session Summary

PrivacySession.Summary() returns placeholder + category only — never the original value — so it is safe to log, display, or return from an API/CLI response. GetAll() / ToJson() carry the cleartext PII; keep them server-side only.

foreach (var item in session.Summary())   // SessionSummaryItem { Placeholder, Category }
    Console.WriteLine($"{item.Placeholder}: {item.Category}");

Document Masking (txt / Markdown / CSV)

DocumentMasker.MaskFile streams a file in chunks and writes a masked copy to name.masked.ext next to the source — the original file is never overwritten. CSV rows are read with a minimal RFC-4180-aware reader so quoted multi-line cells survive.

var result = DocumentMasker.MaskFile("contacts.csv", session: session);
// result.OutputPath → "contacts.masked.csv", result.MaskedMatchCount → total PII matches

Fewer false positives, and an optional ML tier (v1.3.1 / v1.4.0)

A bare number in prose is not PII. Since 1.3.1 a match only counts when it is confirmed: a value that fails its checksum is never detected or masked, and a rule marked requires_context (every bare numeric format: phone, PESEL, BSN, OIB, CNP, ...) additionally needs one of its context_keywords within 25 characters - short keywords match as whole words, so tel is not confirmed by "hotel". A +-prefixed number is E.164 by definition and needs no keyword. Structurally anchored rules (e-mail, IBAN, codice fiscale, NINO, ...) are unaffected.

Optional learned confirmation (1.4.0). PrivacyAnalyzer.MlDetector accepts any IMlEntityDetector; the CavemanML package ships PrivacyGuardML, a small offline model that recognises PII by shape and context:

using var analyzer = new PrivacyAnalyzer { MlDetector = await PrivacyGuardMLDetector.LoadAsync() };
var r = analyzer.Analyze("ssn 123-45-6789 lo trovi nel file");   // r.MlAssistedCount == 1

The model is only a confirmation signal: it can recover a bare value the context gate rejected (when an overlapping span has the right label), it can never override a failed checksum, and without a detector nothing changes.

⚙️ Advanced Configuration

Whitelist (Safe Exclusions)

analyzer.AddToWhitelist(
    "test@company.eu",      // Test email
    "127.0.0.1",             // Localhost IP
    "IT00000000000"         // Dummy VAT number
);

// Removal and reset
analyzer.RemoveFromWhitelist("test@company.eu");
analyzer.ClearWhitelist();  // removes all exclusions

// Inspection
var all = analyzer.GetWhitelist();     // IReadOnlySet<string>
if (analyzer.IsWhitelisted("127.0.0.1"))
    Console.WriteLine("IP is whitelisted");

Custom Validators

// Register a new validator for a custom national format
ValidatorRegistry.Register("MY_CUSTOM_ID", value =>
    value.Length == 10 && value.StartsWith("XYZ") && value.All(char.IsDigit));

// Remove an existing validator
ValidatorRegistry.Unregister("MY_CUSTOM_ID");

// Reset all validators to the built-in defaults (removes custom ones)
ValidatorRegistry.Reset();

Runtime Rule Inspection & Management

// List all currently loaded rule categories
var categories = analyzer.GetLoadedCategories();

// Inspect a specific rule (regex, weight, validator, etc.)
var emailRule = analyzer.GetRule("Email");
Console.WriteLine(emailRule.Pattern);   // compiled regex
Console.WriteLine(emailRule.BaseWeight); // weight

// Remove all rules for a category
analyzer.RemoveRule("Email");

// Clear ALL rules (embedded + custom)
analyzer.ClearRules();

Resource Disposal

PrivacyAnalyzer implements IDisposable to release its ReaderWriterLockSlim. All public methods throw ObjectDisposedException if called after disposal:

using var analyzer = new PrivacyAnalyzer { EnableAutoMasking = true };
var result = analyzer.Analyze("test@example.com");
// analyzer.Dispose() is called automatically on exiting the using block

Async Analysis

For UI or web applications, AnalyzeAsync runs the analysis on the thread pool and supports CancellationToken:

using var cts = new CancellationTokenSource(TimeSpan.FromSeconds(5));

try
{
    var result = await analyzer.AnalyzeAsync(
        "Email: test@example.com, IBAN: IT60X0542811101000000123456",
        cts.Token);

    Console.WriteLine($"Score: {result.Score}");
}
catch (OperationCanceledException)
{
    Console.WriteLine("Analysis cancelled due to timeout.");
}

Available overloads: AnalyzeAsync(input), AnalyzeAsync(input, session, ct), AnalyzeAsync(input, language, ct), AnalyzeAsync(input, language, session, ct).

External and Dynamic YAML

// Load rules from a file (added to the embedded ones)
analyzer.LoadCustomYaml("path/to/custom-rules.yaml");

// Load rules from a YAML string (useful for DB/remote config)
var yaml = @"
version: ""2.0""
countries:
  - code: ""XX""
    rules:
      - category: ""CustomPII""
        pattern: '\bSENSITIVE_\w+\b'
        base_weight: 20
        compliance_tags: [""GDPR Art.4(1)""]";
analyzer.LoadCustomYamlFromString(yaml);

Compliance Tags from YAML

Compliance flags (compliance_tags in the YAML) are automatically collected and merged with the hardcoded ones. You can add custom tags per rule:

- category: "MyCustomID"
  pattern: '\bMY_\d{6}\b'
  base_weight: 15
  compliance_tags: ["GDPR Art.4(1)", "Internal Policy #42"]

JSON Config and Hot-Reload

Besides YAML, JSON configuration with automatic hot-reload is also supported:

// Load rules from a JSON file
analyzer.LoadCustomJson("path/to/custom-rules.json");

// Load from a string
var json = @"{
  ""version"": ""2.0"",
  ""countries"": [
    { ""code"": ""XX"", ""rules"": [
      { ""category"": ""CustomPII"", ""pattern"": ""\\bSENSITIVE_\\w+\\b"", ""base_weight"": 20 }
    ]}
  ]
}";
analyzer.LoadCustomJsonFromString(json);

// Hot-reload: watch the file and reload automatically
analyzer.WatchConfig("path/to/custom-rules.json");

// Event fired after every reload
analyzer.ConfigReloaded += (s, e) => Console.WriteLine("⚡ Config reloaded!");

// Stop the watcher
analyzer.StopWatching();

Rule Validation

Check that all loaded rules are valid (no exceptions during matching):

if (analyzer.ValidateRules())
    Console.WriteLine("✅ All rules are valid");
else
    Console.WriteLine("❌ Invalid rules found");

ILogger Integration

Attach a logger for diagnostics during analysis and configuration:

analyzer.Logger = myLogger;  // ILogger<PrivacyAnalyzer>
// Logged events: detected categories, high scores, configuration loads

Session Export and Persistence

Export and import sessions as JSON to save/restore state across requests:

var session = new PrivacySession();
session.AddEntry("Email", "test@example.com");

// Export
string json = session.ToJson();

// Reconstruct
var restored = PrivacySession.FromJson(json);

// Import into an existing session
existingSession.ImportFromJson(json);

IReadOnlySession

A safe read-only API for public exposure:

IReadOnlySession readOnly = session;
int count = readOnly.Count;
var entry = readOnly.GetEntry("[PG_1]");
string text = readOnly.Restore(aiResponse);

🔐 Compliance & GDPR

PrivacyAnalysisResult.ComplianceFlags automatically combines:

• YAML — compliance_tags defined for each rule in rules.yaml
• Hardcoded — automatic mapping for known categories (GDPR, EU AI Act, NIS2, PCI-DSS, NIST 800-53)

if (result.ComplianceFlags.Contains("GDPR Art.4(1)"))
{
    // Logic for legal basis, DPIA, minimization
}
if (result.ComplianceFlags.Contains("PCI-DSS"))
{
    // Encryption, network segmentation, audit
}

Score Components

Risk Thresholds:

⚠️ Disclaimer: This library is a technical support tool. It does not replace a Data Protection Impact Assessment (DPIA) or the advice of a DPO. GDPR, AI Act, and NIS2 compliance require contextual assessments, legal bases, and organizational processes that no library can substitute for.

🧪 Testing & Performance

// Quick benchmark
var analyzer = new PrivacyAnalyzer();
var sw = System.Diagnostics.Stopwatch.StartNew();

for (int i = 0; i < 10_000; i++)
{
    analyzer.Analyze("Text with email test@example.com and IBAN DE89370400440532013000");
}

sw.Stop();
Console.WriteLine($"⚡ 10k analyses in {sw.ElapsedMilliseconds}ms ({10_000.0 / sw.ElapsedMilliseconds:F1} req/sec)");

🤝 Contributing

Fork the repo Create a branch for your feature (git checkout -b feature/new-country) Add rules in rules.yaml + validators in ValidatorRegistry.cs Add tests in tests/ (xUnit) Submit a PR with a clear description and examples 🌍 To add a new country: follow the existing YAML schema, implement a validator (if needed), and add an entry to the README table.

📄 License

Distributed under the MIT license. See LICENSE for details.

🆘 Support & Roadmap

Technology Partnership

<img src="https://www.digitalsolutions.it/img/partners/novaroutelogo.png" alt="NovaRouteAI" height="180" style="max-width: 100%; height: auto; min-height: 180px; max-height: 190px;">

NovaRouteAI — Build with Chinese AI models through one simple API.

NovaRouteAI helps developers and AI SaaS teams test, compare, and run models like DeepSeek, Qwen, Doubao, Kimi, and GLM without managing multiple provider accounts. Start with test credits and optimize your cost per successful task.

Click here to know NovaRouteAI

🛡️ Protect data, enable AI. Compliance by design.