mostlylucid.botdetection 1.2.4

There is a newer prerelease version of this package available.
See the version list below for details.

dotnet add package mostlylucid.botdetection --version 1.2.4

NuGet\Install-Package mostlylucid.botdetection -Version 1.2.4

This command is intended to be used within the Package Manager Console in Visual Studio, as it uses the NuGet module's version of Install-Package.

<PackageReference Include="mostlylucid.botdetection" Version="1.2.4" />

For projects that support PackageReference, copy this XML node into the project file to reference the package.

<PackageVersion Include="mostlylucid.botdetection" Version="1.2.4" />
                    

                            Directory.Packages.props

<PackageReference Include="mostlylucid.botdetection" />
                    

                            Project file

For projects that support Central Package Management (CPM), copy this XML node into the solution Directory.Packages.props file to version the package.

paket add mostlylucid.botdetection --version 1.2.4

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

#r "nuget: mostlylucid.botdetection, 1.2.4"

#r directive can be used in F# Interactive and Polyglot Notebooks. Copy this into the interactive tool or source code of the script to reference the package.

#:package mostlylucid.botdetection@1.2.4

#:package directive can be used in C# file-based apps starting in .NET 10 preview 4. Copy this into a .cs file before any lines of code to reference the package.

#addin nuget:?package=mostlylucid.botdetection&version=1.2.4
                    

                            Install as a Cake Addin

#tool nuget:?package=mostlylucid.botdetection&version=1.2.4
                    

                            Install as a Cake Tool

The NuGet Team does not provide support for this client. Please contact its maintainers for support.

Mostlylucid.BotDetection

DESTROY ALL ROBOTS! (politely, with HTTP 403s)

Bot detection middleware for ASP.NET Core with multi-signal detection, AI-powered classification with continuous learning, auto-updated blocklists, YARP integration, and full observability.

Key Features

Multi-signal detection: User-Agent + headers + IP ranges + behavioral analysis + client-side fingerprinting
AI-powered classification: Heuristic model (<1ms) with optional LLM escalation for complex cases
Continuous learning: Heuristic weights adapt over time based on detection feedback
Composable policies: Separate detection (WHAT) from action (HOW) for maximum flexibility
Stealth responses: Throttle, challenge, or honeypot bots without revealing detection
Auto-updated threat intel: Pulls isbot patterns and cloud IP ranges automatically
First-class YARP support: Bot-aware routing and header injection
Full observability: OpenTelemetry traces and metrics baked in

Why Use This?

When commercial WAF isn't an option:

Self-hosted apps without Cloudflare/AWS/Azure
Compliance requirements prohibiting third-party request inspection
Cost-sensitive projects where $3K+/month WAF isn't justified

When you need more than User-Agent matching:

Bots spoofing browser User-Agents
Scripts missing Accept-Language, cookies, or timing signals
API abuse from datacenter IPs

When you want adaptive protection:

Detection that improves over time with learning
Different policies per endpoint (strict for checkout, relaxed for static content)
Stealth throttling that bots can't detect

Note: For enterprise applications with stringent security requirements, consider commercial services like Cloudflare Bot Management or AWS WAF Bot Control.

Quick Start

1. Install

dotnet add package Mostlylucid.BotDetection

2. Configure Services

using Mostlylucid.BotDetection.Extensions;

var builder = WebApplication.CreateBuilder(args);

builder.Services.AddBotDetection();

var app = builder.Build();

app.UseBotDetection();
app.Run();

3. Recommended Configuration (appsettings.json)

{
  "BotDetection": {
    "BotThreshold": 0.7,
    "BlockDetectedBots": true,
    "DefaultActionPolicyName": "throttle-stealth",

    "EnableAiDetection": true,
    "AiDetection": {
      "Provider": "Heuristic",
      "Heuristic": {
        "Enabled": true,
        "EnableWeightLearning": true
      }
    },

    "Learning": {
      "Enabled": true,
      "EnableDriftDetection": true
    },

    "PathPolicies": {
      "/api/login": "strict",
      "/api/checkout/*": "strict",
      "/sitemap.xml": "allowVerifiedBots"
    }
  }
}

This enables:

AI detection with Heuristic model (sub-millisecond, learns from feedback)
Learning system that improves detection over time
Stealth throttling (bots don't know they're being slowed)
Path-based policies (strict for sensitive endpoints)

Basic Usage

HttpContext Extensions

if (context.IsBot())
    return Results.StatusCode(403);

var confidence = context.GetBotConfidence();
var botType = context.GetBotType();

Endpoint Filters

app.MapGet("/api/data", () => "sensitive")
   .BlockBots();

app.MapPost("/api/submit", () => "ok")
   .RequireHuman();

MVC Attributes

[BlockBots(AllowVerifiedBots = true)]
public IActionResult Index() => View();

Detection Methods

Method	Description	Latency
User-Agent	Pattern matching against known bots	<1ms
Headers	Suspicious/missing header detection	<1ms
IP	Datacenter IP range identification	<1ms
Version Age	Browser/OS version staleness detection	<1ms
Security Tools	Penetration testing tool detection (Nikto, sqlmap, etc.)	<1ms
Project Honeypot	HTTP:BL IP reputation via DNS lookup	~100ms
Behavioral	Rate limiting + anomaly detection	1-5ms
Inconsistency	Cross-signal mismatch detection	1-5ms
Heuristic AI	Feature-weighted classification with learning	<1ms
LLM	Full reasoning (escalation only)	50-500ms
HeuristicLate	Post-AI refinement with all evidence	<1ms

AI Detection & Learning (Key Differentiator)

The AI detection and learning system is what sets this library apart:

Request → Fast Detectors → Heuristic Model → Decision → Learning Bus
                ↓                                ↓            ↓
           Quick signals                   Risk score    Pattern Reputation
                                                ↓            ↓
                                         Action Policy   Weight Updates

Enable with:

{
  "BotDetection": {
    "EnableAiDetection": true,
    "AiDetection": {
      "Provider": "Heuristic",
      "Heuristic": { "Enabled": true, "EnableWeightLearning": true }
    },
    "Learning": { "Enabled": true }
  }
}

See ai-detection.md and learning-and-reputation.md for details.

Action Policies

Control HOW to respond to detected bots:

Policy	Description
`block`	Return 403 Forbidden
`throttle-stealth`	Delay response (bots don't notice)
`challenge`	Present CAPTCHA or proof-of-work
`redirect-honeypot`	Silent redirect to trap
`logonly`	Shadow mode (log but allow)

See action-policies.md for full details.

Documentation

Feature	Description	Docs
Configuration	Full options reference	configuration.md
AI Detection	Heuristic model, LLM escalation, learning	ai-detection.md
Learning & Reputation	Pattern learning, drift detection	learning-and-reputation.md
Action Policies	Block, throttle, challenge, redirect	action-policies.md
Detection Policies	Path-based detection configuration	policies.md
Extensibility	Custom detectors and policies	extensibility.md
User-Agent Detection	Pattern matching with reputation	user-agent-detection.md
Header Detection	HTTP header anomaly analysis	header-detection.md
IP Detection	Datacenter and cloud IP identification	ip-detection.md
Version Age Detection	Browser/OS version staleness detection	version-age-detection.md
Security Tools Detection	Penetration testing tool detection	security-tools-detection.md
Project Honeypot	HTTP:BL IP reputation checking	project-honeypot.md
Behavioral Analysis	Rate limiting and anomaly detection	behavioral-analysis.md
Client-Side Fingerprinting	Headless browser detection	client-side-fingerprinting.md
YARP Integration	Bot-aware reverse proxy	yarp-integration.md
Telemetry	OpenTelemetry traces and metrics	telemetry-and-metrics.md
YARP Gateway	Companion Docker gateway	yarp-gateway.md

Companion Project: YARP Gateway

For edge deployments, use Mostlylucid.YarpGateway - a lightweight Docker-first reverse proxy:

# Zero-config reverse proxy in seconds
docker run -p 80:8080 -e DEFAULT_UPSTREAM=http://your-app:3000 scottgal/mostlylucid.yarpgateway

Why use it with BotDetection?

Edge routing and load balancing
Hot-reload YARP configuration
Admin API for health/metrics
Multi-arch: amd64, arm64, Raspberry Pi (arm/v7)
~90MB Alpine image

See yarp-gateway.md for integration patterns.

Diagnostic Endpoints

app.MapBotDetectionEndpoints("/bot-detection");

// GET /bot-detection/check   - Current request analysis
// GET /bot-detection/stats   - Detection statistics
// GET /bot-detection/health  - Health check

Service Registration Options

// Default: all detectors + Heuristic AI with learning
builder.Services.AddBotDetection();

// User-agent only (fastest, minimal)
builder.Services.AddSimpleBotDetection();

// All detectors + LLM escalation (requires Ollama)
builder.Services.AddAdvancedBotDetection("http://localhost:11434", "gemma3:4b");

Requirements

.NET 8.0 or .NET 9.0
Optional: Ollama for LLM-based detection escalation

License

The Unlicense - Public Domain

Links

Product	Compatible and additional computed target framework versions.
.NET	net8.0 is compatible. net8.0-android was computed. net8.0-browser was computed. net8.0-ios was computed. net8.0-maccatalyst was computed. net8.0-macos was computed. net8.0-tvos was computed. net8.0-windows was computed. net9.0 is compatible. net9.0-android was computed. net9.0-browser was computed. net9.0-ios was computed. net9.0-maccatalyst was computed. net9.0-macos was computed. net9.0-tvos was computed. net9.0-windows was computed. net10.0 was computed. net10.0-android was computed. net10.0-browser was computed. net10.0-ios was computed. net10.0-maccatalyst was computed. net10.0-macos was computed. net10.0-tvos was computed. net10.0-windows was computed.

Product

.NET

Compatible target framework(s)

Included target framework(s) (in package)

Learn more about Target Frameworks and .NET Standard.

net8.0
- Microsoft.Data.Sqlite (>= 9.0.0)
- Microsoft.Extensions.Caching.Memory (>= 9.0.0)
- Microsoft.Extensions.Hosting.Abstractions (>= 9.0.0)
- Microsoft.Extensions.Http (>= 9.0.0)
- Microsoft.Extensions.Logging.Abstractions (>= 9.0.0)
- Microsoft.Extensions.Options (>= 9.0.0)
- OllamaSharp (>= 4.0.1)
- System.IO.Hashing (>= 9.0.0)
net9.0
- Microsoft.Data.Sqlite (>= 9.0.0)
- Microsoft.Extensions.Caching.Memory (>= 9.0.0)
- Microsoft.Extensions.Hosting.Abstractions (>= 9.0.0)
- Microsoft.Extensions.Http (>= 9.0.0)
- Microsoft.Extensions.Logging.Abstractions (>= 9.0.0)
- Microsoft.Extensions.Options (>= 9.0.0)
- OllamaSharp (>= 4.0.1)
- System.IO.Hashing (>= 9.0.0)

NuGet packages

This package is not used by any NuGet packages.

GitHub repositories

This package is not used by any popular GitHub repositories.

Version	Downloads	Last Updated
1.3.5-rc1	51	12/9/2025
1.2.4	144	12/5/2025
1.2.3	137	12/5/2025
1.2.1	156	12/5/2025
1.2.0	152	12/5/2025
1.1.0	160	12/5/2025
1.0.2	152	12/4/2025
1.0.1	160	12/4/2025
1.0.0-preview4	154	12/4/2025
1.0.0-preview3	151	12/4/2025
0.0.3-preview1	630	12/2/2025
0.0.2-preview1	632	12/2/2025

1.2.0

Enhanced Detection Pipeline & Security Layer:

Security Detection:
- SecurityToolContributor: Detects penetration testing tools (SQLMap, Nikto, Nmap, Burp Suite, etc.)
- ProjectHoneypotContributor: HTTP:BL IP reputation lookups via DNS with test mode simulation
- Pattern fetching from digininja/scanner_user_agents and OWASP CoreRuleSet
- Honeypot test mode: Use <test-honeypot:harvester|spammer|suspicious> markers for testing

AI Detection Improvements:
- HeuristicLateContributor: Post-AI refinement layer that runs after LLM for final classification
- Improved LLM prompt for better accuracy with smaller models
- Fixed localhost IP detection - no longer incorrectly flagged as datacenter IP
- Default LLM model upgraded to gemma3:4b for better reasoning

Demo Enhancements:
- Interactive bot simulator with 20+ preconfigured bot types
- Custom User-Agent input field for testing arbitrary UA strings
- UA Only policy for fast User-Agent-only detection testing
- Honeypot simulator buttons for testing Project Honeypot integration
- Security scanner buttons (Nikto, Nessus, Nmap, Burp Suite, Acunetix)

Production Security Defaults:
- ResponseHeaders.Enabled defaults to false (never leak detection details)
- EnableTestMode defaults to false
- Detection results flow downstream only via HttpContext.Items

New Documentation:
- security-tools-detection.md - Comprehensive security tool detection guide
- project-honeypot.md - HTTP:BL integration and testing guide
- Updated ai-detection.md with HeuristicLate contributor details

Integration Tests:
- Production security defaults verification
- Honeypot test mode simulation tests
- Contributor registration tests

New SignalKeys:
- SecurityToolDetected, SecurityToolName, SecurityToolCategory
- HoneypotChecked, HoneypotListed, HoneypotThreatScore, HoneypotVisitorType
- HoneypotTestMode (for test mode simulation)

---

1.0.0

🎉 First Stable Release!

Bot detection middleware for ASP.NET Core with multi-signal detection,
AI-powered classification with continuous learning, and full observability.

Key Features:
- Multi-signal detection: User-Agent, headers, IP ranges, behavioral analysis, client-side fingerprinting
- AI-powered classification: Heuristic model (<1ms) with optional LLM escalation
- Continuous learning: Heuristic weights adapt over time based on detection feedback
- Composable policies: Separate detection (WHAT) from action (HOW)
- Stealth responses: Throttle, challenge, or honeypot bots without revealing detection
- Auto-updated threat intel: isbot patterns and cloud IP ranges
- Full observability: OpenTelemetry traces and metrics

Breaking Changes from Preview:
- Default LLM model changed from gemma3:1b to gemma3:4b (better reasoning)
- Default LLM timeout increased from 2000ms to 5000ms
- ONNX provider removed in favor of Heuristic provider (faster, no external dependencies)

Migration Guide:
- Replace "Provider": "Onnx" with "Provider": "Heuristic" in config
- Update Ollama model if using LLM escalation: gemma3:4b recommended

---

0.5.0-preview2

Composable Action Policy System:
- Named action policies separate from detection policies
- Built-in: block, throttle, challenge, redirect, logonly
- BlockActionPolicy: Configurable status codes, messages, headers
- ThrottleActionPolicy: Jitter, risk-scaling, exponential backoff, stealth mode
- ChallengeActionPolicy: CAPTCHA, JavaScript, proof-of-work challenges
- RedirectActionPolicy: Honeypot, tarpit, error page with templates
- LogOnlyActionPolicy: Shadow mode, debug headers, metrics
- IActionPolicyFactory for configuration-based creation
- IActionPolicyRegistry for runtime policy lookup
- Custom action policies via IActionPolicy interface
- [BotAction("policy-name")] attribute for endpoint overrides
- ActionPolicyName property on detection policies and transitions

---

0.5.0-preview1

Policy-Based Detection:
- Named policies with configurable detectors per endpoint
- Path-based resolution with glob patterns (/api/*, /admin/**)
- Built-in: default, strict, relaxed, allowVerifiedBots
- Transitions based on risk thresholds and signals
- Per-policy weight overrides
- Actions: Allow, Block, Challenge, Throttle, EscalateToAi

Management Endpoints & Attributes:
- MapBotPolicyEndpoints() for CRUD and testing
- [BotPolicy("strict")] attribute for controllers/actions
- [BotDetector("UserAgent,Header")] for inline ad-hoc detection
- [SkipBotDetection] to bypass detection
- Policy simulation endpoint for testing transitions

Response Headers & TagHelpers:
- Configurable response headers (X-Bot-Risk-Score, X-Bot-Policy)
- <bot-detection-result /> TagHelper for client-side JS integration
- Full JSON result via Base64-encoded header option

Throttling with Jitter:
- Configurable base delay, max delay, jitter percent
- ScaleByRisk option for risk-proportional delays
- DelayResponse to slow bots at TCP level
- Custom throttle messages and challenge types

Blackboard Architecture:
- Event-driven detection with evidence aggregation
- Wave-based parallel execution
- Trigger conditions (WhenSignalExists, WhenRiskExceeds)
- Circuit breakers per detector
- Early exit for high-confidence detections

Pattern Reputation System:
- BotScore, Support, State tracking with time decay
- Online EMA updates, state machine (Neutral → Suspect → ConfirmedBad)
- Manual overrides never auto-downgrade
- Drift detection for bot/human ratio changes

Fast/Slow Path:
- Fast path: sync detectors (<100ms)
- Slow path: async AI/learning via LearningEventBus

New Interfaces:
- IContributingDetector, ILearningEventHandler, IPatternReputationCache
- IPolicyRegistry, IPolicyEvaluator

SQLite Pattern Store, comprehensive documentation

---

0.0.5-preview1

Client-Side Fingerprinting:
- BotDetectionTagHelper for fingerprint collection JS
- Signed token system prevents spoofing
- Detects headless browsers, automation markers

Inconsistency Detection:
- Catches UA/header mismatches
- Cross-signal contradiction detection

Risk Assessment:
- RiskBand enum (Low, Elevated, Medium, High)
- GetRiskBand(), ShouldChallengeRequest(), GetRecommendedAction()

Session-Level Behavioral Analysis:
- Multi-identity tracking (IP, fingerprint, API key, user)
- Anomaly detection (spikes, new paths, timing)

---

0.0.4-preview1

- ONNX-based detection (1-10ms latency)
- Source-generated regex for performance
- OpenTelemetry metrics integration
- YARP reverse proxy integration

---

0.0.3-preview2: Security fixes (ReDoS, CIDR validation)
0.0.3-preview1: Documentation improvements
0.0.2-preview1: Background updates, SQLite storage
0.0.1-preview1: Initial release