Dobco.Nitpicker.AI 1.0.0

Nitpicker AI

AI-powered code review tool that reviews pull requests against configurable rule sets using AWS Bedrock (Claude).

Features

• Rule-based reviews — define review rules in markdown prompt files (per repo)
• Confidence scoring — every finding includes a confidence score; low-confidence issues are suppressed into a collapsible section rather than posted as inline comments
• Agentic context lookup — the AI can grep, find, and cat files in the checked-out codebase to verify issues before raising them
• GitHub suggestion blocks — suggestible rules produce one-click "Apply suggestion" buttons on GitHub
• Fallback posting — gracefully handles unresolvable diff positions by falling back to individual comments or issue comments
• Deduplication — skips issues already reported in previous runs on the same PR
• Batch processing — handles large diffs by splitting into manageable chunks
• Lines-of-code budget — configurable limit on context resolution to control token usage

How it works

1. Fetches the PR diff from GitHub
2. Parses the diff and filters to reviewable files (.cs, .json, etc.)
3. For each prompt file in the repo's nitpicker-prompts/ directory:
   • Sends the diff to AWS Bedrock with the prompt as system instructions
   • The AI may request context lookups (grep/find/cat) against the local checkout
   • Receives structured review issues with rule IDs, severity, confidence scores, and optional code suggestions
4. Filters issues by confidence threshold (configurable, default 70)
5. Validates issue locations against the local checkout to catch hallucinated line numbers
6. Posts the review to GitHub as inline comments (pinned to diff lines), a summary table, and a collapsible section for suppressed low-confidence findings

Installation

As a .NET global tool

dotnet tool install --global NitpickerAI

After installation, the nitpicker command is available globally.

As a .NET local tool (per-repo)

dotnet new tool-manifest   # if you don't have a manifest yet
dotnet tool install NitpickerAI

Then run via dotnet nitpicker.

From source

git clone https://github.com/DedalusDIIT/nitpicker-ai.git
cd nitpicker-ai
dotnet run --project Nitpicker -- --dry-run

Setup

Prerequisites

• .NET 9 SDK
• AWS Bedrock access with an API key
• GitHub App or personal access token with PR read/write permissions

Environment variables

Usage

As a dotnet tool

Once installed, run the tool with environment variables set:

# Set required environment variables
export GITHUB_TOKEN=ghp_...
export PR_NUMBER=42
export REPO=myorg/myrepo
export BASE_SHA=abc123
export HEAD_SHA=def456
export AWS_REGION=eu-central-1
export AWS_BEDROCK_API_KEY=your-key

# Run the review (results posted to GitHub)
nitpicker

# Dry-run mode (prints to console, does not post)
nitpicker --dry-run

# Replay mode (re-renders saved issues without calling Bedrock)
nitpicker --replay

GitHub Actions

Nitpicker needs a GITHUB_TOKEN with permission to read contents and write pull request reviews. The default GITHUB_TOKEN provided by Actions has limited permissions, so the recommended approach is to use a GitHub App and generate an installation token at runtime using peter-murray/workflow-application-token-action:

1. Create a GitHub App with Pull requests: Read & write and Contents: Read permissions
2. Install it on the repositories you want to review
3. Store the App ID as a repository variable (NITPICKER_APP_ID) and the private key as a secret (NITPICKER_APP_PRIVATE_KEY)

Example workflow using the dotnet tool:

name: Nitpicker AI

on:
  pull_request:
    types: [opened, synchronize]

permissions:
  contents: read

jobs:
  nitpicker-ai:
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Get GitHub App Token
        id: app_token
        uses: peter-murray/workflow-application-token-action@v4
        with:
          application_id: ${{ vars.NITPICKER_APP_ID }}
          application_private_key: ${{ secrets.NITPICKER_APP_PRIVATE_KEY }}

      - name: Setup .NET
        uses: actions/setup-dotnet@v4
        with:
          dotnet-version: '9.0.x'

      - name: Install Nitpicker
        run: dotnet tool install --global NitpickerAI

      - name: Run Nitpicker AI
        env:
          GITHUB_TOKEN:                     ${{ steps.app_token.outputs.token }}
          PR_NUMBER:                        ${{ github.event.pull_request.number }}
          REPO:                             ${{ github.repository }}
          BASE_SHA:                         ${{ github.event.pull_request.base.sha }}
          HEAD_SHA:                         ${{ github.event.pull_request.head.sha }}
          AWS_REGION:                       ${{ vars.AWS_REGION }}
          AWS_BEDROCK_API_KEY:              ${{ secrets.AWS_BEDROCK_API_KEY }}
          AWS_BEDROCK_INFERENCE_PROFILE_ID: ${{ vars.AWS_BEDROCK_INFERENCE_PROFILE_ID }}
          NITPICKER_PROMPTS_DIR:            nitpicker-prompts
          NITPICKER_REPO_PATH:              ${{ github.workspace }}
        run: nitpicker

The consuming repo provides its own nitpicker-prompts/ directory with rule definition files.

Alternative: run from source checkout

If you prefer not to use NuGet, you can check out the tool repository directly:

      - name: Checkout Nitpicker
        uses: actions/checkout@v4
        with:
          repository: DedalusDIIT/nitpicker-ai
          path: .nitpicker

      - name: Run Nitpicker AI
        env:
          # ... same env vars as above ...
        run: dotnet run --project .nitpicker/Nitpicker --configuration Release

Local development with Run-Nitpicker.ps1

The PowerShell script handles token generation, PR metadata fetching, and environment setup for you.

1. Create a .nitpicker.env file

AWS_REGION=eu-central-1
AWS_BEDROCK_API_KEY=your-bedrock-api-key
AWS_BEDROCK_INFERENCE_PROFILE_ID=eu.anthropic.claude-sonnet-4-5-20250929-v1:0
NITPICKER_PROMPTS_DIR=C:\path\to\your-repo\nitpicker-prompts
NITPICKER_REPO_PATH=C:\path\to\your-repo

2. Run against a PR

# Using GitHub App authentication (default — generates token automatically)
.\Run-Nitpicker.ps1 https://github.com/YourOrg/your-repo/pull/42 -DryRun

# With explicit App ID and key path
.\Run-Nitpicker.ps1 https://github.com/YourOrg/your-repo/pull/42 -DryRun `
    -AppId 12345 `
    -PrivateKeyPath "C:\keys\your-app.pem"

# Post results to GitHub (remove -DryRun)
.\Run-Nitpicker.ps1 https://github.com/YourOrg/your-repo/pull/42

3. Using a personal access token instead

Add GITHUB_TOKEN to your .nitpicker.env:

GITHUB_TOKEN=ghp_your_personal_access_token
AWS_REGION=eu-central-1
AWS_BEDROCK_API_KEY=your-bedrock-api-key

Dry-run and replay modes

Dry-run runs the full pipeline (fetches diff, calls Bedrock AI) but prints results to the console instead of posting to GitHub. It also saves all output for replay:

.\Run-Nitpicker.ps1 https://github.com/YourOrg/your-repo/pull/42 -DryRun

Output is saved to nitpicker-output/{repo}/{pr}/:

• {prompt}.issues.json — raw issues from the AI
• {prompt}.review.md — formatted review body
• {prompt}.comments.json — inline comments with diff positions

Replay reads from saved issues and re-runs formatting, pinning, and dedup — without calling Bedrock. Use this to iterate on rendering without burning tokens:

.\Run-Nitpicker.ps1 https://github.com/YourOrg/your-repo/pull/42 -Replay

Writing prompt files

Each .md file in the prompts directory defines a review focus area. The file content becomes the AI's system instructions. Example structure:

You are a senior software engineer performing a focused security review.
Your ONLY job is to find security issues.

## Rules to enforce
- SEC001: SQL injection via string concatenation
- SEC002: Secrets hardcoded in source

## Suggestible rules
For the following rules, provide a `suggestedCode` field with the exact replacement code.
SEC001

The ## Suggestible rules section is optional. Rules listed there will produce GitHub "Apply suggestion" buttons when the AI provides a fix.

Publishing

Pack the tool

dotnet pack Nitpicker/Nitpicker.csproj --configuration Release

This produces Nitpicker/bin/Release/NitpickerAI.1.0.0.nupkg.

Push to NuGet

dotnet nuget push Nitpicker/bin/Release/NitpickerAI.1.0.0.nupkg \
    --api-key YOUR_NUGET_API_KEY \
    --source https://api.nuget.org/v3/index.json

Push to a private feed (e.g. Azure Artifacts)

dotnet nuget push Nitpicker/bin/Release/NitpickerAI.1.0.0.nupkg \
    --api-key az \
    --source https://pkgs.dev.azure.com/yourorg/_packaging/yourfeed/nuget/v3/index.json

Running tests

dotnet test Nitpicker.Tests

Project structure

nitpicker-ai/
├── Nitpicker/                  # Main application (dotnet tool)
│   ├── Bedrock/                # AWS Bedrock API client and review logic
│   ├── Configuration/          # Environment variable configuration
│   ├── DiffParsing/            # Git diff parser
│   ├── Github/                 # GitHub API client and context resolver
│   ├── Reviewing/              # Review formatting, filtering, pinning, posting
│   └── Program.cs              # Entry point
├── Nitpicker.Tests/            # Unit tests (xUnit)
├── Run-Nitpicker.ps1           # Local development runner
└── Nitpicker.sln