RagMeUp.Client 0.6.0

RAG engine

Document RAG over Postgres + pgvector. The engine owns ingestion, embedding, and reranking; the app owns filtered retrieval as direct SQL on tables it owns.

service/   Python FastAPI engine (`ragmeup` package): /embed, /documents, /rerank
client/    C# read-path library (RagMeUp.Client): rank, build passages, rerank, read spans
db/        schema.sql — documents + chunks tables, HNSW + GIN indexes
           migrations/ — deltas on top of it, applied by the client at startup

Each half carries its own Azure DevOps pipeline (*/azure-pipelines.yml) and releases independently — the client to nuget.org, the engine as a container image. See CI/CD.

Database

Needs pgvector >= 0.7.0. A .NET application sets its own schema up at startup, on its own connection:

await using (var ddl = await ownerSource.OpenConnectionAsync())
    await RagSchema.Upgrade(ddl, engineHttp);

Passing the engine client is what makes it a compatibility check: it reads the schema version that engine requires and refuses, before touching the database, when the installed package is too old — naming the version to install instead of failing later on a missing column. The engine itself never runs DDL on your tables (why).

Anything not .NET, or a database you would rather build by hand:

psql "$DATABASE_URL" -f db/schema.sql

Same result, and an existing database built this way is adopted in place the first time a client runs. See database setup.

Service

Models live on a volume mounted at /models, not baked into the image, so code changes rebuild and push only small layers. A one-shot seed populates the volume.

cd service
DATABASE_URL=postgres://… docker compose up --build

Compose runs model-seed first (downloads weights to the models volume), then starts engine offline against it. On OpenShift, the equivalent is a PVC at /models seeded by an init Job (or initContainer) running python -m ragmeup.download_models.

Local dev without containers: pip install -r requirements.txt then DATABASE_URL=… uvicorn ragmeup.main:app (from service/). Models download from the hub on first run.

On a GPU dev box, install torch first, from PyTorch's CUDA wheel index (--index-url https://download.pytorch.org/whl/cu130), matching your driver; Blackwell cards need cu128 or later. requirements.txt names no index, and on Windows the default PyPI wheel is CPU-only, so the requirements install leaves a GPU box quietly running every embed and rerank on the CPU. Verify with torch.cuda.is_available(), and check the card's compute capability appears in torch.cuda.get_arch_list(), or the kernels are JIT-compiled from PTX at first use. This is the same trap the image flavours below spell out, from the other direction.

Then keep it that way by naming the device instead of leaving it to torch: with RAG_EMBED_DEVICE and RAG_RERANK_DEVICE set to cuda:0, a venv that has lost its CUDA wheel fails with No CUDA GPUs are available instead of quietly serving from the CPU at a fraction of the speed. The run-backend*.ps1 dev scripts pin it for this reason (-Device cpu when you want the CPU path deliberately).

You should not have to infer any of this, so the engine states it at startup, one line per model:

INFO ragmeup.main: embedder BAAI/bge-m3 on cuda:0
INFO ragmeup.main: reranker BAAI/bge-reranker-v2-m3 on cuda:0

Read that line before trusting a benchmark. Note where a wrong device instead fails: on the first /embed or /rerank, as a 500, not at startup. Constructing a backend does not touch the device (FlagEmbedding moves the weights on the first forward pass), so lifespan builds both models and /healthz answers 200 on a box whose GPU is gone. A 200 there proves the weights are resident, which is what it claims; it does not prove they are on the device you asked for. The startup line is what proves that.

Image flavours

One Dockerfile, two bases, selected by FLAVOR — the shared tail (deps, app, entrypoint) is written once so the flavours cannot drift:

docker build .                          # cuda, the default — GPU host (~8 GB)
docker build --build-arg FLAVOR=cpu .   # cpu — CI and a GPU-less VPS (~2.4 GB)

The CPU flavour installs torch from PyTorch's CPU wheel index; a plain pip install torch would pull the CUDA build and ~2.5 GB of nvidia-* wheels that a CPU host can never use.

Configuring a container

Everything is environment variables — the image has no config file. Only DATABASE_URL is required; every other knob has a working default, and the model paths are already set in the image.

docker run -d --name rag-engine \
  -e DATABASE_URL=postgres://user:pass@db:5432/ragdb \
  -e RAG_TORCH_THREADS=4 \
  -v models:/models -e HF_HUB_OFFLINE=1 -e TRANSFORMERS_OFFLINE=1 \
  -p 127.0.0.1:8000:8000 \
  ghcr.io/jgauffin/ragmeup-engine:latest

(†) can be set per target. These decide what an ingest writes, so they belong to a corpus rather than to the process: an engine serving several applications can chunk and enrich each one's documents differently. Give the target an object instead of a DSN in RAG_TARGETS — see sharing one engine. Everything else here (the embedder, the rerankers, devices, thread counts) is a property of the loaded process and is necessarily shared.

Two of these are not really optional in production:

• Publish on loopback, not 0.0.0.0. No endpoint is authenticated and /documents writes with a privileged connection that bypasses your RLS by design. The engine is a trusted internal service — see deployment: trust boundary & scaling.
• Set RAG_TORCH_THREADS. Unset, torch claims every logical core, which over-subscribes SMT pairs (measured: 8 → 16 threads at 0.98x, i.e. slower), and concurrent /rerank calls then fight for the whole machine.

For a Linux VPS there are two ready-made deployments. service/deploy/vps/ is the recommended one — Podman quadlets with unattended, health-gated auto-update, and the engine on an internal network with no published port at all; see hosting the engine on a Linux VPS. service/docker-compose.vps.yml is the compose equivalent for a host without Podman: it pulls the published image, reserves no GPU, binds loopback, and reads the knobs above from a .env file.

The container also serves GET /healthz — cheap, dependency-free, and 200 once the models are resident. Both deployments gate their health check on it.

Client

The app builds an NpgsqlDataSource with UseVector() and points HttpClient.BaseAddress at the engine. Filters live in the app's own SQL, which produces the candidate chunk-id set passed to Rank.

var q        = await rag.Embed(query);            // engine /embed
long[] ids   = await app.FilteredCandidates(...); // app's WHERE: RLS, geo, joins
var matches  = await rag.Rank(conn, q, query, ids, k: 50); // same `query` as embedded
var passages = rag.BuildPassages(matches);
var top      = await rag.Rerank(query, passages, topK: 5);
var spans    = await rag.ReadSpans(conn, top);    // clean original text for /answer

Methods drop the Async suffix — the whole client is async, so it adds nothing.

Tests

Both sides isolate the expensive boundaries (the GPU models, the database) so the logic around them is tested directly, and a thin layer of integration tests exercises the real SQL against Postgres+pgvector.

A throwaway pgvector instance for the integration tests:

docker run -d --name rag-test-pg -e POSTGRES_PASSWORD=test -e POSTGRES_DB=ragtest \
  -p 55432:5432 pgvector/pgvector:pg16

Service (service/) — unit tests need no model and no DB (the model boundary is mocked); the /documents integration test needs the DB above.

cd service
python -m venv .venv && .venv/Scripts/pip install -r requirements-dev.txt
.venv/Scripts/python -m pytest                       # unit only (integration auto-skips)
RAG_TEST_DATABASE_URL=postgres://postgres:test@localhost:55432/ragtest \
  .venv/Scripts/python -m pytest                     # + the /documents integration test

Client (client/) — unit tests are pure (no DB/HTTP). The integration tests use RAG_TEST_DATABASE_URL if set, otherwise spin a pgvector container via Testcontainers; they auto-skip when neither is available.

cd client
dotnet test tests/RagMeUp.Client.Tests               # unit
RAG_TEST_DATABASE_URL=postgres://… \
  dotnet test tests/RagMeUp.Client.IntegrationTests  # rank/read-path vs real pgvector

Both suites honour the same RAG_TEST_DATABASE_URL, so one local pgvector serves both. The integration tests reset their tables on start, so point it at a scratch database, not one with data you care about.

Retrieval quality (client/tests/RagMeUp.Client.EvalTests) — a labelled corpus in eval/ measured through the real read path. Needs a running engine with real weights as well as the database, and skips without one. Reports recall / MRR / nDCG rather than asserting a bar; the assertions are comparative, proving the harness can see a deliberate sabotage. See measuring retrieval quality.

RAG_EVAL_ENGINE_URL=http://127.0.0.1:8000 RAG_TEST_DATABASE_URL=postgres://… \
  dotnet test tests/RagMeUp.Client.EvalTests

The engine's own DATABASE_URL must point at that same database — it ingests, the harness reads. Do not run this concurrently with the integration suite against one database: both truncate the corpus tables. Set RAG_EVAL_DATABASE_URL to separate them.