Falqobit.Fahras 0.1.1

Falqobit.Fahras

Arabic full-text search for .NET. A persistent, on-disk index tuned for the ways Arabic text actually gets typed: with or without diacritics, with hamza and alef spelled inconsistently, with spaces in the wrong places or missing entirely.

Fahras (فهرس — "index") is lexical, not semantic. There are no embeddings and no vector search. Recall comes from indexing every text field five different ways and OR-ing boosted queries across all of them.

index.Add(users);
index.Search("محمد إبراهيم");   // also matches مُحَمَّد إِبْرَاهِيم, محمد ابراهيم, محمدابراهيم

Install

dotnet add package Falqobit.Fahras

Targets net10.0.

Getting started

With dependency injection

services.AddFahras<AdUser>("ad_users_index", opt => opt
    .WithPlainField(u => u.Username)
    .WithFields(u => u.FullName, u => u.Department)
    .WithIndexDir(@"E:\indexes"));

Then inject IFahras<AdUser> anywhere. AddFahras lives in the Microsoft.Extensions.DependencyInjection namespace, so no extra using is needed. One index per type T — registering a second for the same T throws at startup.

IFahras<T> carries the operations only — searching, reading and writing. Field registration is not on it, because an index is configured once at construction; nor is IDisposable, because the container owns the singleton's lifetime. Depend on it rather than on Fahras<T> so your own services can be tested against a substitute — though note that a substitute drops the matching behaviour that is the point of the index, so tests worth writing about recall still want a real index over a temp directory.

Standalone

using var index = Fahras<AdUser>.Create("ad_users_index")
    .WithPlainField(u => u.Username)
    .WithFields(u => u.FullName, u => u.Department)
    .WithIndexDir(@"E:\indexes");

Fields

WithIndexDir(dir) resolves to dir/indexes/{name}. Omit it and the index lands in {CurrentDirectory}/indexes/{name}, on either construction path.

T is stored as JSON and read back with System.Text.Json, so T must round-trip through STJ — no constructor-only types without matching JSON support.

Searching

List<SearchResult<AdUser>> hits = index.Search("محمد");
foreach (var hit in hits)
{
    Console.WriteLine($"{hit.Id} {hit.Item.FullName} {hit.Score}");
}

With no options, Search returns at most 50 results using the Basic and Normalized strategies. To choose your own:

var options = SearchOptions.Create()
    .WithMaxResults(20)
    .WithMinScore(0.4f)
    .WithNormalizedQuery()
    .WithNormalizedStemmedQuery()
    .WithNormalizedNoSpacesQuery();

var hits = index.Search("محمدابراهيم", options);

Each strategy is a separate query, OR-ed into one boolean query with its own boost:

Scores are normalized against the top hit, so Score is relative within one result set and not comparable across searches. Results scoring at or below MinScore (default 0) are dropped.

Normalization is also available directly, via the static ArabicText class: NormalizeArabicText, RemoveNoise, Stem, InsertSpaces.

Writing

index.Add(users);                        // append; each item gets a fresh id
index.Update(u => u.Username, users);    // upsert keyed on a WithPlainField field
index.Delete(docIds);
index.DeleteAll();                       // empty the index, keep the directory
index.DeleteIndexDir();                  // delete the directory outright

Update derives a stable document id from the key field, so SearchResult.Id for a given item survives re-indexing. Add mints a new id every time.

Concurrency

Fahras is built for many reader processes and an occasional writer sharing one directory on disk — for example a web API searching an index that a background service rewrites on a schedule.

• Reads take no lock. Any number of threads or processes may search the same directory at the same time. Searching a directory that has no index yet returns an empty result rather than throwing, and Count is 0.
• Each write call is one locked, committed batch. Exactly one writer is permitted per directory, enforced machine-wide by a write.lock file. Every write opens a writer, runs the batch, and commits before returning.
• A write colliding with one already in progress throws LockObtainFailedException after the write-lock timeout (1 second by default, see WithWriteLockTimeout).

Two consequences worth designing around:

• Pass whole batches. Every call is a separate lock acquisition and commit, so Update in a loop is dramatically slower than handing it the whole sequence at once.
• Readers only observe committed state. There is no read-your-own-uncommitted-writes, and a batch that throws partway rolls back entirely — no partial batch is ever visible.

Building

dotnet build Falqobit.Fahras.slnx
dotnet test  Falqobit.Fahras.slnx

Documentation

Full documentation is at https://fahras.falqobit.sa/docs/.

Licence

Elastic License 2.0 — source-available, not open source.

Free to use, in production, in commercial and closed-source products, whatever the size of your company. No thresholds, no registration, no licence key, no telemetry.

The one restriction: you may not offer Fahras to third parties as a hosted or managed search service.

If you are not sure whether your use case is covered, ask — https://fahras.falqobit.sa. Hosted Fahras, support and integration are also available.

Copyright © Falqobit.