Back to Projects

Narsil

Technology

Narsil is a search engine for full-text, vector, hybrid, and geospatial queries on one typed index, which you can embed in your app or deploy behind its own server.

Narsil, the search engine that scales with your data, for full-text, vector, hybrid, and geosearch queries on one typed index

Narsil is an open-source search engine. It splits a large index into partitions and merges the results of those partitions into one ranked list. It stores every index in one binary format, .nrsl, which an implementation in another language can load. It works in Node.js, Bun, Deno, and the browser. It is already in production, since I use Narsil for the Cmd+K search on this site and on both project documentation sites. Every feature has a page on the documentation site. You can read the source on GitHub and install the package from npm as @delali/narsil.

Four search modes in one engine

The engine scores with and supports field boosting, fuzzy matching through a bounded Levenshtein distance, and thresholds on both term coverage and score. You can combine a query with filters, facets, sorting, grouping, highlighting, cursor pagination, pinned results, and autocomplete suggestions.

Benchmark results

I measure Narsil in two classes. Over HTTP, against Elasticsearch, OpenSearch, Qdrant, Weaviate, Typesense, and Meilisearch on BEIR SciFact and NFCorpus, Narsil's BM25 ranking has the top on both, at 0.6814 and 0.3278 against 0.6789 and 0.3206 for Elasticsearch and OpenSearch. Although those margins are narrow, Narsil processes more keyword queries a second than either engine, at 4,200 on SciFact against 2,796 for Elasticsearch and 3,027 for OpenSearch, and at 4,940 on NFCorpus against their 3,128 and 3,252. On vector search at the same recall, it processes the most queries a second on SciFact and NFCorpus, while OpenSearch processes more on the DBpedia set of 995,000 documents, at 869 against Narsil's 670. In one process, against Orama and MiniSearch, Narsil has the top nDCG@10 on SciFact, at 0.6840 against 0.4351 and 0.2506. Its BM25 ranking is within 0.006 nDCG@10 of the published Anserini baseline on both BEIR datasets. A script fills the full results from the recorded run of 28 September 2026.

The engine routes each document to a partition by a deterministic hash. When you call rebalance(), it reshapes the partitions online and buffers incoming writes in a write-ahead queue. You can choose among three scoring modes for BM25 statistics that differ across partitions and instances. The HTTP server subpath wraps the engine in a REST API with health probes, bulk NDJSON import, and snapshot and restore endpoints. For observability, the engine emits events and typed errors, accepts plugins, and reports its memory use. Narsil has 107 language modules, each a separate entry point. For thirty-seven of those languages, the engine tokenises, stems, and filters stop words. Seven of them are written without spaces between words, which is why the engine splits their text into character n-grams. For the remaining sixty-three, it tokenises and filters stop words.

Where this is going

When a team chooses a search engine, it can find itself trading power against ease of operation. Elasticsearch is powerful, though a team has to put real operational work into keeping it up. A lightweight library is easy to embed, though it can slow down under a production workload. I build Narsil to be both at once, so that you can embed it in your application with no server to operate and move the same engine onto a server once your workload needs one.

The deeper ambition is language equity. Much of the search software in wide use is built around European languages, so a reader who searches in a language that has no stemmer, tokeniser, or stop-word list gets worse results than an English reader does for the same effort. For sixty-three of the 107 languages, the engine tokenises and filters stop words but has no stemmer, because Snowball publishes none for them. I'll build those stemmers next, with native speakers and computational linguists.

The goal is wider than any one language. One engine, one index format, and one query API should serve a browser tab, a single server process, and a cluster alike, in whichever language the application is written in. A team should be able to keep the engine that it starts with as its workload grows, with both the power and the ease of operation.

More in Technology

All projects

Search

Search for blog posts, studies, projects, and pages