using nlp to filter through prayers

you carry an intention you can barely name, a worry, a hope, a goal, and somewhere in the ninety-nine names of allah, the quran, and the hadith there is language already shaped for exactly that. duaos is built to close that gap by meaning, not keywords: you describe what's on your heart, it finds the names and verses that match, and then it helps you refine a du'a in a prophetic style. i wanted it to be technically serious precisely because the subject is sacred. retrieval that touches scripture has to refuse to lie.

cosine search, in the database

every name, verse, and narration is embedded into a 1536-dimension vector and stored in postgres with pgvector. the match happens in sql, cosine distance via the <=> operator, with the similarity returned as 1 - distance and a jsonb containment filter that lets one function serve both the open search and the typed fallbacks.

SELECT id, content, metadata,
       1 - (embedding <=> query_embedding) AS similarity
FROM spiritual_assets
WHERE embedding IS NOT NULL
  AND (filter_metadata = '{}' OR metadata @> filter_metadata)
  AND (1 - (embedding <=> query_embedding)) >= match_threshold
ORDER BY embedding <=> query_embedding
LIMIT match_count;

that @> containment operator is the quiet hero, it means a single rpc handles "find anything relevant" and "find me exactly one bukhari hadith" without a second function. if a category comes back empty, three parallel queries with the threshold dropped to zero guarantee you always get at least one name, some verses, and some hadith. the user is never left empty-handed.

hyde: hallucinate, then search with it

short queries are the hard case. "i'm anxious" is three words and barely enough signal for a vector to grab. so for any query under twelve words, duaos uses a technique called hyde, hypothetical document embeddings: it asks a model to write a short, made-up du'a-style passage for the query, embeds that, and averages the two vectors element-wise.

export function blendEmbeddings(queryEmbedding, hydeEmbedding) {
  if (queryEmbedding.length !== hydeEmbedding.length) return queryEmbedding;
  return queryEmbedding.map((q, i) => (q + hydeEmbedding[i]) / 2);
}

you're not searching with the user's three words, you're searching with the richer shape of what they probably mean. the hallucination is a throwaway, never shown, used only to aim the search. if it fails, the code falls straight back to the raw embedding.

retrieval here is less a speed trick than a discipline: the model may only speak from what it was given, and what it was given is the actual text.

reranking for du'a, against jurisprudence

raw similarity has a failure mode here that matters: a legal ruling about marriage scores high on a query about wanting a spouse, but it isn't a du'a, it's fiqh. so a deterministic rerank multiplies the score by lexical signals, boosting supplication language and penalizing rulings and bare narrator fragments.

if (hasDuaSignal(m.content))          multiplier *= 1.2;   // "o allah", "grant me"
if (hasJurisprudenceSignal(m.content)) multiplier *= 0.75;  // "the ruling", "iddah"
if (isShortNarratorOnly(m.content))    multiplier *= 0.75;
sim = Math.min(1, sim * multiplier);

then a small model does a final relevance pass to drop the tangential, and on any error it returns everything unfiltered. graceful degradation at every llm step, because the worst outcome is a blank page when someone came looking for words to pray.

the rule that matters most: never invent arabic

this is the line i care about more than any latency number. the refine agent runs on a strict instruction: arabic script is allowed only when copied verbatim from a provided snippet. it must not generate, reconstruct, or transliterate into arabic on its own. a model confidently producing fake quranic arabic would be worse than useless, it would be a small act of corruption. so the prophetic-style refinement happens in the user's language, grounded in real retrieved text, and the sacred script is quoted, never synthesized. there's even a hardcoded prior: ask about a spouse or provision and the du'a of musa, rabbi inni lima anzalta ilayya min khairin faqir (al-qasas 28:24), gets injected into context, because some matches are too well-established to leave to a vector.

retrieval-augmented generation is usually sold as a way to make models more accurate. here it's something closer to a discipline: the model may only speak from what was given, and what was given is the actual text. the prophet ﷺ taught that du'a is the very essence of worship. the least i could do was build the tool so it never puts words in god's mouth. آمين.