SSiddiq.in
RAG Β· Vector search

Your knowledge, ask-able.

Every SOP, contract, wiki page and PDF β€” searchable in plain English. Cited answers. Access-controlled. Not another Slack thread asking "where's that doc?"

What's our refund policy for enterprise?
Answer. Enterprise refunds are pro-rated within the first 30 days of a new annual term. Beyond 30 days, credits apply for the remainder.
πŸ“„ Sales-Handbook.pdf Β· p.14 Β· Refund policy
πŸ“„ MSA-Template-v3.docx Β· Β§9.2
πŸ“„ Notion / Ops / Finance policies

Under the hood.

Stage 1

Ingest

Notion, Drive, Confluence, Sharepoint, PDFs, GitHub, Slack

Stage 2

Chunk

Semantic splitting with overlap for context preservation

Stage 3

Embed

OpenAI, Cohere or open-source models depending on cost

Stage 4

Store

pgvector, Pinecone or Qdrant with metadata filters

Stage 5

Retrieve

Hybrid search: keyword + vector with re-ranking

Stage 6

Answer

GPT-4 / Claude with citations and confidence score

Where teams put it to work.

Internal wiki AI

Ask any question about your ops, product, HR, finance.

Customer-facing search

Turn your docs site into a chat interface.

Compliance & policy

Instant answers grounded in your governance docs.

Enterprise-safe by default.

  • ACL-aware retrieval
  • No training on your data
  • Source citations always
  • Prompt injection guardrails
  • Audit log of every query
  • Self-hosted option available

RAG FAQs.

The AI looks things up in your own documents before answering, so it stays grounded in your data instead of guessing.

Chunking strategy β€” the choice that decides 60% of retrieval quality

Vector DB and model choice matter less than how you chunk. Here's what actually moves the needle.

StrategyChunk sizeBest forFailure mode
Fixed-token300-500 tokensHomogeneous proseSplits mid-sentence, mid-table
Recursive character500-1200 with overlapMixed docsOverlap inflates index cost
Semantic (embedding-based split)VariableHigh-quality proseSlow to build
Structural (heading-aware)One per H2/H3Docs, manualsFails on unstructured PDFs
Proposition-based1-3 sentencesQA-heavy corporaExtraction cost at ingest
Parent-child (small-to-big)Retrieve small, feed largeLong-context modelsMore complex index

Retrieval evaluation β€” the metrics that matter

"It works in demos" is not evaluation. These are the metrics we track on every retrieval change.

  • Context recall

    Of the facts needed to answer, what % appear in retrieved chunks? Below 80% means retrieval is the ceiling, not the LLM.

  • Context precision

    Of retrieved chunks, what % are actually relevant? Low precision wastes context window and confuses the model.

  • Answer faithfulness

    Does the answer only use facts from context? Measured with an LLM-as-judge on a held-out set.

  • Answer relevance

    Does the answer actually address the question? Catches on-topic-but-off-question failures.

  • Hit@k and MRR

    Classic IR metrics on a labelled question set β€” the fastest way to compare embedding models.

  • Latency p95

    Retrieval + generation p95 under 3s or users abandon. Reranker adds 200-800ms; budget accordingly.

Turn docs into answers.

Start with a pilot
Automate this workflow

Automate what's costing your team hours

Describe the workflow β€” we'll reply with what's possible, what it costs, and where to start.

  • Personal reply from Siddiq within 24 hours
  • Honest advice β€” no pushy sales, no fake urgency
  • Clear next steps, timeline, and estimate

We only use your details to reply to this enquiry. Privacy.