Aquib Haq

github.com/Clufy/Sentinel · open source · MIT

Sentinel: AI compliance assistant for NIST 800-53, ISO 27001 and CIS Controls

Ask security compliance questions in plain English and get streamed answers that cite the exact controls they come from, such as [NIST AC-2]. When Sentinel finds nothing relevant, it says so instead of guessing.

  • NIST SP 800-53 rev5
  • ISO/IEC 27001:2022 Annex A
  • CIS Controls v8
  • RAG
  • FastAPI
  • pgvector
  • Ollama

Features

Cited answers

Every claim cites a control, for example [NIST AC-2], [ISO A.5.15] or [CIS CIS-5].

Refuses instead of guessing

Sentinel refuses when nothing relevant is found, and when an answer cites a control it never retrieved.

Maps audit findings

Paste an audit finding and get the related controls in all three frameworks, each with a one-sentence reason.

Hybrid search

Keyword and vector search merged with Reciprocal Rank Fusion, then reranked by a cross-encoder and de-duplicated with MMR.

Any LLM

A local model through Ollama by default, or Anthropic or OpenAI with an API key.

Measured quality

An evaluation suite scores recall, citation correctness, faithfulness and refusals, and fails the build if quality drops.

How it works

  1. Ingest. Loads the standards, embeds each control and stores it in Postgres with pgvector.
  2. Search. Runs keyword and vector search, merges them with Reciprocal Rank Fusion (k=60), reranks with a cross-encoder and removes near-duplicates with MMR (λ=0.7).
  3. Answer. An agent loop lets the LLM call search tools, then write an answer with citations.
  4. Check. Refuses if the answer has no citations, or cites a control that no search returned.

Quick start

Needs Docker with Compose v2 and, for the default local model, Ollama.

ollama pull llama3.1
git clone https://github.com/Clufy/Sentinel.git && cd Sentinel
cp .env.example .env
docker compose up --build
curl -X POST http://localhost:8000/ingest      # load NIST, ISO and CIS

Ask a question (the answer streams as Server-Sent Events):

curl -N -X POST http://localhost:8000/query \
  -H "Content-Type: application/json" \
  -d '{"question": "What controls govern account management?"}'

Map an audit finding:

curl -X POST http://localhost:8000/map \
  -H "Content-Type: application/json" \
  -d '{"finding_text": "Terminated employees retained active VPN accounts for weeks."}'

For this finding Sentinel returns NIST PS-4 (personnel termination), CIS 5.3 (disable dormant accounts) and ISO A.6.5 (responsibilities after leaving).

Evaluation results

40 questions (5 unanswerable) on a 30-control fixture corpus with a scripted LLM, pgvector 0.8.0 on CPU.

RetrievalRecall@8Citation correctnessFaithfulnessRefusal accuracy
Hybrid1.0000.7931.0000.975
Vector-only0.8570.1341.0000.825

FAQ

What is Sentinel?

Sentinel is an open-source, self-hosted API that answers governance, risk and compliance questions about NIST SP 800-53 rev5, ISO/IEC 27001:2022 Annex A and CIS Controls v8, citing the exact controls behind every answer.

Can Sentinel map an audit finding to controls?

Yes. POST an audit finding to /map and Sentinel returns the related NIST, ISO and CIS controls, each with a one-sentence rationale, or refuses when nothing is similar enough.

Which LLMs does Sentinel support?

It runs on a local model through Ollama by default (llama3.1), or on Anthropic or OpenAI models with an API key.

Does Sentinel give legal or audit advice?

No. Sentinel is a research aid that finds and summarizes control text. Always check the published standards before acting.

Is Sentinel free?

The source code is MIT licensed. The CIS-derived data stays under CC BY-NC-SA 4.0.