AskStein

Documentation

How a question becomes an answer.

The language model's job is deliberately narrow: read the retrieved evidence and write an answer grounded in it. It is designed not to introduce fabricated citations — citation accuracy is enforced by a separate post-processing step, not by the model's judgment — and when retrieval doesn't surface adequate evidence, the system abstains. Citation resolution does not by itself guarantee that every generated interpretation is fully supported by the underlying passage; claim-level grounding remains subject to ongoing evaluation.

The pipeline

  1. 01

    Ingestion & curation

    Papers are collected from approved open-access sources according to the project's ingestion and licensing criteria, deduplicated, and split into passages.
  2. 02

    Embedding

    Each passage is converted into a vector using OpenAI's text-embedding-3-large.
  3. 03

    Indexing

    Those vectors are stored in a FAISS index built for fast similarity search.
  4. 04

    Retrieval

    An incoming question is embedded and matched against the index to pull the most relevant passages.
  5. 05

    Generation

    A fine-tuned language model composes an answer using only the retrieved evidence.
  6. 06

    Citation resolution

    Generated citations are resolved against canonical records in the corpus during post-processing. Citations without a valid corpus match are rejected rather than shown.
  7. 07

    Abstention

    If the evidence isn't strong enough, the system says so instead of manufacturing an answer.

The system today

Model
Qwen2.5-32B-Instruct with a fine-tuned LoRA adapter
Embeddings
OpenAI text-embedding-3-large
Retrieval
FAISS over an orthopedic corpus built from 17,700 acquired and classified papers, with 16,400 retained at a relevance score of 0.7 or higher — see the corpus →
Citations
Generated citations are resolved against canonical records in the corpus; citations without a valid corpus match are rejected during post-processing. This verifies that a cited source exists and is correctly attributed — it does not by itself guarantee that every generated interpretation of that source is fully supported
Interface
A hosted web frontend

This is the current orthopedic proof-of-concept. The architecture is designed to scale to a broader medical evidence engine while preserving source traceability and honest abstention.

Corpus provenance

Source channels
Not yet documented — see the corpus page for what's currently published about corpus composition
Licensing / inclusion criteria
Not yet documented
Corpus coverage dates
Not yet documented
Last ingestion
Not yet documented
Build version
Not yet documented
Docs updated
October 14, 2026

Evaluation and limitations

The current figures come from an internal 200-question benchmark using the same prompts and passages across systems. They are development results intended to measure progress; independent clinical validation and publication remain future work.

97.5%
Grounding
99.5%
Faithfulness
96.0%
Relevance
94.5%
Abstention
Evaluation set
200 questions · internal benchmark
Citation integrity
No fabricated citations observed across the 200 evaluated answers
Claim checks
Claims are checked against the cited passage before output
External validation
Planned; internal results should not be interpreted as independent clinical validation

Where it's headed

  1. 01

    Acquire at scale

    Expand ingestion toward all PubMed abstracts and openly licensed full text.
  2. 02

    Process & embed

    Normalize sources, preserve provenance, and build specialty-aware evidence representations.
  3. 03

    Expand the retrieval engine

    Extend the orthopedic proof point into a multi-specialty evidence layer.
  4. 04

    Validate & ship

    Pursue external validation, harden the product, and make the evidence engine available through an enterprise API.
  5. 05

    Connect the care journey

    Explore standards-based clinical workflow integration for intake, point-of-care evidence, follow-up, and revenue integrity — with clinicians retaining decision authority.

Abstention is not an error — it's a feature. When AskStein says it doesn't know, it's reporting the honest boundary of what its corpus contains, not failing to answer.