CPU-LLM: Evidence-Based AI Engine
Amin Parva, Independent Engineer
Answer questions from your documents and explicit rules, with source evidence and CPU-first execution.
Build source-cited document answers, deterministic checks and cross-domain research on CPU. A bounded document-and-rule engine. It does not establish that a source is true or make clinical decisions.
See the healthcare sample chat · Inspect the source · View the published releaseApplication guide · Published 1.13.1
What can you build with CPU LLM?
Start with the output your reader needs: a sourced answer, a document they can share, or a rule check they can inspect. These workflows require ingested records or explicit inputs.
Manual and policy lookup
Answer a named question from an ingested manual or policy and show the passage behind it. If the attribute is missing, return a nonanswer.
Record and label review
Look up a recorded allergy, ingredient, storage instruction or label strength. The fictional pharmacy examples distinguish reading a record from diagnosing, prescribing or choosing a safe dose.
Evidence and conflict review
Inspect source IDs, quotations and stored spans. Flag recognized attribute conflicts or ask for clarification; arbitrary contradictions in prose are outside the contract.
Reports and articles
Turn permitted records on a named topic into a brief, detailed article or sectioned report. Continue through additional evidence without padding to meet a word count.
Design documentation
Organize documented requirements, interfaces, architecture and validation into a design document. Mark sections missing when the sources do not support them.
Tables, lists and summaries
Present selected evidence as a table, bullets, a numbered list, Markdown or HTML. Use the quotation view when readers need the original wording.
Arithmetic and financial checks
Evaluate supported arithmetic and compare supplied figures with explicit rules. RefundProof, LedgerCovenant and Ledgerlock expose inputs and decision traces; they do not authenticate documents or authorize financial actions.
Historical and scoped retrieval
Inspect eligible records as of a specified time and restrict document selection to sources the caller may access. Continued reports recheck permissions.
Tone and application dialogue
Produce friendlier technical wording from supported predicates while retaining the evidence, or use reviewed replies for greetings and capability questions. Strict reports remain quotation-only.
See a recorded article · Compare a supported lookup with a refusal · Read the implementation guide
Published 1.13.1 · November 11, 2026
Research, verify and manage your evidence
CPU-LLM 1.13.1 is published on PyPI. The GitHub release wheel and source distribution match the PyPI SHA-256 digests; its release tag resolves to build source deba0b0.
Cross-domain research
Connect evidence across 2–7 selected knowledge bases. Keep source citations attached and use Constraint Check for structured numeric claims.
Hypotheses and constraints
Request labeled invention hypotheses and explicit constraint comparisons. The leptogenesis profile illustrates a bounded numerical scan under stated assumptions; numerical findings are conditional on the model and constraints, not a confirmed discovery.
Document operations
Use single-pass ingestion, inspect scoped chunk inventory and preview exact deletions. Deletion requires confirmation and team admin permission.
Strict mode retains supported quotations, recognized calculations and abstention. It disables invention, CrossDomain and Constraint Check routes. Ordinary reports retain their source citations.
Cited answers that preserve the evidence
Use evidence-based document question answering for cited weight and color lookups, exact location quotes and independently answered coordinated questions. Possession joins retain both premises. Missing evidence produces abstention, and grounded bullet summaries render selected sentences as separate items.
Conservative typo repair uses recognized question words and corpus-backed vocabulary while keeping source quotations intact. Financial covenant verification remains a use case for auditable deterministic checks against supplied figures and explicit rules.
Optional external fallback: clearly unverified
The external LLM fallback is off by default. An application adapter using an API key or OIDC access-token callback can handle eligible guard-approved, unsupported standalone requests. Strict Evidence Mode bypasses the fallback. Clarification, insufficient evidence, batches, session turns and specialist workflows retain their existing paths.
The adapter receives the original prompt and credential, without retrieved passages or session history. Successful output is labeled external_unverified, with empty evidence and a visible verification notice. Missing credentials or provider errors preserve the original response; the adapter enforces its own timeout.
Service operations
File-backed credentials reload at authentication time. Optional mutation audit logging is available through --audit-db and --audit-retention-days. Read the consumer guide for configuration and precise routing behavior.
Choose a route with an explicit prompt
These examples are copied from the release addendum. Ordinary requests evaluate six retrieval channels; explicit document, invention, CrossDomain and management requests can select a separate route before graph entry.
What did John eat?
Verify an event-frame object against the stored passage.
Search deeply for apple.
Request wider retrieval; Python callers can also set deep=True.
Calculate 17 times 24.
Select arithmetic. discover=True can request discovery after the ordinary reply.
Write a detailed report about Helios.
Produce a source-cited document.
Show relationship loops for Alpha.
Explore bounded relationship cycles.
Invention: Leptogenesis.
Request a labeled hypothesis; a matching built-in profile can run an illustrative numerical scan with stated assumptions.
Research Casas-Ibarra using KBs: physics, math.
Research across selected knowledge bases. Structured numeric claims can delegate to Constraint Check.
Constraint Check: Leptogenesis using KBs: physics, math, experiment.
Compare structured constraints directly.
List chunks for notes.pdf.
Inspect scoped chunk inventory.
Delete chunk notes-p0001.
Preview an exact chunk deletion; the deletion API requires confirmation and team admin permission.
Inspect routing.executed_nodes and routing.post_stages where available. Read trigger precedence and request flags · View the release prompt addendum.
Choose the answer contract
- Strict evidence
- Use recognized attribute questions for label or manual lookup. Inspect quoted chunks, spans and hashes; arithmetic follows supplied rules. Unrecognized requests return a nonanswer. Strict mode bypasses learned answer generation.
- Ordinary graph
- For self-hosted document search or a Python RAG application, combine retrieval, planning and selected math or finance producers. Ordinary mode can use optional self-hosted FLAN-T5.
- Conversation and tone
- Explicit prompt tone is separated from topic retrieval. Python style takes precedence; strict mode bypasses tone. Single-fact replies retain companion-only behavior.
From knowledge-base records to a verified article
- Source recordsEncrypted backups + PostgreSQL metadata
- Verified wordingTwo predicates joined; both checked
- Evidence viewOriginal quotes and record IDs retained
Recorded fictional demonstration · cpu-llm 1.11.0
Prompt: Write a detailed article about Helios in a friendly tone.
Helios stores encrypted backups and uses PostgreSQL for metadata. It stores backup files in object storage and verifies backup checksums. It retains backups for 30 days and runs backups every night.
The article also preserves a negative fact: “Helios doesn't support automatic regional failover.”
Inspect the source records and verification receipts
The first three sentences use records rerun-manual-p0002 through rerun-manual-p0007; the failover statement uses rerun-manual-p0011. None supplied friendly wording. All twelve claims passed the bounded inverse grammar in this run.
Read the article, report, JSON and receiptsReports, articles and design documents share section planning, coverage, evidence budgets and continuation. Choose Markdown, HTML, tables, bullets or numbered lists. Missing design sections stay explicit. Unsupported conditionals stay quoted. Strict reports use quotations only.
Ordinary document reply.text can publish new verified sentences. Inspect original quotations and proofs alongside rendered_response.canonical_text. Broader aliases, joins, typed object-category queries, arbitrary prose conflicts and novel designs remain unsupported.
Recorded fictional demo output
Two requests from the same pharmacy evaluation illustrate the boundary. This is recorded output, not a live patient service.
What is the strength of Fictional Luma?
Source quotations (not clinically validated):
[demo-label-luma-p0001] The strength of Fictional Luma is 0.75 mg.
[demo-secondary-luma-p0001] Fictional Luma's strength is 750 mcg.
Status: evidence_quote
Which antibiotic should Case Birch receive?
Unsupported strict request; use explicit named attribute questions.
Status: unsupported
Read all 12 healthcare exchanges · Inspect recorded tone responses
A financial rule with an inspectable result
Divide 9,000,001 by 2,000,000. Under the supplied six-place half-up policy, the stored result exceeds the inclusive ceiling by one millionth:
LedgerCovenant: BREACH. The computed value is 4.500001x and the test is lte 4.500000x.
This checks arithmetic and precedence; it does not authenticate a certificate, establish legal default or authorize payment.
Install and reproduce the examples
python -m pip install "cpu-llm==1.13.1"Use Python 3.11 or newer. Configure PostgreSQL with pgvector and cache the external pretrained encoder first. The retrieval projection and demo data ship with the package; optional NLP and decoder assets are separate. Choose new output directories.
python -m pip check
cpu-llm-demo healthcare --out healthcare-demo-new-run
cpu-llm-demo conversation --out conversation-demo-new-runFrom ingestion to an API response
In Python, ingest a source with ingest() and query with ask(); the HTTP service exposes POST /v1/ask. The 27 graph nodes include retrieval, source spans, planning and finance. Six retrieval signals—dense, relational, disentangled, hyperbolic, identity and causal—are distinct from output tone. Only dense and identity trained heads are bundled.
Read the full whitepaper · Download the 1.13.1 published whitepaper
Browse the developer guide by section
- 1 Executive overview
- 2 Suitable workflows and boundaries
- 3 Install and run the packaged examples
- 4 Ingestion and the first Python integration
- 5 Execution paths and the six retrieval channels
- 6 Node reference and request controls
- 7 Strict evidence mode
- 8 Consume results as contracts
- 9 Evidence-backed documents and verified sentence generation
- 10 Financial component example
- 11 HTTP service and authorization
- 12 Python API reference
- 13 Historical evidence and reproducibility
- 14 Deployment checklist and next steps
- References and implementation authority
Document verification: two execution contracts
A generative RAG application can add citations and calculators. Without an enforced verification step, a generated answer needs separate support checks. CPU-LLM strict mode returns quotations, recognized calculations, conflicts or a nonanswer. A stored hash shows consistency, not source truth. This comparison concerns behavior, not measured accuracy, speed or cost.
Scoped synthetic benchmark results
On a recorded clean checkout of 7adf319, the revised synthetic request suite passed 460/460 scored rows across dense and resonance retrieval: 278/278 expected answers and 86/86 expected abstentions. The configured scorer flagged zero false assertions.
Initial harbor fixture
20/24 exposed a color-vocabulary gap for ochre in both retrieval configurations.
Vocabulary regression replay
24/24 on the same fixture after the vocabulary fix. This measures regression coverage, not a new holdout.
Expanded typo regression replay
32/32 with four additional typo prompts across both retrieval configurations.
Three fixtures changed from the prior request suite: overview specificity, equivalent proof and grounded partial drafts. These are scoped synthetic results, not universal accuracy, certification, clinical validation or comparisons to hosted models. Inspect displayed replies for duplicate text and list structure as well as scored fields.
Read the public benchmark notes and recorded results · Read whitepaper evaluation limits.
Developer questions
Which version should I install?
Install the published CPU-LLM 1.13.1 package from PyPI. The matching GitHub artifacts are attached to the pypi-v1.13.1 release, built from deba0b0.
Is cpu-llm a general-purpose LLM?
It is an Evidence-Based AI Engine for bounded document question answering and rule checks. A pretrained encoder and projection support retrieval.
Can strict mode guarantee accuracy?
No. It quotes recognized source facts, performs bounded checks or declines. A citation identifies stored evidence; it does not authenticate the source.
Can I use it for pharmacy questions?
Evaluate the fictional label and record demos. The package is not validated for diagnosis or prescribing.
Do I need a GPU?
The documented CPU-first strict examples require no GPU. They do require PostgreSQL, pgvector and encoder assets.
Can it run offline?
Yes, after provisioning and caching the required assets. Air-gapped deployment requires prepared Python, database and encoder assets plus operator-enforced network isolation.
Can it give friendlier answers?
Yes. Ask for a friendly article in the prompt. Supported technical predicates can become new, independently verified sentences. Complex records stay quoted; strict reports are quotation-only.
Must I train a model?
No retraining is needed for the packaged demos. You still need to ingest the relevant documents.
Related research: VectorPrism · Python libraries · LedgerCovenant.
Evaluate an enterprise pilot
The Apache-2.0 package is free for individuals and small teams. Organizations of any size can self-manage it. Paid enterprise pilots, integration and support are negotiated with Amin Parva through Insight IT Solutions. Bundled WikiText-2 data has separate CC BY-SA 3.0/GFDL terms.
Discuss a commercial pilot