Better3
let's talk

data · models · 2026

ai-ready data makes a lab's own corpus something a model can learn from, and be checked against

Extraction, structure, embeddings and a frozen validation split: the groundwork before any model, done so that whatever is trained or asked can be measured.

  • python
  • postgres
  • pgvector
  • specter2
  • hmmer
  • claude
read the case study ↓

What it is

Most groups have the data a model would need: papers, tables, sequences, measurements, years of them. What they do not have is the data in a shape a model can use, or a way to tell whether the model is right. This is the work before the model: extract, structure, embed, split, and only then train or ask.

the sequence
fig. 01 — the split is frozen before anything is trained, so the hold-out number means something

Three places it has been done

Papers. 528 papers in three research vaults, each one summarised with typed frontmatter, indexed by subject and by argument, and asked through a read-only agent.

A feed. SciFlow ingests new microbiology papers and preprints from Europe PMC and bioRxiv on seven schedules, embeds them with SPECTER2 into pgvector under an HNSW index, and serves hybrid keyword and semantic search.

Genomes. The methane-cycle pipeline trains a per-target threshold on a reference panel with a frozen train / hold-out split, and reports the hold-out score with its interval before the headline number.

across the three
fig. 02 — numbers from the current state of each project
sciflow · embed and index
fig. 03 — SPECTER2 is trained on citation relationships, so similar means cited together, not worded alike
a retrieval agent that cannot write
fig. 04 — the tools list is the safety boundary; it is short on purpose
validation · methane cycle
fig. 05 — the limits come first in the report; the headline score comes after

Design decisions

  • Structure before models. Typed frontmatter and one folder per record make every later step a script, not a person.
  • Split by group, freeze it, write it down. A hold-out that shares a genus or an author with the training set is not a hold-out.
  • Read-only where a model answers. An agent that can answer questions from your data gets no tool that can change it.

What is not done

Fine-tuning large language models on private corpora is scoped per project and priced after the extraction step, when the size of the usable data is known. Nothing leaves your machines unless you decide it does.