FIELD NOTES
Field notes
Short, precise notes on how we build: retrieval, agents, and the data foundation underneath.
- CASE STUDY · CLASSIFICATION
Case study: product categorization
A drop-in replacement for a word-matching classifier: exact match, then a multilingual encoder, then kNN corroboration, then an LLM. Each layer catches what the previous one missed.
- HYBRID SEARCH
Where vector search misses
Embeddings find meaning and lose precision. BM25 finds exact strings and loses meaning. Production retrieval runs both.
- EMBEDDINGS · VECTOR DB
What a vector database does
An embedding turns text into coordinates. A vector database finds the nearest neighbors of a point, fast, at millions of points.
- GRAPH RAG
Graph RAG, when relationships are the answer
Vector search finds similar text. Some questions are about connections: which supplier, which contract, what depends on what. That is a graph problem.
- LLM · TRANSFORMERS
What a language model is
A transformer turns text into numbers that carry meaning. Encoders read, decoders write, and which one you need depends on the job.
- MCP · INTEGRATION
MCP, or how models reach your systems
MCP is one protocol between models and tools. Build one server per system and every model that speaks it can use them all, under your permissions.
- AGENTS
Agents with real tool use
An agent is a loop: the model plans, calls a tool, reads the result, and decides again, inside guardrails.
- PIPELINES · PLATFORM
Ingestion and the data foundation
Before AI can read your data, something has to collect, clean, and shape it. That is the foundation.
- DATA QUALITY
Data quality in production
Quality is enforced by tests that run every time data moves, or it decays. There is no third state.
- CATALOG · LINEAGE
Data catalog and lineage
A catalog tells people what data exists. Lineage tells them where it came from and what breaks if it changes.
- MIGRATION
Migrating off legacy systems
You move off a legacy system without a big-bang cutover, by running the new path beside the old one.