Building with AI
The hands-on engineering: RAG, agents, evaluation, running models and shipping AI to production.
26 articles
Building RAG Over Your ERP Data: The Architecture That Actually Survives Production
Most ERP chatbots fail the moment someone asks for a total. Here is the two-plane architecture, the hybrid retrieval SQL, the reranking step and the permission model that separate a demo from a system finance will actually trust.
Self-Hosting Open-Weight Models: The VRAM and Cost Math Nobody Shows You
Before you buy a GPU, do the arithmetic. Weights, KV cache, quantisation, batching and bandwidth decide whether self-hosting beats an API - and for most African teams the honest answer is a hybrid, not a rack.
Agentic AI: The Promise and the Peril of AI That Acts
AI is shifting from answering to acting. Agentic systems pursue goals and take real actions - powerful, oversold, and risky. The winners will deploy them narrowly, wrap them in guardrails, and grant autonomy sparingly.
Prompt Injection Is the New SQL Injection - And Most African AI Projects Are Wide Open
OWASP ranks prompt injection as the number one risk in LLM applications, and there is still no complete fix. Here is how direct and indirect injection actually work, and the defence-in-depth pattern that limits the blast radius.
AI for Garages: Bookings, Quotes and Follow-Ups That Never Slip
Garages and service businesses lose money in the gaps - the quote never sent, the service reminder never made, the customer who called while everyone was under a car. Here is what AI can automate, what stays human, and how to begin.
Stop Regex-Parsing LLM Output: Structured Outputs, JSON Schema and Constrained Decoding
The retry-until-it-parses loop was never engineering. Constrained decoding makes schema conformance a property of the sampler, not a hope. Here is how it works, how to use it on hosted and local models, and the traps that remain.
How to Evaluate an LLM Feature Before It Touches a Customer
Vibes are not a release gate. Build the eval set from real traffic, split deterministic checks from LLM-as-judge scoring, calibrate the judge against humans, and wire the whole thing into CI so a prompt change cannot silently regress.
Agentic Coding as an Engineering Discipline: My Daily Claude Code Workflow
An agentic coding tool is not autocomplete. It is a process that runs unattended, so the engineering is in the verification loop, the context budget and the repo scaffolding. Here is the setup and the daily rhythm that actually works.
Why Tool Calls That Work on Every Other Model Return a 400 on the Newest Open-Weight Model
Your agent's tool calls 400 against gpt-oss-120b but work everywhere else. It is a chat-template and Harmony encoding mismatch, not a framework bug. Here is the adapter fix.
Human-in-the-Loop Edits Get Ignored: The Agent Runs the Original Tool Call
A human edits an agent's proposed tool call and approves it, then the agent re-proposes the original. Why HITL middleware is not an authorization boundary, and where the real guard belongs.
Every Model Upgrade Quietly Renegotiates Your Prompts: Catch It Before Your Users Do
A prompt that worked for months breaks the day the model is upgraded: format drift, weaker instruction-following, new refusals. Here is the regression-testing discipline that catches it first.
Your RAG Metadata Filter Isn't Filtering: Silently Serving the Wrong Tenant's Data
A vector-store metadata filter that is silently ignored returns cross-tenant results with no error. Why the filter-translation layer fails, and the retrieval test that catches it before an audit does.
Why Your RAG Demo Falls Apart on Real Production Data
Top-k similarity search returns confident, wrong chunks as your corpus grows. Here is the mechanism, the mitigations with honest limits, and an eval harness.
Your structured-output parser broke the day you switched to a reasoning model
Reasoning models return analysis, commentary and final as separate channels. A parser that assumes one message equals one answer chokes on the extra channels. Here is how to unwrap it.
Your PII Filter Leaks in Streaming Mode: The Guardrail Gap Nobody Tests
A post-hoc moderation or PII filter passes every test, then leaks the moment you turn on streaming. Why whole-response guardrails cannot protect a token stream, and the buffered fix.
Why Your Token Counter Thinks a 2MB Audio Clip Is 3 Million Tokens
Character-based token estimators count the base64 payload of images and audio, inflating counts 10 to 100 times and triggering false context overflow. Here is how to estimate multimodal tokens properly.
Why Two Jobs Writing to the Same Vector Store Can Permanently Corrupt It
Embedded vector stores like Chroma PersistentClient have no multi-writer safety, so concurrent upserts can permanently corrupt the index. Here is the architecture that prevents it.
Your Agent Keeps Calling Tools That Don't Exist: Fix the Toolset, Not the Retry
The error says the tool is not valid, so you add a retry that tells the model to try again. It hallucinates a different wrong name. Here is the real fix.
Your Embedded Vector DB Will Run Out of RAM: Size It Before Production Does
An in-process vector index scales with corpus size, not traffic. Here is the sizing math to see the RAM wall before an OOM kill finds it in production.
Your LLM Cost Dashboard Says $0 and It's Lying: get_openai_callback
get_openai_callback returns 0 tokens and $0 after wrapping in AgentExecutor or upgrading LangChain, with no error. Here is why it silently detaches and how to count cost reliably.
The JSON Schema That Works on GPT and Silently Breaks on Every Other Model
Structured output is not one feature. A schema that returns clean JSON on OpenAI can degrade silently on Gemini. Here is the portable subset and a CI check that catches it before you ship.
Why RetryError hides the real reason your LLM call failed
A tenacity RetryError buries whether your LLM call died from a rate limit, bad auth or a timeout. And raw httpcore errors leak through elsewhere. Here is a typed boundary that fixes both.
Why 'maximum context length exceeded' Keeps Happening After You Switched Models
A bigger context window hides the bug, it does not fix it. Here is the token-budgeting algorithm that stops 'maximum context length exceeded' on every model.
4-Bit Quantization Doesn't Mean 4x Less VRAM: What Eats the Rest
A 4-bit 7B model 'fits' a 12GB GPU on paper then OOMs. Quantization only shrinks weights; KV cache, activations and CUDA overhead take the rest. Here is the real budget.
One Bad Request Killed Your Whole vLLM Server: AsyncEngineDeadError Explained
AsyncEngineDeadError means one request crashed vLLM's shared background loop and took every other request down with it. Here is the architecture behind it and the mitigations that actually contain it.
Your Multi-Model Gateway Is Not OpenAI-Compatible for Every Model Behind It
OpenRouter and LiteLLM guarantee the OpenAI request shape, not model behaviour. Here is why swapping models silently breaks tools and JSON, plus a smoke test.