AI Engineering
30 articles in this topic.
Metering, model routing and the regression suite that stops your agent rotting
Two questions decide whether an AI feature survives its first year: does each request make money, and will a prompt edit break something silently? An append-only credit ledger, two-tier routing, and a suite that asserts question-to-metric mappings are the answers.
Shipping a business agent on WhatsApp: templates, webhooks and the 24-hour window
The in-app chat panel was good and nobody opened it. Putting the agent on WhatsApp means the Cloud API's template rules, idempotent webhooks, per-tenant scheduled jobs, alert hysteresis, and testing that Sheng routes to the right metric.
The agent drafts, the human commits: designing the write path for a business AI
A wrong read costs minutes. A wrong write costs a reconciliation. So the commit lives behind an HTTP endpoint the agent cannot call - and everything it can do is scoped to the requesting user, written to an audit row in the same transaction, and given a tested reversal.
Building RAG Over Your ERP Data: The Architecture That Actually Survives Production
Most ERP chatbots fail the moment someone asks for a total. Here is the two-plane architecture, the hybrid retrieval SQL, the reranking step and the permission model that separate a demo from a system finance will actually trust.
Hosted brain, dumb connector: splitting an AI agent so the data never leaves the client
Ship your agent to the client and you hand over your product. Pull their data to your platform and you become the custodian of their business records. The third option is a split: a hosted brain that decides what to fetch, and an on-prem connector with no intelligence in it at all.
Self-Hosting Open-Weight Models: The VRAM and Cost Math Nobody Shows You
Before you buy a GPU, do the arithmetic. Weights, KV cache, quantisation, batching and bandwidth decide whether self-hosting beats an API - and for most African teams the honest answer is a hybrid, not a rack.
Why we did not point an LLM at ERPNext: building a semantic layer for a business agent
Text-to-SQL against a real accounting schema produces confidently wrong numbers, because ERPNext stores revenue in the general ledger, stock valuation in the stock ledger, and gross margin nowhere at all. Here is the metric catalog and catalog-first hybrid engine we built instead.
Prompt Injection Is the New SQL Injection - And Most African AI Projects Are Wide Open
OWASP ranks prompt injection as the number one risk in LLM applications, and there is still no complete fix. Here is how direct and indirect injection actually work, and the defence-in-depth pattern that limits the blast radius.
Rai: what I am building, and which decisions were load-bearing
An AI business analyst for hardware and electronics shops, embedded in ERPNext. The v1 scope is locked, so here is what is in it, which four decisions the whole thing rests on, and the two things I expect to get wrong.
Stop Regex-Parsing LLM Output: Structured Outputs, JSON Schema and Constrained Decoding
The retry-until-it-parses loop was never engineering. Constrained decoding makes schema conformance a property of the sampler, not a hope. Here is how it works, how to use it on hosted and local models, and the traps that remain.
How to Evaluate an LLM Feature Before It Touches a Customer
Vibes are not a release gate. Build the eval set from real traffic, split deterministic checks from LLM-as-judge scoring, calibrate the judge against humans, and wire the whole thing into CI so a prompt change cannot silently regress.
Agentic Coding as an Engineering Discipline: My Daily Claude Code Workflow
An agentic coding tool is not autocomplete. It is a process that runs unattended, so the engineering is in the verification loop, the context budget and the repo scaffolding. Here is the setup and the daily rhythm that actually works.
Why Tool Calls That Work on Every Other Model Return a 400 on the Newest Open-Weight Model
Your agent's tool calls 400 against gpt-oss-120b but work everywhere else. It is a chat-template and Harmony encoding mismatch, not a framework bug. Here is the adapter fix.
Human-in-the-Loop Edits Get Ignored: The Agent Runs the Original Tool Call
A human edits an agent's proposed tool call and approves it, then the agent re-proposes the original. Why HITL middleware is not an authorization boundary, and where the real guard belongs.
Every Model Upgrade Quietly Renegotiates Your Prompts: Catch It Before Your Users Do
A prompt that worked for months breaks the day the model is upgraded: format drift, weaker instruction-following, new refusals. Here is the regression-testing discipline that catches it first.
Your RAG Metadata Filter Isn't Filtering: Silently Serving the Wrong Tenant's Data
A vector-store metadata filter that is silently ignored returns cross-tenant results with no error. Why the filter-translation layer fails, and the retrieval test that catches it before an audit does.
Why Your RAG Demo Falls Apart on Real Production Data
Top-k similarity search returns confident, wrong chunks as your corpus grows. Here is the mechanism, the mitigations with honest limits, and an eval harness.
Your structured-output parser broke the day you switched to a reasoning model
Reasoning models return analysis, commentary and final as separate channels. A parser that assumes one message equals one answer chokes on the extra channels. Here is how to unwrap it.
Your PII Filter Leaks in Streaming Mode: The Guardrail Gap Nobody Tests
A post-hoc moderation or PII filter passes every test, then leaks the moment you turn on streaming. Why whole-response guardrails cannot protect a token stream, and the buffered fix.
Why Your Token Counter Thinks a 2MB Audio Clip Is 3 Million Tokens
Character-based token estimators count the base64 payload of images and audio, inflating counts 10 to 100 times and triggering false context overflow. Here is how to estimate multimodal tokens properly.
Why Two Jobs Writing to the Same Vector Store Can Permanently Corrupt It
Embedded vector stores like Chroma PersistentClient have no multi-writer safety, so concurrent upserts can permanently corrupt the index. Here is the architecture that prevents it.
Your Agent Keeps Calling Tools That Don't Exist: Fix the Toolset, Not the Retry
The error says the tool is not valid, so you add a retry that tells the model to try again. It hallucinates a different wrong name. Here is the real fix.
Your Embedded Vector DB Will Run Out of RAM: Size It Before Production Does
An in-process vector index scales with corpus size, not traffic. Here is the sizing math to see the RAM wall before an OOM kill finds it in production.
Your LLM Cost Dashboard Says $0 and It's Lying: get_openai_callback
get_openai_callback returns 0 tokens and $0 after wrapping in AgentExecutor or upgrading LangChain, with no error. Here is why it silently detaches and how to count cost reliably.
The JSON Schema That Works on GPT and Silently Breaks on Every Other Model
Structured output is not one feature. A schema that returns clean JSON on OpenAI can degrade silently on Gemini. Here is the portable subset and a CI check that catches it before you ship.
Why RetryError hides the real reason your LLM call failed
A tenacity RetryError buries whether your LLM call died from a rate limit, bad auth or a timeout. And raw httpcore errors leak through elsewhere. Here is a typed boundary that fixes both.
Why 'maximum context length exceeded' Keeps Happening After You Switched Models
A bigger context window hides the bug, it does not fix it. Here is the token-budgeting algorithm that stops 'maximum context length exceeded' on every model.
4-Bit Quantization Doesn't Mean 4x Less VRAM: What Eats the Rest
A 4-bit 7B model 'fits' a 12GB GPU on paper then OOMs. Quantization only shrinks weights; KV cache, activations and CUDA overhead take the rest. Here is the real budget.
One Bad Request Killed Your Whole vLLM Server: AsyncEngineDeadError Explained
AsyncEngineDeadError means one request crashed vLLM's shared background loop and took every other request down with it. Here is the architecture behind it and the mitigations that actually contain it.
Your Multi-Model Gateway Is Not OpenAI-Compatible for Every Model Behind It
OpenRouter and LiteLLM guarantee the OpenAI request shape, not model behaviour. Here is why swapping models silently breaks tools and JSON, plus a smoke test.