The AI we chose not to use
Building an EU AI Act compliance tool taught us that the most senior AI decision is often where not to put a model.
By Logical
We build with LLMs every week. So when we set out to build a compliance platform for the EU AI Act — the kind of product where "AI" is right there in the brief — the interesting decision wasn't which model to use. It was deciding, deliberately, to keep generative AI out of the part everyone expected it in.
The temptation
The pitch writes itself: feed a company's answers into a model, let it reason about the regulation, generate a tailored compliance report. It demos beautifully. It's also exactly the wrong tool for the job.
A compliance assessment has to be auditable and repeatable. The same inputs must produce the same classification, every time, and you must be able to point at why. An LLM gives you neither for free. Two runs can diverge. Ask it to cite the article and it may confidently invent one. When a regulator — or the client's lawyer — asks how a system landed in the "high-risk" tier, "the model felt strongly about it" is not an answer.
The risk classification is a decision tree, not a vibe. So we wrote a decision tree.
Where the model earns its place
So the assessment is deterministic: a rules engine maps each answer to the Act's four risk tiers, and the report is generated from those rules. Same input, same output, with a traceable path from question to obligation. No hallucinated articles.
That doesn't mean no AI. It means AI where it actually helps:
- Turning legalese into plain language in the training material, with a human reviewing every line before it ships.
- Drafting lesson content and quiz questions that an expert then edits — the model as a faster first draft, never the final word.
- Helping users phrase what their systems do, so the deterministic engine gets cleaner inputs.
The general lesson
The most senior AI engineering decision is usually subtractive. Knowing when a model is the wrong abstraction — too unpredictable for a path that has to be defensible, too expensive for a job a lookup table does perfectly — is worth more than knowing the latest model's context window.
"We use AI" is easy. Knowing exactly where to draw the line, and being able to explain it to a regulator, is the part clients are actually paying for.