Ask a language model to turn "what was revenue last quarter by region?" into SQL and it will produce something that runs. That is the problem. The demo is convincing because the query executes and returns numbers, and the numbers are wrong in ways only the finance team would notice: the wrong revenue column, refunds not subtracted, a fiscal quarter treated as a calendar one, regions from an old mapping table.

This post is about the version of natural-language analytics we ship, which is less magical and more correct.

The demo that answered wrong

For a client with a well-run warehouse, we built a first text-to-SQL prototype in a few days. The model saw the table schemas and wrote queries. In a review session with the finance lead, five of the first ten answers were wrong. Not syntactically; semantically. The warehouse had three revenue-like columns for historical reasons, the model picked the most obviously named one, and it was the wrong one. The finance lead knew this. The model could not.

Schemas tell a model what columns exist. They do not tell it what the business means by revenue.

The semantic layer is where the meaning lives

The fix was not a smarter model. It was a semantic layer: a defined set of metrics and dimensions, each with a name, a description in business language, the exact SQL that computes it, and the grain it is valid at. "Revenue" became one definition, owned by finance, computed one way. "Region" became the current mapping, with a note about the change in the old data.

The model no longer writes SQL against raw tables. It maps a question onto metrics and dimensions from the semantic layer, and the layer generates the SQL. If a question asks for something the layer does not define, the system says so rather than improvising a join.

The path from a question to a number: the model selects from defined metrics and dimensions, the semantic layer writes the SQL, and undefined requests are refused rather than guessed.
The path from a question to a number: the model selects from defined metrics and dimensions, the semantic layer writes the SQL, and undefined requests are refused rather than guessed.

Guardrails on what can run

Even with the semantic layer, the generated query runs against production data, so it runs under controls. The database role used for these queries is read-only, restricted to the modelled tables, with row-level security applied from the asking user's identity so that a regional manager sees their region. Queries have a cost and time limit. The generated SQL is logged with the question and the user, and any query outside the semantic layer's patterns is blocked before execution.

This is also where prompt injection is handled. A question is untrusted input; a user, or a value inside the data, could try to steer the model toward a query it should not run. Because the model only selects definitions and the layer generates the SQL, there is no path from question text to arbitrary SQL.

Showing the working

The answer to a question is not just a number. It is the number, the metric definitions used, the filters applied, the time period as interpreted, and the SQL for anyone who wants it. When the finance lead sees "revenue, net of refunds, fiscal Q2 2026, by current region mapping," she can check the interpretation in a glance. Most of the remaining errors in our testing were interpretation errors, and showing the interpretation is what turned them from silent to visible.

What an answer carries besides the number, so that a wrong interpretation is visible to the person asking.
What an answer carries besides the number, so that a wrong interpretation is visible to the person asking.

The review loop

Before launch we collected the questions leadership actually asked in the previous quarter, from emails and meeting notes, and had the analytics team write the correct answer and definition for each. That became the evaluation set. Each change to the model, the prompts or the semantic layer runs it. After launch, every answer has a "this looks wrong" control that files the question, the interpretation and the answer to the analytics team, who either correct the semantic layer, add a definition, or explain the result. Those corrections are the most valuable thing the system produces, because they are the business's own definitions being written down, often for the first time.

What it does not do, deliberately

It does not answer questions the semantic layer cannot express. It does not write arbitrary joins. It does not run against tables outside the model. It does not let a user see rows their role cannot see. Each of these was a capability the prototype had and the product removed. The client's trust in it is a direct consequence of what it refuses.

Purposeful analytics

The value of natural-language analytics is not that leaders can type questions. It is that the organisation is forced to define its metrics precisely, once, and then everyone, human and model, computes them the same way. The AI feature is the reason the semantic layer finally gets built. The semantic layer is the reason the AI feature can be trusted.

  1. Define metrics and dimensions with business-owned SQL; let the layer write queries.
  2. Run under a read-only, row-secured role with cost limits.
  3. Show the interpretation with every answer.
  4. Build the evaluation set from real questions before launch.
  5. Route "this looks wrong" to the people who own the definitions.
  6. Refuse what the layer cannot express.