Why text-to-SQL fails on GTM data, and how owning the schema fixes it
Generated SQL rarely errors out. It returns a plausible number from the wrong join. Kiwi owns the schema its queries read, so it can check them before they run.
Melanie Chen ·
Ask a model "how much pipeline did we add last month?" and you will get SQL that runs and a number that looks right. That is the problem. Text-to-SQL almost never fails loudly. It fails with a confident answer that is off by a join, a definition, or a week of data, and checking it takes exactly the SQL expertise the tool was meant to replace.
How it fails
- Words the schema never defines. "Active customer", "qualified pipeline", "met with them". The database has columns, not your definitions, so the model guesses.
- The wrong join that still looks right. Join deals to contacts the wrong way and the total doubles quietly. Nothing errors; the number is just wrong.
- Tables and columns that do not exist. Hand a model a large schema and it invents a column. Hand it too little and it guesses which table holds the answer.
- A different answer every time. The same question produces different SQL on different days, so the number moves and nobody sees why.
- SQL nobody checked before it ran. A generated query can write, run for minutes, or read rows the person asking should never see.
GTM data makes every one of these worse. The account lives in five tools that share no ids, the same person shows up as a CRM contact, a meeting attendee, and an email sender, and the CRM everyone queries is the system updated last. Read live from those APIs and even identical SQL disagrees with itself a minute later.
The fix: own the schema the SQL reads
Most text-to-SQL points a model at somebody else's warehouse and hopes. Kiwi is built the other way around. It syncs your CRM, calendar, email, call notes, and documents into a store it owns, and defines the five views every query reads: accounts, people, meeting notes, messages, and documents. Each column has a description the model reads, and the joins are made once, inside the views: every person, meeting, message, and document already carries the account it belongs to.
Because Kiwi owns the schema, it does not have to trust generated SQL. It checks it, and it can promise what the SQL does:
- It reads only the five views. A query that names any other table, or a column the views do not have, is refused before it runs.
- It cannot change anything. Exactly one read-only statement, run in a read-only transaction with a time limit.
- It sees only what you may see. Every query runs as a database role that can read nothing but those views, filtered row by row to what the person asking may read.
- The same question gets the same rows. Every query reads the store as of one moment, so the same SQL at the same moment returns the same rows.
- Every row traces to its source. Each row carries the records it came from, down to the original CRM record, email, or meeting.
- Saved SQL does not drift. An agent step pins one exact version of a saved query. Editing the query makes a new version instead of changing what the agent reads.
What no schema can promise
SQL can be valid, safe, and repeatable and still answer a slightly different question than the one you meant. So Kiwi never hides the query: every answer shows the SQL and the rows behind it. Once a query is right, you save it, and agents reuse that exact version instead of writing a new one each time.
Questions
Why does text-to-SQL give wrong answers?
Mostly because the model does not know what your terms mean or how your tables join, so it writes SQL that runs but answers a slightly different question.
Can Kiwi guarantee generated SQL is correct?
Kiwi guarantees the SQL reads only its own views, cannot change data, sees only what you may see, and returns the same rows at the same moment. Whether it answers the question you meant, you check from the SQL and rows it shows.
Which data can Kiwi's SQL read?
Five views Kiwi defines: accounts, people, meeting notes, messages, and documents, built from the tools you connect.
Further reading
- The six failures of text-to-SQL (and how to fix them with agents), Karl Weinmeister, Google Cloud