Teradata SQL, where the DDL is the physical design.
A 6,336-line grammar across four files that reads what nothing else reads: MULTISET, FALLBACK, journaling, DATABLOCKSIZE, CHECKSUM, MERGEBLOCKRATIO, MAP, PRIMARY INDEX, COMPRESS value lists, LOCKING ROW FOR ACCESS and COLLECT STATISTICS USING SAMPLE.
Teradata is not a language people write applications in. It is the language a warehouse is configured in — and every one of those clauses is a migration decision somebody has to make on purpose rather than discover in production.
In every other dialect on our list, CREATE TABLE declares columns. In Teradata it also declares duplicate-row semantics, the fallback copy policy, whether the table is journaled before and after, the data block size in bytes, the checksum level, the merge block ratio, the hashing map, the primary index and the compression values for individual columns. Then it declares the columns.
Those clauses arrive in a comma-separated list before the column list, in almost any order, with values that are themselves keyword phrases: DEFAULT MERGEBLOCKRATIO, CHECKSUM = DEFAULT, MAP = TD_MAP1. It is a large, ambiguous keyword surface, and a generic SQL grammar has no rules for any of it — not partial rules, none.
Which is the whole commercial point. When a Teradata estate moves to Snowflake, Databricks, BigQuery or Synapse, those clauses are exactly the part somebody has to decide about. A PRIMARY INDEX becomes a clustering key or nothing at all. A COMPRESS value list becomes nothing. FALLBACK becomes nothing.
Parsing them means you can produce a report of what is being dropped, table by table, before the migration starts. Skipping them means finding out afterwards, from a query that used to take four seconds.
The same is true of the operational statements around the DDL. COLLECT STATISTICS USING SAMPLE 10 PERCENT and LOCKING ROW FOR ACCESS are not decoration — they are why the nightly batch finishes, and they have grammar rules of their own here.
Input — Teradata table definition
create_table_statement.sql
CREATE MULTISET TABLE prodeiw.SB_TRANS_EVENT ,FALLBACK ,NO BEFORE JOURNAL, NO AFTER JOURNAL,DATABLOCKSIZE = 524288BYTES, CHECKSUM = DEFAULT,DEFAULT MERGEBLOCKRATIO, MAP = TD_MAP1
(
SB_Event_Id INTEGER NOT NULL,
SB_Event_Processing_Dt DATE FORMAT'YYYY-MM-DD',
SB_Activity_Cd INTEGER COMPRESS (10251 ,10000 ,-10 ),
SB_Host_Val CHAR(10) CHARACTER SET LATINNOT CASESPECIFIC
)
PRIMARY INDEX ( SB_Event_Id );
Seven marked clauses, none of which is a column. This body is taken from the shipped example corpus rather than written for this page — the shape is one the engine is tested against.
These are the real rule names in the shipped grammar. The AST classes are generated from that grammar with Starlasu annotations rather than written by hand, so the model and the rules cannot drift apart; every node carries a position and difficult input returns an issue list, not an exception.
BTEQ: what actually happens to the dot-commands
This is the first question every Teradata prospect asks, and it deserves an exact answer rather than a reassuring one.
Read it literally. A line that begins at column 0 with . or = is consumed to the end of the line and routed to a hidden channel. So a real BTEQ script does not break the parser and the SQL inside it parses — but the dot-commands themselves are not parsed. There is no node for .IF ERRORCODE, no branch structure, no .EXPORT target. The column-0 predicate is what keeps db.table and the = operator from being swallowed with them.
FastLoad, MultiLoad, TPump and TPT are a different matter: they are separate utility languages with their own control syntax, and they are out of scope — no token, no rule, no fixture. If your corpus is mostly MultiLoad scripts, this is not the product, and we would rather tell you on the first call.
What it handles
Four constructs that decide whether a parser survives a real warehouse export. None of them is exotic Teradata; all four are in the first thousand lines of any DDL dump.
COMPRESS value lists
A column definition can carry a parenthesized list of dozens of literal values — negative numbers, space-padded CHAR constants — and it is syntactically indistinguishable from any other parenthesized list without knowing where you are. It is also the clause that has no target-platform equivalent at all.
cd INTEGER COMPRESS (10251 ,10000 ,-10 ),
val CHAR(10) COMPRESS
('0001 ','SSVS01 ')
QUALIFY, a fourth filtering clause
Alongside WHERE and HAVING, evaluated after the window functions, which means the clause-ordering grammar is not the ANSI one. Teradata also spells row limits TOP n with no parentheses — the opposite of Transact-SQL, and a genuine source of silent mis-parses.
SELECT TOP10 acct, amt,
RANK() OVER (ORDER BY amt DESC) r
FROM ledger
QUALIFY r <= 3;
LOCKING as a statement prefix
A modifier that comes before the statement it modifies, at database, table, view or row scope, with seven lock types including READ OVERRIDE and LOAD COMMITTED. LOCKING ROW FOR ACCESS SELECT … is a Teradata idiom with no equivalent anywhere else, and it is all over production reporting code.
LOCKING ROW FOR ACCESSSELECT acct, bal FROM prodeiw.balances;
REPLACE as a DDL verb, and period types
Where every other dialect writes CREATE OR REPLACE or ALTER, Teradata writes REPLACE PROCEDURE, REPLACE VIEW, REPLACE FUNCTION — a one-word tell that this is not ANSI SQL. NORMALIZE with MEETS OR OVERLAPS brings temporal period semantics that the other four dialects simply do not have.
REPLACE VIEW prodeiw.v_open ASSELECT NORMALIZE ON MEETS OR OVERLAPS
acct, valid_period FROM prodeiw.spans;
Coverage, by construct
Every rule name in the middle column is a rule in the grammar we ship. The three “not supported” rows are the ones that decide deals, so they are on the page rather than in an appendix.
Functions, with the full option set — DETERMINISTIC, null-input behavior, security, parameter style, GLOP sets
CREATE/REPLACE/ALTER/DROP FUNCTION
Supported
BTEQ dot-commands
BTEQ_COMMAND
Recognized at column 0 and sent to a hidden channel — scripts parse, the commands themselves do not
Cursor manipulation — OPEN, FETCH, CLOSE
teradata_sql_cursor_declaration
Declaration only
Loops in stored procedures — WHILE, LOOP, REPEAT, FOR, ITERATE, LEAVE
—
Not supported — no loop construct of any kind
Macros — CREATE MACRO, REPLACE MACRO, EXECUTE
—
Not supported — there is no MACRO token in any grammar file
ALTER TABLE, GRANT/REVOKE, EXPLAIN, HELP, CREATE DATABASE/USER, dynamic SQL
—
Not supported
FastLoad, MultiLoad, TPump, TPT control languages
—
Out of scope — separate utility languages
We do not publish a certified Teradata release range, and we are not going to invent one. What can be said is what the grammar demonstrably accepts: the MAP = TD_MAP1 clause in the shipped corpus is a 16.10-and-later feature, so the grammar is at least that modern. Every release of the engine ships its own example set, and that set is the honest record of what each version added.
We license five SQL-family engines. This is what the first page of a Teradata DDL export looks like, and exactly one of the five reads it.
Input — warehouse export, table plus statistics
prodeiw_ddl.sql
LOCKING ROW FOR ACCESSCREATE MULTISET TABLE prodeiw.SB_TRANS_EVENT ,FALLBACK ,NO BEFORE JOURNAL, NO AFTER JOURNAL,DATABLOCKSIZE = 524288BYTES, CHECKSUM = DEFAULT,DEFAULT MERGEBLOCKRATIO, MAP = TD_MAP1
(
SB_Event_Id INTEGER NOT NULL,
SB_File_Number SMALLINT NOT NULL,
SB_Event_Processing_Dt DATE FORMAT'YYYY-MM-DD'NOT NULL, SB_Activity_Type_Cd INTEGER NOT NULL COMPRESS (10251 ,10000 ,-10 ),
SB_Host_Activity_Val CHAR(10) CHARACTER SET LATINNOT CASESPECIFIC COMPRESS ('0001 ','SSVS01 ')
)
PRIMARY INDEX ( SB_File_Number );COLLECT STATISTICS USING SAMPLE10PERCENTCOLUMN (SB_Event_Processing_Dt) ON prodeiw.SB_TRANS_EVENT;
The table body is taken verbatim from the shipped example corpus, so this is a tested shape rather than an illustration. Nothing else on the list accepts it: Oracle PL/SQL has no MULTISET table, no journaling clauses and no PRIMARY INDEX; Transact-SQL spells physical design WITH (DISTRIBUTION = …); Firebird and our generic SQL engine have no equivalent surface at all.
You do not have to take this on trust. Paste a statement of your own into the Strumenta Playground, pick Teradata SQL, and read the tree the engine actually produces — including the issue list when it does not like something.
WHAT THIS ENGINE IS FOR
The tree arrives in Python, and Spark SQL comes out the other end.
Data engineers do not work in Kotlin. Two things follow from taking that seriously, and both of them are shipped code rather than a roadmap.
First, a generated Python API. The same metamodel that produces the Kotlin AST classes also generates Pylasu node classes and the unserializers to go with them, so a Python wrapper reads the parser’s JSON straight into typed Python objects. The model cannot drift between the two languages, because one grammar generates both. No other SQL engine in our catalog has that.
Second, a Teradata SQL to Spark SQL transpiler built on this parser. It is a separate component, and this page is not going to pretend it is a finished migration product — but the claim that Teradata trees can be walked into Spark SQL is demonstrated by code, not by a slide.
Be clear about the runtime: the Python package delegates parsing to the JVM engine and unserializes the result. A JVM is still required underneath. What you get in Python is the model and the API, not a pure-Python parser, and it is better to know that before your platform team designs around it.
# one JSON tree per source file
./gradlew run --args="parse prodeiw_ddl.sql"# the metamodel, as EMF/XMI
teradata-sql-parser metamodel --output TeradataSql.xmi
# and the generated Python API
starlasu-tools nodegen --using pylasu \
--from TeradataSql.xmi
starlasu-tools unserializergen --using pylasu
The command line, the metamodel export, and the generator that produces the Pylasu node classes. The AST documentation is generated from the same grammar, so the document and the model are never out of step.
6,336
LINES OF ANTLR GRAMMAR, FOUR FILES
12
RELEASES WITH THEIR OWN EXAMPLE CORPUS
7
LOCK TYPES, AT FOUR SCOPES
2
LANGUAGES THE AST IS GENERATED FOR
What you receive
A jar, a Python package on top of it, the generated documents that describe the tree, and a person to write to. It runs inside your own network, with no connection back to us and none to the warehouse the DDL came from.
01The engine
A JVM library that is also its own command-line tool
Kotlin on the JVM, built with ANTLR on Kolasu — the JVM implementation of Starlasu. Use it as a library, or run parse over a directory of exports and collect one JSON tree per file. The Python wrapper sits on top of the same jar.
library · CLI · Python wrapper
02The model
An AST generated from the grammar, in two languages
A typed AST with positions on every node and a positioned issue list on difficult input. Export the metamodel as EMF/XMI, serialize trees as JSON, and generate the Pylasu Python classes from the same metamodel. The engine supports LionWeb, so the model travels to tooling we did not write.
JSON · EMF/XMI · Pylasu · LionWeb
03The documents
Generated AST documentation, and the release corpus
The AST documentation is generated from the grammar rather than written alongside it. With it comes the example set for each release, which is the clearest record we have of what the engine accepts — and the one artifact we would rather you read than a coverage percentage.
AST document · examples per release
04The license
Standard, Distribution or Service — with support included
Standard for use inside your own organization, Distribution if the engine ships inside a product you sell, Service if it runs behind a service you operate. Support is part of the license, not an upsell. The guarantee runs a year and is extendable, or can be taken pay-as-you-go.
three tiers · support included
Onboarding
Week one is running the command line over your own DDL export and reading the issue list — not waiting for an environment.
A warehouse export is a single large file or a directory of them, which is the easiest possible input to test with. If the integration needs help, that is what the support in the license is for; if it needs more than help, we do that work too, as training or architectural design alongside your team. Write to products@strumenta.com.
THE REST OF THE ESTATE
The warehouse is the destination, not the source.
Nothing loads a Teradata warehouse by hand. The rows arrive from a mainframe COBOL batch or an IBM i application, and the columns you are trying to trace were named by a program written twenty years before the warehouse existed.
“Where does this column come from?” is not a Teradata question. It starts in a COBOL copybook or an RPG file definition, passes through an extract, and lands in a table whose PRIMARY INDEX somebody chose to match a key that only makes sense upstream. An inventory that stops at the warehouse edge answers the easy half.
Every Strumenta engine is built on Starlasu, so a Teradata tree, a COBOL tree and an RPG tree have the same shape, the same traversal model and the same API. One tool walks all of them — and where the SQL is embedded in the host program, it is a node in that program’s own tree rather than a string somebody has to re-parse. That is precisely what you cannot get by gluing unrelated open-source parsers together.
Every engine also supports LionWeb, so the models interchange with LionWeb-compliant tooling instead of being locked inside one process — and every engine is available through bindings from Java, Kotlin, Python, TypeScript and C#. The host application picks the language; the parser does not.
Why license this rather than bend a generic SQL grammar
For Oracle or Transact-SQL there is a free public grammar to start from, and it is a reasonable starting point. For Teradata there is not: the platform is a closed commercial product and its documentation sits behind a login. The realistic free option is a generic ANSI SQL grammar, so it is worth being concrete about where that stops.
What a generic ANSI SQL grammar does with the first page of a Teradata export. This is not a straw man — it is the normal way teams discover the problem.
In your export
A generic ANSI SQL grammar
This engine
CREATE MULTISET TABLE … FALLBACK, MAP = TD_MAP1
Syntax error at MULTISET
A rule per option, in the tree
COMPRESS (10251, 10000, -10) on a column
No such column attribute
Parsed, so it can be reported as dropped
DATE FORMAT 'YYYY-MM-DD' inside a type
A string literal where a type belongs
Supported
LOCKING ROW FOR ACCESS SELECT …
A statement that starts with a modifier
Supported
A BTEQ script with dot-commands at column 0
Fails on the first .LOGON
Dot-commands hidden, SQL inside them parses
A model your tooling can share with COBOL and RPG
Raw parse-tree contexts
Starlasu AST, LionWeb, Python and JVM APIs
Someone to call when a customer file fails
The issue tracker, and good luck
Support included in the license
Writing the physical-design DDL alone is weeks of careful reading against documentation you have to have access to, and getting the BTEQ column-0 rule right is the kind of detail that only surfaces when a real customer script fails. The method behind this work has a name: read the Chisel Method, or Parsers and transpilers for the three ways into it.
What to discuss before you license
Six things we would rather tell you now than have you find in week three. Two of them lose deals, and they are still here.
Macros are not supported. There is no MACRO token anywhere in the grammar, so CREATE MACRO, REPLACE MACRO and EXECUTE macroname do not parse. Teradata macros are a real and widely used feature, and in some estates they carry a large share of the business logic. Count yours before anything else; if the answer is “most of it”, this engine does not fit yet and we should talk about what it would take.
Stored procedures have no loops.WHILE, LOOP, REPEAT, FOR, ITERATE and LEAVE are absent — the compound statement, declarations, handlers, cursor declarations and IF/ELSEIF/ELSE are all there, and then the loops are not. Neither are SIGNAL/RESIGNAL or OPEN/FETCH/CLOSE. A procedure with a cursor loop in it will not parse today.
BTEQ is skipped, not understood. Dot-commands at column 0 go to a hidden channel so your scripts do not break. There is no node for them and no control flow. .IF ERRORCODE <> 0 THEN .QUIT 8 is invisible to any analysis you build on the tree.
The load utilities are out of scope. FastLoad, MultiLoad, TPump and TPT have their own control languages, and none of them is in this grammar.
Parts of the operational DDL are missing.ALTER TABLE, GRANT/REVOKE, EXPLAIN, HELP, CREATE DATABASE and CREATE USER are not in the grammar. An export that is mostly CREATE TABLE is the good case; an export that is mostly ALTER TABLE history is not.
This is a young, focused component. It is at an early version number and we are not going to dress it up as a mature platform. It does one job — reading warehouse DDL and query code so a migration can be sized and executed — and the release-by-release example corpus is the evidence for what it does today. We do not publish a test count for it, so we are not printing one.
What we will ask you
Is your corpus DDL exports, BTEQ scripts, macro bodies or stored procedures — and roughly in what proportions?
Do your procedures use loops or cursor manipulation?
Are FastLoad, MultiLoad or TPump scripts in scope?
Which Teradata release, and do you use period data types, NORMALIZE or column partitioning?
What is the target platform — Spark SQL, where a transpiler already exists, or something else where you supply the generator?
Kotlin, Java or Python for the consuming code? Remember the Python API still needs a JVM underneath.
Do you want the tree as JSON, as EMF/XMI, as LionWeb, or as in-process objects?
How many statements and how many distinct tables? Sizing changes the shape of the first engagement.
Straight answers
Can we feed it our BTEQ scripts as they are?
Yes, and the SQL inside them will parse. A line beginning at column 0 with a dot or an equals sign is consumed to the end of the line and sent to a hidden channel, so the dot-commands do not break the parse. What you will not get is a model of them: no node for .IF ERRORCODE, no branch structure, no .EXPORT target. If your analysis needs the script logic and not just the statements, that is extra work and we will scope it as such.
Does it handle Teradata macros?
No. There is no MACRO token in any of the grammar files, so CREATE MACRO, REPLACE MACRO and EXECUTE are not supported today. We know that matters — macros carry real logic in many estates — which is why it is on the page rather than in a footnote. Tell us what proportion of your corpus they are and we will tell you honestly whether to buy this now.
Can we work with the tree from Python?
Yes, and it is generated rather than hand-maintained: the same metamodel that produces the Kotlin classes generates the Pylasu node classes and their unserializers, and a Python wrapper reads the parser’s JSON into typed Python objects. One caveat that matters for your architecture — parsing still happens on the JVM. The Python package delegates to it. There is no pure-Python Teradata parser here.
Does it convert Teradata SQL to something else?
There is a Teradata SQL to Spark SQL transpiler built on this parser, as a separate component. It is real code and it is the reason we can talk about targets rather than only about analysis. It is not a finished, warranted migration product, and this engine does not write Teradata SQL back out from a modified tree at all. For any other target, you get the model and write the generator — which is a normal and estimable piece of work.
What is the first useful thing we can do with it?
Parse the DDL export and count. Tables by multiplicity, by index strategy, by partitioning; which tables carry COMPRESS lists that the target platform has no equivalent for; which have no statistics collected. That inventory is normally the first paid engagement in a warehouse migration, it is the artifact the bid is built on, and it needs nothing but the export and the command line.
Send us a DDL exportand a BTEQ script, if you have one
Bring us the export, and we will count what parses.
A representative slice — the CREATE TABLE statements, a few stored procedures, one real BTEQ job — and we will come back with what the engine reads, how many macros and loops are in the way, and what an inventory of your warehouse would actually cost. If the answer is that this is not the right product yet, we will say that.