* (Update Jul 2011: Just fixing the download links as I got many emails even after 3 years regarding the download)* As I started implementing each and every ABAP keyword in Java, I stopped for a moment and realized I am going no-where. Based on the comments of my part #1 of this blog series, I may not (or may) be able to simulate some keywords, but most of the keywords can be covered and simulated. I even asked myself what would be the actual use of it – but realized, it will open a new window to ABAPers to develop for the first time outside SAP environment. It can even be a tool for many new starters to learn ABAP without an SAP server. With all these in my mind, I started to plan a good architecture for building the ABAP parser in a flexible and a scalable fashion. h2. Parser Architecture 
h2. Architecture description Every language whether it is spoken language or a computer language, it will have a grammar. This grammar defines the rules or syntaxes for that language. Though many languages including ABAP don’t have an official version of this grammar, it is still available in other forms or can construct the grammar from the language syntax help. Well, what is this grammar? Is there any representation for it? YES. EBNF means Extended Backus-Naur Form, the grammar for describing parser grammars. Like lex and yacc, ANTLR is one such parser generator that understands EBNF grammar files. The idea for this ABAP parser is to have a grammar defined for each ABAP keyword or statement. All the grammar files are compiled to generate Java files and then to Jar using an ANT task. This generated Java files will call the semantic code for either interpreting or translating. This ABAP parser is completely coded in ANTLR v3.0.1. Please note that the EBNF grammar for ABAP provided here must not be used to validate ABAP syntax because, the motive is just to parse. The reason being ABAP supports multiple additions for many statements and the yet other complexity is that it can occur in any order. Just to parse and make sure the parser traverses through all the tokens, zero or multi occurrences for each additions would easily provide a solution for a parser but fails if the same addition for that ABAP statement occurs twice. There are options like memoize =true in ANTLR, but some commands in ABAP do allow multiple additions. h2. A sample rule To explain a bit further let’s take only the SELECT statement in ABAP,
In the above diagram the whole flow of the parsing logic for SELECT statement is provided. Below is the EBNF grammar for SELECT written in ANTLR. select :'SELECT' ('SINGLE'|'FOR' 'UPDATE'|'DISTINCT')* (abapVariable+|'*'|fieldVariable)
select_into_part?
'FROM' (abapVariable|fieldVariable) ('CLIENT' 'SPECIFIED'
|'BYPASSING' 'BUFFER'|'UP' 'TO' abapValueOrVar 'ROWS')*
('WHERE' sql_condition|'FOR' 'ALL' 'ENTRIES' 'IN' abapVariable)*
('GROUP' 'BY' (abapVariable+|fieldVariable))? ('ORDER' 'BY' (abapVariable+|fieldVariable))?
;
fragment select_into_part
:'INTO'
(abapVariable|'CORRESPONDING' 'FIELDS' 'OF' ('TABLE')? abapVariable
|'(' abapVariable (',' abapVariable)+ ')'|'TABLE' abapVariable) |'APPENDING' ('TABLE' abapVariable|'CORRESPONDING' 'FIELDS' 'OF' 'TABLE' abapVariable) ; sql_condition : simpleSqlLogicalExpr (('AND'|'OR') simpleSqlLogicalExpr)*
;
fragment simpleSqlLogicalExpr
:
sqlLogicOperand (sqlLogicExprOperator sqlLogicOperand)* | abapVariable ('NOT')? 'BETWEEN' abapValueOrVar 'AND' abapValueOrVar | abapVariable 'IN' '(' abapValueOrVar (',' abapValueOrVar)* ')' | abapVariable 'IN' abapValueOrVar ; fragment sqlLogicOperand : abapVariable|ABAP_STR|floatToken|INTEGER|'(' sql_condition ')' ; fragment sqlLogicExprOperator :logicalNumericalOperator|logicalStringOperator|logicalBinaryOperator; h2. Download {code:html}*ABAP2Java*{code} (Bad URL Fixed) *License:* Creative Commons Attribution +(free for commerical and personal use as long as you give credit to the author)+ Got the code, what to do next or how to compile and use it? Once you download, you can find the entire development environment like ant and ANTLR works including the grammar files in the folder grammar similar to one in the sample. Each grammar file represents the ABAP keyword. All the grammar files are complete but let me know if there are any fixes. You will find build.bat that invokes ant tasks for compiling every grammar using ANTLR. Please be patient for the compile job to complete as it will compile 147 grammars to Java code. In my dual-core laptop it took around 5 minutes.
h2. Architecture description Every language whether it is spoken language or a computer language, it will have a grammar. This grammar defines the rules or syntaxes for that language. Though many languages including ABAP don’t have an official version of this grammar, it is still available in other forms or can construct the grammar from the language syntax help. Well, what is this grammar? Is there any representation for it? YES. EBNF means Extended Backus-Naur Form, the grammar for describing parser grammars. Like lex and yacc, ANTLR is one such parser generator that understands EBNF grammar files. The idea for this ABAP parser is to have a grammar defined for each ABAP keyword or statement. All the grammar files are compiled to generate Java files and then to Jar using an ANT task. This generated Java files will call the semantic code for either interpreting or translating. This ABAP parser is completely coded in ANTLR v3.0.1. Please note that the EBNF grammar for ABAP provided here must not be used to validate ABAP syntax because, the motive is just to parse. The reason being ABAP supports multiple additions for many statements and the yet other complexity is that it can occur in any order. Just to parse and make sure the parser traverses through all the tokens, zero or multi occurrences for each additions would easily provide a solution for a parser but fails if the same addition for that ABAP statement occurs twice. There are options like memoize =true in ANTLR, but some commands in ABAP do allow multiple additions. h2. A sample rule To explain a bit further let’s take only the SELECT statement in ABAP,In the above diagram the whole flow of the parsing logic for SELECT statement is provided. Below is the EBNF grammar for SELECT written in ANTLR. select :'SELECT' ('SINGLE'|'FOR' 'UPDATE'|'DISTINCT')* (abapVariable+|'*'|fieldVariable)select_into_part?
'FROM' (abapVariable|fieldVariable) ('CLIENT' 'SPECIFIED'
|'BYPASSING' 'BUFFER'|'UP' 'TO' abapValueOrVar 'ROWS')*
('WHERE' sql_condition|'FOR' 'ALL' 'ENTRIES' 'IN' abapVariable)*
('GROUP' 'BY' (abapVariable+|fieldVariable))? ('ORDER' 'BY' (abapVariable+|fieldVariable))?
;
fragment select_into_part
:'INTO'
(abapVariable|'CORRESPONDING' 'FIELDS' 'OF' ('TABLE')? abapVariable
|'(' abapVariable (',' abapVariable)+ ')'|'TABLE' abapVariable) |'APPENDING' ('TABLE' abapVariable|'CORRESPONDING' 'FIELDS' 'OF' 'TABLE' abapVariable) ; sql_condition : simpleSqlLogicalExpr (('AND'|'OR') simpleSqlLogicalExpr)*
;
fragment simpleSqlLogicalExpr
:
sqlLogicOperand (sqlLogicExprOperator sqlLogicOperand)* | abapVariable ('NOT')? 'BETWEEN' abapValueOrVar 'AND' abapValueOrVar | abapVariable 'IN' '(' abapValueOrVar (',' abapValueOrVar)* ')' | abapVariable 'IN' abapValueOrVar ; fragment sqlLogicOperand : abapVariable|ABAP_STR|floatToken|INTEGER|'(' sql_condition ')' ; fragment sqlLogicExprOperator :logicalNumericalOperator|logicalStringOperator|logicalBinaryOperator; h2. Download {code:html}*ABAP2Java*{code} (Bad URL Fixed) *License:* Creative Commons Attribution +(free for commerical and personal use as long as you give credit to the author)+ Got the code, what to do next or how to compile and use it? Once you download, you can find the entire development environment like ant and ANTLR works including the grammar files in the folder grammar similar to one in the sample. Each grammar file represents the ABAP keyword. All the grammar files are complete but let me know if there are any fixes. You will find build.bat that invokes ant tasks for compiling every grammar using ANTLR. Please be patient for the compile job to complete as it will compile 147 grammars to Java code. In my dual-core laptop it took around 5 minutes.
6 Comments
Bluesky
