Clean RPC without SOAP or XML-RPC?
No doubt a number of you have read DJ Adams' article Real Web Services with REST and ICF, which discusses how to create real Web Services using the ICF, REST and the power of the HTTP protocol. I wanted to explore this a bit more, and show how to create your very own generic RPC mechanism for R3, without having to use SOAP, or XML-RFC, but still use widely available Open standards, while borrowing from RESTian principles.
The Problem
I just don't buy into really complicated things, these days.I don't like them - I feel it best to at least try and keep life simple, so the warning bells are immediately triggered when I find that I have to spend more than a few hours getting my head round a new subject area. For me - SOAP is one of those things, especially when it comes to RPC.
I want a simple RPC mechanism. I want to use HTTP as the transport protocol, but I am interested in the possible application of the RESTian concept of broadening the use of HTTP into the application level protocol (OK - bad word) of RPC. To me, SOAP hides the identity of the application within the payload, and I wanted to bring this back out into the URI (open), where I can see it (a good policy with your enemies :-). SOAP mixes the error handling of the application, with the error handling of the transport protocol (typically it throws a 500, with a fault document, essentially duplicating information yet not giving anything useful at the HTTP layer), but I wanted to separate this between what is a generic error (application not found 404, payload is corrupted 400, if an unhandled exception occurs in the WebAS then a 500 is issued, etc), and what is an application specific error (EXCEPTIONS in the result document with a 200 OK returned). The benefits of this are quite substantial, as it enables client applications to glean quite a bit of information about where a problem lies eg. is it a transport problem, a generic RPC request format error, an authorisation issue etc., without actually having to unpack the response to do it.
The Solution
The solution that I have in mind is to use the WebAS and HTTP as the transport protocol, identify the application as the RFC name on the URI, and to represent the RPC interface parameters in YAML.YAML - what is it?
SOAP, by virtue of XML, is a data encoding language (for want of a better description). YAML is a data serialisation mechanism. This means that YAML is not interested in creating a document structure, so much as it is interested in creating a format for data that is readily transferable between machines, and (possibly more importantly) between programming languages.YAML vs the SOAP Envelope or XML-RPC payload
YAML vs SOAP or XML-RPC is more like - encoding vs serialisation.RPC doesn't need to have encoding, it just needs some form of serialisation, a well defined contract between the two players (client and server), and a format for data exchange between systems, or programming languages - not necessarily a mechanism for deriving/delivering semantic meaning ie. an encoding, as this is allready defined by the contract between the partners.
For this reason SOAP, via XML seems over engineered to me, before you even start looking at the cling-ons of WSDL, UDDI, etc.
"Whats the benefit", you say
- easily machine readable, and human readable payload, as it carries only the data, and the bare minimum metadata to identify it
- the application is identified by the endpoint, so there is some transparency of what is going on without unpacking the payload (great for firewalls that can inspect the URI eg. CheckPoint, being able to control who can do what)
- Errors related to application access, and serialisation issues are handled by HTTP specific responses - application related errors are handled by data serialised values, so a distinction is made between generic transport, and application identity/compliance issues, and the application specific issues within. This helps keep the server, and client light weight
"this isn't new", I hear you cry, "so why reinvent things?"
This is very true - it isn't new, and it is reinventing things that already exist, but there can be good reasons for doing that.
- Low barrier to entry with a simplified application blue print
- SOAP is not a light weight implementation - if performance is a key issue then you can do better
- SAP is a unique area, in that it has its own very specific form of RPC in the guise of RFC (or more specifically - Function Modules), yet it has a very broad user base world wide. If you create a solution for SAP, even though it is for a very specific product (R/3) it is immediately relevent to a huge audience. This IS instant interoperabilty if you choose to share it.
- If your own inter-application, or inter-system RPC requirement is of an internal nature, yet broad reaching due to your business operating environment, why shouldn't you use a solution that is tuned to your own needs? - as much as I don't like it - XP would agree with this kind of development strategy.
Support?
one of the reasons I chose YAML for this experiment, is because there is broad ranging support for it across programming languages, and platforms. There are currently library implementations for C, Perl, Ruby, and Python (that list maybe incomplete), and as you can see from my handler described below, it wasn't exactly difficult to implement a coarse de-serialiser in ABAP. This could be true of any number of other serialisation formats such as:- URL Encoding
- CSV - Comma Separated Values
- Tab Delimiters (or any delimiter for that matter)
- ...
Back to my implementation
The RPC mechanism I created, is a straight forward ICF handler. The source code for it is available here. As usual, create the class in transaction SE24, and set the interface to IF_HTTP_EXTENSION. The source code goes in the only method - IF_HTTP_EXTENSION~HANDLE_REQUEST.The service is configured through R/3 transaction SICF, and my config looks like:
I have written a small program in Perl to act as the client. It looks like this
It runs like this:
cat rfc_read_report.yml | ./yaml.pl -u developer -p developer -i http://seahorse.local.net:8000/basis/yaml/RFC_READ_REPORTThe input from file rfc_read_report.yml makes up the body of the HTTP POST, and looks like:
And the output looks like:
Having a closer look at the request/response cycle, the HTTP request looks like this:
POST /piers/yamlrpc/RFC_READ_REPORT HTTP/1.1 te: deflate,gzip;q=0.3 connection: TE, close authorization: xxxxxxxxx host: seahorse.local.net:8000 user-agent: sneakyhack/0.1 libwww-perl/5.79 content-length: 32 --- #YAML:1.0 PROGRAM: SAPLGRAP
You can see that the PATH_INFO of the URI asks for a specific application (Function Module) to run, and the request body contains the serialised list of parameters.
The response looks like:
HTTP/1.1 200 OK Set-Cookie: sap-recorder_sid=%2cc%3dNW4-0A010129111040F9AF1B007A00000000--1; path=/; domain=local.net Content-Type: text/plain; charset=iso-8859-1 Content-Length: 1354 server: SAP Web Application Server (1.0;640) --- #YAML:1.0 SYSTEM: NW4 TRDIR: SAPLGRAP .... QTAB: - '* regenerated at 04.08.1995 15:50:21 by GRAPH' - '*******************************************************************' - '* System-defined Include-files. *' - '*******************************************************************' - '' - 'INCLUDE LGRAPTOP. " Global Data' - 'INCLUDE LGRAPUXX. " Function Modules' - ...
The response contains the serialised list of exporting parameters, and tables.
The RFC_READ_REPORT example is the simplest of forms. There are no complex values, such as structured import parameters, or tables, after all - it is my version of "Hello World", for RFC.
For a slightly more complex example, I have RFC_READ_TABLE. It allows us to select rows from an arbitrarily named table, limit the rows selected, and pass options directly to the SQL WHERE clause. Again, the application is identified through the URI, so that becomes http://seahorse.local.net:8000/basis/yaml/RFC_READ_TABLE,and so my serialised input parameters look like:
--- #YAML:1.0 QUERY_TABLE: TRDIR ROWCOUNT: 10 OPTIONS: - NAME LIKE 'SAPL%RFC%'
The results look like: this
Summary:
Stepping back a bit - what I have shown is a generic framework for doing arbitrary FM calls (you could call ANY FM in R/3 - not just RFCs), and you could use any serialisation/deserialisation mechanism you like.SAP have just made it possible by providing the engine that runs the RPC (just look at the ABAP help for CALL FUNCTION and the options for Dynamic parameter passing), and the transport mechanism (WebAS) to carry it.
There is nothing written anywhere that says when you want to have interoperability that you have to use SOAP (or XML-RPC), or even XML to achieve it. I hope I have proven this through the use of open, and easily accessable standards, that have wide acceptance across the various programming language communities, and (by this virtue) the tools to support it.
Operating Environment notes:
The environment I used for developing this was:
- RedHat 9
- Perl 5.8.0
- YAML 0.35
- SAP: Enterprise 4.7 (should work with any 6.20+ kernel)
