Calling AWS API using Signature 4

[Stephen3]

Hi All;

I have already asked this question but provide a complicated example program. To simplify the question, I am trying to duplicate the results from this AWS example:

https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html

I can get a program to get the results for "Task 1" and "Task 2" as per AWS's example, but I cannot get the Example Signature result. I have a sample program below which hard codes the values for "Task 3" but I still don't get the Example Signature. I can get the desired results using Python but I would think I could get them from ABAP. I think the issue is the Python Command "HexEncode" represents a function that returns a base16 encoding of the result sin lower case as per AWS's explaintion.

Does anyone know how to generate a base16 hex xstring? Here is my test program.

REPORT ZTEST_CHAR_TO_HEX.

data: sSigningKey type string.
data: xSigningKey type xstring.
data: buffer1 type xstring.
data: StringToSign type string.
data: lf type string.
data: sSignature type string,
kSignature type xstring,
EF_HMACB64String type string.

DATA: conv TYPE REF TO cl_abap_conv_out_ce.

lf = cl_abap_char_utilities=>newline. " '\n'.

sSigningKey = 'c4afb1cc5771d871763a393e44b703571b55cc28424d1a5e86da6ed3c154a4b9'.
* Convert to Xstring
TRY.
xSigningKey = cl_abap_hmac=>string_to_xstring( sSigningKey ).
CATCH cx_abap_message_digest .
ENDTRY.

concatenate 'AWS4-HMAC-SHA256' lf '20150830T123600Z' '20150830/us-east-1/iam/aws4_request' lf 'f536975d06c0309214f805bb90ccff089219ecd68b2577efef23edd43b7e1a59' into StringToSign.

TRY.
cl_abap_hmac=>calculate_hmac_for_char(
EXPORTING
if_algorithm = 'SHA256'
if_key = xSigningKey
if_data = StringToSign
IMPORTING
EF_HMACSTRING = sSignature
EF_HMACXSTRING = kSignature
EF_HMACB64STRING = EF_HMACB64STRING
).
CATCH cx_abap_message_digest. "
CLEAR kSignature.
ENDTRY.

DATA(xstr) = cl_abap_codepage=>convert_to( source = sSignature
codepage = 'utf-16le' ).


cl_demo_output=>write_data( sSignature ).
cl_demo_output=>write_data( kSignature ).
cl_demo_output=>write_data( EF_HMACB64STRING ).
cl_demo_output=>write_data( xstr ).
cl_demo_output=>display( ).

conv = cl_abap_conv_out_ce=>create(
encoding = '4102'
endian = 'L' ).

conv->convert( EXPORTING data = sSignature
IMPORTING buffer = buffer1 ).

write: sSignature.
write: buffer1.

[Stephen3]

Hi All;

I have already asked this question but provide a complicated example program. To simplify the question, I am trying to duplicate the results from this AWS example:

https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html

I can get a program to get the results for "Task 1" and "Task 2" as per AWS's example, but I cannot get the Example Signature result. I have a sample program below which hard codes the values for "Task 3" but I still don't get the Example Signature. I can get the desired results using Python but I would think I could get them from ABAP. I think the issue is the Python Command "HexEncode" represents a function that returns a base16 encoding of the result sin lower case as per AWS's explaintion.

Does anyone know how to generate a base16 hex xstring? Here is my test program.

REPORT ZTEST_CHAR_TO_HEX.

data: sSigningKey type string.
data: xSigningKey type xstring.
data: buffer1 type xstring.
data: StringToSign type string.
data: lf type string.
data: sSignature type string,
kSignature type xstring,
EF_HMACB64String type string.

DATA: conv TYPE REF TO cl_abap_conv_out_ce.

lf = cl_abap_char_utilities=>newline. " '\n'.

sSigningKey = 'c4afb1cc5771d871763a393e44b703571b55cc28424d1a5e86da6ed3c154a4b9'.
* Convert to Xstring
TRY.
xSigningKey = cl_abap_hmac=>string_to_xstring( sSigningKey ).
CATCH cx_abap_message_digest .
ENDTRY.

concatenate 'AWS4-HMAC-SHA256' lf '20150830T123600Z' '20150830/us-east-1/iam/aws4_request' lf 'f536975d06c0309214f805bb90ccff089219ecd68b2577efef23edd43b7e1a59' into StringToSign.

TRY.
cl_abap_hmac=>calculate_hmac_for_char(
EXPORTING
if_algorithm = 'SHA256'
if_key = xSigningKey
if_data = StringToSign
IMPORTING
EF_HMACSTRING = sSignature
EF_HMACXSTRING = kSignature
EF_HMACB64STRING = EF_HMACB64STRING
).
CATCH cx_abap_message_digest. "
CLEAR kSignature.
ENDTRY.

DATA(xstr) = cl_abap_codepage=>convert_to( source = sSignature
codepage = 'utf-16le' ).


cl_demo_output=>write_data( sSignature ).
cl_demo_output=>write_data( kSignature ).
cl_demo_output=>write_data( EF_HMACB64STRING ).
cl_demo_output=>write_data( xstr ).
cl_demo_output=>display( ).

conv = cl_abap_conv_out_ce=>create(
encoding = '4102'
endian = 'L' ).

conv->convert( EXPORTING data = sSignature
IMPORTING buffer = buffer1 ).

write: sSignature.
write: buffer1.

[Sofidel]

Dear Stephen,

Did you solved?

I am facing the same problem to get the right signature for aws.

Thank you.

Regards.

[Stephen3]

Hi Guido

I did get it working. Here is a simple program that reads an S3 bucket. You can probably tighten up the code but I wanted to keep it simple so I could get it working.

It didn't cut and paste well from NotePad++. Not sure why.

You have to replace "FILL IN" with your own AWS secret key and AWS access key.

*&---------------------------------------------------------------------* *& Report ZAWS_GET_S3 * *& * *& Reads an S3 Queue. Based on this example * *& https://docs.aws.amazon.com/general/latest/gr/sigv4-calculate-signature.html *& *& AWS link for sqs API calls *& https://docs.aws.amazon.com/AWSSimpleQueueService/latest/APIReference/Welcome.html *& *& SSH 2020-09-01 - Works * *&---------------------------------------------------------------------* *& Note, Polling Factor in AWS assigned to the queue determines how * *& often a message is availabel for downloading. Once it is downloaded * *&, it moves to the "InFlight" queue. After x number of seconds, it * *& moves back to the being available. * *&----------------------------------------------------------------------* REPORT ZAWS_GET_S3. DATA: utctime type timestamp, utcstring type string, amzdate type string, datestamp type string. DATA: lf TYPE string. DATA: AccessKey TYPE STRING, secret_key TYPE STRING. DATA: method type string, service type string, host type string, region type string, endpoint type string, request_parameters type string, canonicalresourcepath type string, canonicalqueryparameters type string, canonicalheaders type string, canonicalheadernames type string, canonical_request type string, canonical_request_hash type string, xcanonical_request_hash type xstring, algorithm type string, credential_scope type string, string_to_sign type string, string_to_sign_hash type string, payload type string, bodyhash type string, xpayload_hash type xstring, signing_key type string, signature type string, Authorization_Header type string, request_url type string, request_uri type string. DATA: AWSkey TYPE string, utf8temp TYPE xstring, kSecret TYPE xstring, kDate TYPE xstring, kRegion TYPE xstring, kService TYPE xstring, kSigningKey TYPE xstring, EF_HMACSTRING TYPE STRING, EF_HMACXSTRING TYPE XSTRING, EF_HMACB64STRING TYPE STRING, sSignature TYPE STRING, kSignature TYPE XSTRING, sSigningKey TYPE STRING, StringToSign TYPE string, signedSignature TYPE string. DATA: client TYPE REF TO if_http_client, hfields TYPE TIHTTPNVP, ffields TYPE TIHTTPNVP. DATA: code TYPE I, cresponse TYPE string, reason TYPE string. DATA: ef_hashstring TYPE STRING, EF_HASHXSTRING TYPE XSTRING, EF_HASHB64STRING TYPE STRING, EF_HASHX TYPE XSTRING. ** SELECTION SCREEN **************************************************** PARAMETERS: PDATE TYPE CHAR16. ** INITIALIZATION ****************************************************** INITIALIZATION. PERFORM UTC_DATETIME_ISO_8601. PDATE = AMZDATE(16). ************************************************************************ * START OF SELECTION * ************************************************************************ START-OF-SELECTION. lf = cl_abap_char_utilities=>newline. " '\n'. payload = ''. Secret_Key = 'FILL IN'. AccessKey = 'FILL IN'. amzdate = pdate. datestamp = amzdate(8). service = 's3'. method = 'GET'. request_parameters = ''. canonicalqueryparameters = request_parameters. host = 's3.amazonaws.com'. region = 'us-east-1'. canonicalresourcepath = '/'. * Need HMAC-SHA256 * Create the signature with the key and the request string * Values for "if_algorithm * SHA256 * SHA384 * SHA512 TRY. cl_abap_message_digest=>CALCULATE_HASH_FOR_CHAR( EXPORTING if_algorithm = 'SHA256' if_data = payload * if_length = IMPORTING ef_hashstring = bodyhash " Hash value as hex encoded string EF_HASHXSTRING = ef_hashxstring " Binary hash value as XString EF_HASHB64STRING = ef_hashb64string " Hash value as base64 encoded string EF_HASHX = EF_HASHX " Hash value as XSequence ). CATCH cx_abap_message_digest. " CLEAR bodyhash. ENDTRY. * Must be lower case translate bodyhash to lower case. canonicalheadernames = 'host;x-amz-content-sha256;x-amz-date'. canonicalheaders = 'host:' && host && lf && 'x-amz-content-sha256:' && bodyhash && lf && 'x-amz-date:' && amzdate && lf. * cl_demo_output=>write_data( bodyhash ). * cl_demo_output=>display( ). * canonical_request concatenate method lf canonicalresourcepath lf canonicalqueryparameters lf canonicalheaders lf canonicalheadernames lf bodyhash into canonical_request. * cl_demo_output=>write_data( canonical_request ). * cl_demo_output=>display( ). canonical_request_hash = ''. TRY. cl_abap_message_digest=>CALCULATE_HASH_FOR_CHAR( EXPORTING if_algorithm = 'SHA-256' " Algorithm Options SHA256 sha-256 if_data = canonical_request * if_length = IMPORTING ef_hashstring = canonical_request_hash " Hash value as hex encoded string EF_HASHXSTRING = ef_hashxstring " Binary hash value as XString EF_HASHB64STRING = ef_hashb64string " Hash value as base64 encoded string EF_HASHX = EF_HASHX " Hash value as XSequence ). CATCH cx_abap_message_digest. CLEAR canonical_request_hash. ENDTRY. translate canonical_request_hash to lower case. * cl_demo_output=>write_data( canonical_request_hash ). * cl_demo_output=>display( ). algorithm = 'AWS4-HMAC-SHA256'. concatenate datestamp '/' region '/' service '/' 'aws4_request' into credential_scope. concatenate algorithm lf amzdate lf credential_scope lf canonical_request_hash into string_to_sign. * cl_demo_output=>write_data( string_to_sign ). * cl_demo_output=>display( ). concatenate 'AWS4' secret_key into AWSkey. TRY. kSecret = cl_abap_hmac=>string_to_xstring( AWSkey ). CATCH cx_abap_message_digest . ENDTRY. TRY. cl_abap_hmac=>calculate_hmac_for_char( EXPORTING if_algorithm = 'SHA256' if_key = kSecret if_data = datestamp IMPORTING EF_HMACSTRING = EF_HMACSTRING EF_HMACXSTRING = kDate EF_HMACB64STRING = EF_HMACB64STRING ). CATCH cx_abap_message_digest. " CLEAR kDate. ENDTRY. TRY. cl_abap_hmac=>calculate_hmac_for_char( EXPORTING if_algorithm = 'SHA256' if_key = kDate if_data = region IMPORTING EF_HMACSTRING = EF_HMACSTRING EF_HMACXSTRING = kRegion EF_HMACB64STRING = EF_HMACB64STRING ). CATCH cx_abap_message_digest. " CLEAR kRegion. ENDTRY. TRY. cl_abap_hmac=>calculate_hmac_for_char( EXPORTING if_algorithm = 'SHA256' if_key = kRegion if_data = service IMPORTING EF_HMACSTRING = EF_HMACSTRING EF_HMACXSTRING = kService EF_HMACB64STRING = EF_HMACB64STRING ). CATCH cx_abap_message_digest. " CLEAR kService. ENDTRY. TRY. cl_abap_hmac=>calculate_hmac_for_char( EXPORTING if_algorithm = 'SHA256' if_key = kService if_data = 'aws4_request' IMPORTING EF_HMACSTRING = sSigningKey EF_HMACXSTRING = kSigningKey EF_HMACB64STRING = EF_HMACB64STRING ). CATCH cx_abap_message_digest. " CLEAR kService. ENDTRY. * translate sSigningKey to lower case. * cl_demo_output=>write_data( sSigningKey ). * cl_demo_output=>display( ). StringToSign = 'AWS4-HMAC-SHA256' && lf && amzdate && lf && datestamp && '/' && region && '/' && service && '/aws4_request' && lf && canonical_request_hash. TRY. cl_abap_hmac=>calculate_hmac_for_char( EXPORTING if_algorithm = 'SHA256' if_key = kSigningKey if_data = StringToSign IMPORTING EF_HMACSTRING = sSignature EF_HMACXSTRING = kSignature EF_HMACB64STRING = EF_HMACB64STRING ). CATCH cx_abap_message_digest. " CLEAR kService. ENDTRY. TRANSLATE sSignature TO LOWER CASE. * cl_demo_output=>write_data( StringToSign ). * cl_demo_output=>write_data( sSignature ). * cl_demo_output=>write_data( kSignature ). * cl_demo_output=>write_data( EF_HMACB64STRING ). * cl_demo_output=>display( ). Authorization_Header = 'AWS4-HMAC-SHA256 Credential=' && accesskey && '/' && credential_scope && ', SignedHeaders=' && canonicalheadernames && ', Signature=' && sSignature. request_url = 'https://s3.amazonaws.com/'. request_uri = 'https://s3.amazonaws.com/'. CALL METHOD cl_http_client=>create_by_url EXPORTING url = request_url proxy_service = '' IMPORTING client = client EXCEPTIONS argument_not_found = 1 plugin_not_active = 2 internal_error = 3 OTHERS = 4. IF sy-subrc <> 0. write: / 'ERROR creating client.'. EXIT. ENDIF. cl_http_utility=>set_request_uri( request = client->request uri = request_uri ). CALL METHOD client->request->set_header_field EXPORTING name = 'x-amz-content-sha256' value = bodyhash. client->propertytype_accept_cookie = client->co_enabled. client->request->set_method( if_http_request=>co_request_method_get ). CALL METHOD client->request->set_header_field EXPORTING name = 'x-amz-date' value = amzdate. CALL METHOD client->request->set_header_field EXPORTING name = 'Host' value = host. CALL METHOD client->request->set_header_field EXPORTING name = 'Authorization' value = Authorization_Header. * CALL METHOD client->request->set_header_field * EXPORTING * name = 'Action' * value = 'ListUsers'. * * CALL METHOD client->request->set_header_field * EXPORTING * name = 'Version' * value = '2010-05-08 HTTP/1.1'. CALL METHOD client->send EXCEPTIONS http_communication_failure = 1 http_invalid_state = 2 http_processing_failed = 3 http_invalid_timeout = 4 OTHERS = 5. CALL METHOD client->receive EXCEPTIONS http_communication_failure = 1 http_invalid_state = 2 http_processing_failed = 3 OTHERS = 4. * Get fields returned CALL METHOD client->response->get_header_fields CHANGING fields = hfields. CALL METHOD client->response->get_form_fields CHANGING fields = ffields. CALL METHOD client->response->get_status IMPORTING code = code reason = reason. CALL METHOD client->response->if_http_entity~get_cdata RECEIVING data = cresponse. * This works * cl_demo_output is failing for some reason * cl_demo_output=>display_json( cresponse ). * cl_demo_output=>DISPLAY_XML( cresponse ). data: l_xml type ref to cl_xml_document . create object l_xml. call method l_xml->parse_string exporting stream = cresponse. "cresponse is the variable which is holding the xml string call method l_xml->display. *************************************************************************************** * * * FORM UTC_DATETIME_ISO_8601 * * * *************************************************************************************** FORM UTC_DATETIME_ISO_8601. DATA: lf_date TYPE dats, lf_time TYPE tims, lf_timestamp TYPE timestampl, lf_tz_utc TYPE timezone VALUE 'UTC', AMZTS TYPE STRING, timechar(32) TYPE c, tempCreatedAfter TYPE STRING, tempCreatedBefore TYPE STRING, AmazonTimestamp TYPE STRING. "Amazon can expect a timestamp from any timezone, but for simplicity's sake, we convert "our own timestamp to Amazon's own timezone (UTC+0) get time stamp field lf_timestamp. convert time stamp lf_timestamp time zone lf_tz_utc into date lf_date time lf_time. MOVE lf_timestamp TO timechar. CONDENSE timechar. AMZDATE = lf_date(4) && lf_date+4(2) && lf_date+6(2) && 'T' && lf_time(2) && lf_time+2(2) && lf_time+4(2) && 'Z'. TRANSLATE amzdate TO UPPER CASE. ENDFORM.

Stephen

[former_member433346]

Hi Stephen,

I am stuck hard time trying to write flat file to AWS server using ABAP.

Could you please send sample code for PUT method.

Thanks.

[Stephen3]

Hi Ahmed

I will look around. I don't think we every used PUT. Just GET. Conceptually it should work once you have successfully logged into AWS. Have you been able to get the "GET" working?

Stephen

[former_member433346]

Hi Stephen,

I am able to write the empty file to AWS server, but not the content.

Any pointers would be highly appreciated.