Question Re: Consume s3 Connections property inside Python3 operator in Technology Q&A

Consume s3 Connections property inside Python3 operator

former_member730080 — Mon, 06 Dec 2021 05:23:50 GMT

Hii,

I want to consume the properties/Parameter values which is already there in the connection management specifically for S3 bucket Access keys and secret access key.

I saw somewhere that we can do the same for HanaDB using api.config (Something like this).

The Use is when putting the objects inside s3, We have to provide Tags to those objects. Now I am currently doing it by hardcoding those values inside Python operator and and using Boto3 for the process. Now this is not recommended in further developments.

So If there is a way to consume those values from the connection that we have already made in Connection management for the same S3 bucket, then there won't be need of hardcoding values.

Hoping to hear some suggestions from the community!

Thanks,

Samarth

Re: Consume s3 Connections property inside Python3 operator

andy_yin — Mon, 06 Dec 2021 05:49:01 GMT

Hi Samarth,

This link Managed Connections in Script Operators maybe useful to you.

Re: Consume s3 Connections property inside Python3 operator

former_member730080 — Mon, 06 Dec 2021 13:05:26 GMT

Hey Andy,

Thanks for a showing direction.

I followed all steps showing in that link but i changed the Connection type from OPENAPI to S3.

I am monitoring the output at the wiretap but its showing no response.(No output)

Any idea what is the reason?

Thanks,

Samarth

Re: Consume s3 Connections property inside Python3 operator

andy_yin — Tue, 07 Dec 2021 03:59:06 GMT

Samarth,

Check and verify if you S3 connection is in the returned list from https://YOUR_DI_INSTANCE/app/datahub-app-connection/connections?connectionTypes=s3.
Check Status in connection manager to verify if your S3 connection is OK.
Try to change the code host = managedConnectionProperties['host'] to host = managedConnectionProperties['endpoint'] if you did not do this previously.

Hope this helpful.