Question Re: hdbcli cursor usage with Multiprocessing in Python in Technology Q&A

hdbcli cursor usage with Multiprocessing in Python

ailang — Mon, 16 Aug 2021 22:47:37 GMT

I'm using the hdbcli to connect to the HANA DB and was able to get the result set, but I can't seem to use the result set directly in the enumeration in Python, rather I have to run the result set via the loop and then use each row.

This means I have to use an additional loop for post-HANA read and write operations.

If I do use it I'm getting the error "cannot pickle 'pyhdbcli.ResultRow' object", let me know if you have any inputs.

Re: hdbcli cursor usage with Multiprocessing in Python

Vitaliy-R — Thu, 19 Aug 2021 15:28:30 GMT

Could you please share the code you are using to make sure others get the complete picture?

Is it what you are looking for, or something else?

cursor = conn.cursor()
sql_command = "select * from M_TABLES"
cursor.execute(sql_command)
rows = cursor.fetchall()
for row in rows:
    print(row)
cursor.close()

Re: hdbcli cursor usage with Multiprocessing in Python

ailang — Thu, 19 Aug 2021 16:34:53 GMT

Thank you vitaliy.rudnytskiy for your response.

This is the code I'm currently using, as you can see we'll have to loop through each record to process it.

If I use the same code with Python enumeration like below throws the cannot pickle 'pyhdbcli.ResultRow' object

resultset = pool.apply_async( function_name, row for row in rows )

Re: hdbcli cursor usage with Multiprocessing in Python

Vitaliy-R — Thu, 19 Aug 2021 20:39:24 GMT

Could you try

resultset =pool.apply_async( function_name, row.column_values for row in rows)

`row.column_values` should give you a tuple of values from the row, instead of an object of `pyhdbcli.ResultRow` data type.

Re: hdbcli cursor usage with Multiprocessing in Python

ailang — Mon, 23 Aug 2021 04:45:45 GMT

Thank you Witalij, I was able to move forward.