Azure Cloud Hotfolder not fetching data with WSL

[superpalmi]

Hi,

I'm trying to connect to my Sap Commerce Cloud Hotfolder using my local machine (WSL) to download some data for my local environment configuration. I've followed this guide https://help.sap.com/docs/SAP_COMMERCE_CLOUD_PUBLIC_CLOUD/403d43bf9c564f5a985913d1fbfbf8d7/4abf9290a...: 

I created a folder called "myfolder" inside "hybris/master/hotfolder" using Azure Storage Explorer and set these properties:

cluster.node.groups=integration,yHotfolderCandidate
azure.hotfolder.storage.account.connection-string=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=mykey;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1
azure.hotfolder.storage.container.hotfolder=${tenantId}/hotfolder/myfolder

I uploaded the "test.csv" file like stated in the guide but the only log I get on my Hybris instance is this:

Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:48.979+0100].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:48.979+0100].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:18 CET 2025].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:18 CET 2025].
DEBUG [HotfolderLeaderInitiator-master-1] [LockRegistryLeaderInitiator] Acquiring the lock for LockContext{role=yHotfolderServices, id=master-0, isLeader=true}
DEBUG [HotfolderLeaderInitiator-master-1] [LockRegistryLeaderInitiator] Acquiring the lock for LockContext{role=yHotfolderServices, id=master-0, isLeader=true}
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:50.944+0100].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:50.944+0100].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:20 CET 2025].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:20 CET 2025].

How do i test if my local machine is connected to the hotfolder correctly? I tried the same with my customer's Sap Commerce Cloud instance hotfolder but i still didn't get to connect correctly

[mansurarisoy]

Wrong place to comment 🙂 Updating the conversation below...

[mansurarisoy · Accepted solution]

Why are you creating a folder inside "hybris/master/hotfolder"? Does it work when you upload the file under hybris/master/hotfolder folder with the default configuration?

azure.hotfolder.storage.account.connection-string =DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1

And for the clarification, hybris is the container name and inside this container you should create a folder named master and inside of it a new folder named hotfolder.

In Microsoft Azure Storage Explorer, you should see something similar to the following screenshots

hybris containerFolders inside hybris container

[superpalmi · Accepted solution]

Hi, no it does not work even if i point to hybris/master/hotfolder instead of hybris/master/hotfolder/myfolder, the thing is that Hybris does not throw any error or warning even if i'm using wrong credentials, so I don't know what is the source of the problem

[mansurarisoy · Accepted solution]

If the properties you shared are correct, it means that you are providing AccountKey value as mykey. Can you change it to the following and try again?

Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==

 The final version of the property should be as follow

azure.hotfolder.storage.account.connection-string=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1

This is because the connection string for Azurite is fixed.

[superpalmi · Accepted solution]

Hi, sorry I accepted as a solution but the problem is not yet solved, of course I tried with the key in the guide but it still didn't work, i replaced it with a base 64 string but it did not solve the issue

[mansurarisoy · Accepted solution]

I tried it on my local with SAP Commerce version 2211.31 and cloudhotfolders-2211azuresdk12-20241205.zip file from SAP Note. When I uploaded test.csv following messages are printed in console and the file moved to error folder.

INFO  [AzureIntegrationTaskExecutorThread-master-1] [LoggingMonitorRecorder] HOT_FOLDER:test.csv1737538248000 Status [IN_PROGRESS]

Step [DOWNLOADED] Node [0] Status [SUCCESS] Started [2025-01-22T12:30:59.706+03:00] Ended [2025-01-22T12:30:59.746+03:00] Duration [40]ms Message [Successfully read blob [master/hotfolder/test.csv] of size [108 bytes]] Actions [none]

WARN  [AzureIntegrationTaskExecutorThread-master-1] [LoggingMonitorRecorder] HOT_FOLDER:test.csv1737538248000 Status [WARNING]

Step [DOWNLOADED] Node [0] Status [SUCCESS] Started [2025-01-22T12:30:59.706+03:00] Ended [2025-01-22T12:30:59.746+03:00] Duration [40]ms Message [Successfully read blob [master/hotfolder/test.csv] of size [108 bytes]] Actions [none]

Step [FILE_ROUTED] Node [0] Status [FAILURE] Started [null] Ended [null] Duration [null]ms Message [File [test.csv] modified [1737538248000] was not routed as didn't match any configurations] Actions [none]

Which version of SAP Commerce and hotfolder extensions are you using? And could you try to find log messages during startup stating that something is wrong with hotfolder extensions or setup, if any?

[superpalmi · Accepted solution]

I'm using SAP Commerce 2211.32 and IEP 30, i updated the classes following the guide but I remember hotfolders didn't work even with older versions with this installation. I note that i can't find any log regarding AzureIntegrationUnprocessedFileTaskExecutorThread, so there might be an issue with these threads starting?

[mansurarisoy · Accepted solution]

Strangely, I could not get the logs you shared in your question even if I open DEBUG level logging. I see that junit is enabled in your local, can you try disabling it by setting installed.tenants as empty and restart the server?

[superpalmi · Accepted solution]

The logs in the question were enabled using these properties:

log4j2.logger.azure.name = de.hybris.platform.cloud.azure.hotfolder
log4j2.logger.azure.level = DEBUG
log4j2.logger.azure.appenderRef.stdout.ref = STDOUT
log4j2.logger.cloud.name = de.hybris.platform.cloud.hotfolder
log4j2.logger.cloud.level = DEBUG
log4j2.logger.cloud.appenderRef.stdout.ref = STDOUT
log4j2.logger.cloudcommons.name = de.hybris.platform.cloud.commons
log4j2.logger.cloudcommons.level = DEBUG
log4j2.logger.cloudcommons.appenderRef.stdout.ref = STDOUT

[superpalmi · Accepted solution]

I removed any tenant from installed.tenants and application started without errors, but i'm still not able to see any logs, I also tried with this kba https://me.sap.com/notes/3506703/E but the group id seems correct

[mansurarisoy · Accepted solution]

Just to make sure, is your local instance the only one connected to the database? Or is it a shared database located outside of your local machine? I am asking this because I've seen some strange behaviors on locals using the shared database.

[mansurarisoy · Accepted solution]

After enabling the same loggers, I've seen the following logs:

DEBUG [AzureIntegrationTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Preparing thread AzureIntegrationTaskExecutorThread-master-1.
DEBUG [AzureIntegrationUnprocessedFileTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Preparing thread AzureIntegrationUnprocessedFileTaskExecutorThread-master-1.
DEBUG [AzureIntegrationTaskExecutorThread-master-1] [AzureBlobSession] Listing contents of container [hybris] on path [master/hotfolder].
DEBUG [AzureIntegrationUnprocessedFileTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Unprepared thread AzureIntegrationUnprocessedFileTaskExecutorThread-master-1.
DEBUG [AzureIntegrationTaskExecutorThread-master-1] [AzureBlobInboundSynchronizer] 0 files transferred from 'master/hotfolder'
DEBUG [AzureIntegrationTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Unprepared thread AzureIntegrationTaskExecutorThread-master-1.
DEBUG [HotfolderLeaderInitiator-master-1] [LockRegistryLeaderInitiator] Acquiring the lock for LockContext{role=yHotfolderServices, id=master-0, isLeader=true}
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-23T09:25:34.201+0300].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Thu Jan 23 09:26:04 TRT 2025].

I think the classes starting with Azure do not print log in your local. There might be some misconfiguration in Spring.

Can you try the file uploaded yesterday to note (https://me.sap.com/notes/0002817992)? File name is cloudhotfolders-2211azuresdk12-SNAPSHOT-202501101831836.zip

[superpalmi · Accepted solution]

Hi, finally I ended up reinstalling the SAP upgrade along with extensions and the azure hotfolder started working correctly, maybe I did something wrong during the upgrade process but I didn't figure out what, thank you very much for your support!