cancel
Showing results for 
Search instead for 
Did you mean: 

Azure Cloud Hotfolder not fetching data with WSL

superpalmi
Explorer
0 Kudos
238

Hi,

I'm trying to connect to my Sap Commerce Cloud Hotfolder using my local machine (WSL) to download some data for my local environment configuration. I've followed this guide https://help.sap.com/docs/SAP_COMMERCE_CLOUD_PUBLIC_CLOUD/403d43bf9c564f5a985913d1fbfbf8d7/4abf9290a...

I created a folder called "myfolder" inside "hybris/master/hotfolder" using Azure Storage Explorer and set these properties:

cluster.node.groups=integration,yHotfolderCandidate
azure.hotfolder.storage.account.connection-string=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=mykey;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1
azure.hotfolder.storage.container.hotfolder=${tenantId}/hotfolder/myfolder


I uploaded the "test.csv" file like stated in the guide but the only log I get on my Hybris instance is this:

 

 

Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:48.979+0100].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:48.979+0100].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:18 CET 2025].
DEBUG [HotfolderLeaderInitiator-junit-1] (junit) [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:18 CET 2025].
DEBUG [HotfolderLeaderInitiator-master-1] [LockRegistryLeaderInitiator] Acquiring the lock for LockContext{role=yHotfolderServices, id=master-0, isLeader=true}
DEBUG [HotfolderLeaderInitiator-master-1] [LockRegistryLeaderInitiator] Acquiring the lock for LockContext{role=yHotfolderServices, id=master-0, isLeader=true}
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:50.944+0100].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-21T17:20:50.944+0100].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:20 CET 2025].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Tue Jan 21 17:21:20 CET 2025].

 

 

 

How do i test if my local machine is connected to the hotfolder correctly? I tried the same with my customer's Sap Commerce Cloud instance hotfolder but i still didn't get to connect correctly

View Entire Topic
mansurarisoy
Contributor

Why are you creating a folder inside "hybris/master/hotfolder"? Does it work when you upload the file under hybris/master/hotfolder folder with the default configuration?

azure.hotfolder.storage.account.connection-string =DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1

And for the clarification, hybris is the container name and inside this container you should create a folder named master and inside of it a new folder named hotfolder.

In Microsoft Azure Storage Explorer, you should see something similar to the following screenshots

hybris containerhybris containerFolders inside hybris containerFolders inside hybris container

superpalmi
Explorer
0 Kudos
Hi, no it does not work even if i point to hybris/master/hotfolder instead of hybris/master/hotfolder/myfolder, the thing is that Hybris does not throw any error or warning even if i'm using wrong credentials, so I don't know what is the source of the problem
mansurarisoy
Contributor
0 Kudos

If the properties you shared are correct, it means that you are providing AccountKey value as mykey. Can you change it to the following and try again?

 

Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==

 

 The final version of the property should be as follow

 

azure.hotfolder.storage.account.connection-string=DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://127.0.0.1:10000/devstoreaccount1;QueueEndpoint=http://127.0.0.1:10001/devstoreaccount1;TableEndpoint=http://127.0.0.1:10002/devstoreaccount1

 

This is because the connection string for Azurite is fixed.

superpalmi
Explorer
0 Kudos
Hi, sorry I accepted as a solution but the problem is not yet solved, of course I tried with the key in the guide but it still didn't work, i replaced it with a base 64 string but it did not solve the issue
mansurarisoy
Contributor
0 Kudos
I tried it on my local with SAP Commerce version 2211.31 and cloudhotfolders-2211azuresdk12-20241205.zip file from SAP Note. When I uploaded test.csv following messages are printed in console and the file moved to error folder.
 
INFO  [AzureIntegrationTaskExecutorThread-master-1] [LoggingMonitorRecorder] HOT_FOLDER:test.csv1737538248000 Status [IN_PROGRESS]
Step [DOWNLOADED] Node [0] Status [SUCCESS] Started [2025-01-22T12:30:59.706+03:00] Ended [2025-01-22T12:30:59.746+03:00] Duration [40]ms Message [Successfully read blob [master/hotfolder/test.csv] of size [108 bytes]] Actions [none]
WARN  [AzureIntegrationTaskExecutorThread-master-1] [LoggingMonitorRecorder] HOT_FOLDER:test.csv1737538248000 Status [WARNING]
Step [DOWNLOADED] Node [0] Status [SUCCESS] Started [2025-01-22T12:30:59.706+03:00] Ended [2025-01-22T12:30:59.746+03:00] Duration [40]ms Message [Successfully read blob [master/hotfolder/test.csv] of size [108 bytes]] Actions [none]
Step [FILE_ROUTED] Node [0] Status [FAILURE] Started [null] Ended [null] Duration [null]ms Message [File [test.csv] modified [1737538248000] was not routed as didn't match any configurations] Actions [none]
Which version of SAP Commerce and hotfolder extensions are you using? And could you try to find log messages during startup stating that something is wrong with hotfolder extensions or setup, if any?
superpalmi
Explorer
0 Kudos
I'm using SAP Commerce 2211.32 and IEP 30, i updated the classes following the guide but I remember hotfolders didn't work even with older versions with this installation. I note that i can't find any log regarding AzureIntegrationUnprocessedFileTaskExecutorThread, so there might be an issue with these threads starting?
mansurarisoy
Contributor
0 Kudos
Strangely, I could not get the logs you shared in your question even if I open DEBUG level logging. I see that junit is enabled in your local, can you try disabling it by setting installed.tenants as empty and restart the server?
superpalmi
Explorer
0 Kudos

The logs in the question were enabled using these properties:

log4j2.logger.azure.name = de.hybris.platform.cloud.azure.hotfolder
log4j2.logger.azure.level = DEBUG
log4j2.logger.azure.appenderRef.stdout.ref = STDOUT
log4j2.logger.cloud.name = de.hybris.platform.cloud.hotfolder
log4j2.logger.cloud.level = DEBUG
log4j2.logger.cloud.appenderRef.stdout.ref = STDOUT
log4j2.logger.cloudcommons.name = de.hybris.platform.cloud.commons
log4j2.logger.cloudcommons.level = DEBUG
log4j2.logger.cloudcommons.appenderRef.stdout.ref = STDOUT
superpalmi
Explorer
0 Kudos
I removed any tenant from installed.tenants and application started without errors, but i'm still not able to see any logs, I also tried with this kba https://me.sap.com/notes/3506703/E but the group id seems correct
mansurarisoy
Contributor
0 Kudos
Just to make sure, is your local instance the only one connected to the database? Or is it a shared database located outside of your local machine? I am asking this because I've seen some strange behaviors on locals using the shared database.
mansurarisoy
Contributor
0 Kudos

After enabling the same loggers, I've seen the following logs:

DEBUG [AzureIntegrationTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Preparing thread AzureIntegrationTaskExecutorThread-master-1.
DEBUG [AzureIntegrationUnprocessedFileTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Preparing thread AzureIntegrationUnprocessedFileTaskExecutorThread-master-1.
DEBUG [AzureIntegrationTaskExecutorThread-master-1] [AzureBlobSession] Listing contents of container [hybris] on path [master/hotfolder].
DEBUG [AzureIntegrationUnprocessedFileTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Unprepared thread AzureIntegrationUnprocessedFileTaskExecutorThread-master-1.
DEBUG [AzureIntegrationTaskExecutorThread-master-1] [AzureBlobInboundSynchronizer] 0 files transferred from 'master/hotfolder'
DEBUG [AzureIntegrationTaskExecutorThread-master-1] [HybrisAwareThreadPoolExecutor] Unprepared thread AzureIntegrationTaskExecutorThread-master-1.
DEBUG [HotfolderLeaderInitiator-master-1] [LockRegistryLeaderInitiator] Acquiring the lock for LockContext{role=yHotfolderServices, id=master-0, isLeader=true}
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Acquiring lock for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] for node [0].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Deleting lock records for key [7c67706a-d0d1-3e8a-b38e-1693a2906c22] in region [yCloudHotfolders] older than [2025-01-23T09:25:34.201+0300].
DEBUG [HotfolderLeaderInitiator-master-1] [HybrisLockRepository] Updating record for lock [lockKey=7c67706a-d0d1-3e8a-b38e-1693a2906c22,region=yCloudHotfolders,clusterId=0,timestamp=Thu Jan 23 09:26:04 TRT 2025].

I think the classes starting with Azure do not print log in your local. There might be some misconfiguration in Spring.

Can you try the file uploaded yesterday to note (https://me.sap.com/notes/0002817992)? File name is cloudhotfolders-2211azuresdk12-SNAPSHOT-202501101831836.zip

superpalmi
Explorer
Hi, finally I ended up reinstalling the SAP upgrade along with extensions and the azure hotfolder started working correctly, maybe I did something wrong during the upgrade process but I didn't figure out what, thank you very much for your support!