How to migrate to the Commerce Cloud CCv2 (part 2 - media)

[kai_unewisse]

Many steps are already given in the CXWorks article Migrate to SAP Commerce Cloud: Migrate Media with azcopy.

This article was republished due to a switch of my account.

Getting a SAS Token

Use the public IP that is for the outgoing Proxy. This will ensure that your IP is always the same, since this the provided IP is in whitelist added for the SAS Token. Other option: check your public IP adress that the server is using by accessing e.g. whatismyip.com or similar service.

However you need to setup the proxy before you use azcopy in the bash (and do not forget to replace username, password, proxyhost and port) .

export http_proxy='http://username:password@proxyhost:port/'
export https_proxy='https://username:password@proxyhost:port/'

As of Nov 2023 is the request of a SAS token done differently than before. The old way is mentioned in the azcopy article.
You need to create a Service request (and not any more a Support ticket)  in SAP for Me - Services and Support. Create a new request and select the option "CRM und Customer Experience" , then select / open Category "CX Commerce Cloud" and select "CX Commerce- Migration Projects" . Select the tile "CX Commerce - Creation of Media SAS token". it is required : environment (e.g. search for *-d1) and add the public IP address you got earlier. if the IP address is wrong, the SAS token is useless.

Not required anymore, but maybe helpful....
The storage id of the hot folder is NOT the same for the media storage and the SAS token will not work.
It might result in a authorization issue. The Media Storage Account Name can be found in the HAC of the cloud system in the Property:  media.globalSettings.cloudAzureBlobStorageStrategy.connection

The value is something like:
DefaultEndpointsProtocol=https;AccountName=xx123xx456xxxx78;AccountKey=XXXXXXXX;EndpointSuffix=core.windows.net

The SAS token is given in the ticket like this containing start and end date (see key st and se) of that token validity. Usually it is valid for 14 days.. You will receive an email with the following details:

======SAS Token List ======
st=2022-11-02T00%3A01Z&se=2022-12-30T00%3A01Z&sp=xxxxxx&sip=1.2.3.4&spr=https&sv=2022-03-28&ss=bqtf&srt=sco&sig=<secretKey>
Storage Name: xx123xx456xxxx78
SecretKey:<One Time Secret>

Please checkout the SecretKey and store it at a save place, since you can view it only one time.
Combine then the path (line one starting with "st=") and exchange the value of the last attribute sig with the secretKey you received.
This together with the host "https://[account-name].blob.core.windows.net/[container]" is needed for the next steps.

Download the azcopy executable from Microsoft to the system where you have the media stored. You do NOT need to have an AD account for this, since you are using a token.
Also : Testing this on a different system will not work, since the outgoing IP address is most likely different.

In case azcopy is not working as expected, you can check the connection and the logs.

Run this PowerShell command to test basic internet connectivity from the host. (Note: you can use any public URL in this test)

[System.Net.WebRequest]::Create("http://azcopyvnextrelease.blob.core.windows.net/releasemetadata/latest_version.txt").GetResponse().StatusCode 

If you run that in PowerShell on the host, one of the following things should happen:

1. It prints OK. This is the expected result if you run it on your desktop machine.
2. It fails with an error, that includes the text “(407) Proxy Authentication Required”. Note that this text will be buried inside a larger error message. If this happens, it shows that the traffic is being directed through a proxy that requires authentication. Either the proxy is not used or the current version of AzCopy doesn't work with those proxies.
3. It fails with some other error. That probably suggests that the host cannot reach the internet. In this case, this is not an AzCopy issue but is a network (configuration) issue.
   
   

Using the SAS token

I created an variable (e.g. sas to shorten the links a little, after i checked, if the connection works.

export sas="st=2024-05-15T00%3A01Z&se=2024-05-19T00%3A01Z&sp=rwdlacup&sip=193.41.37.3&spr=https&sv=2024-05-19&ss=bqtf&srt=sco&sig=TZADFFz9%2ByIJlLANPi6dQZTuYYiY9gG2qcnfLiY%3D"

You can use then ${sas} as shown below.
I first sync the sys-master-root and exclude the subfolders (default and also custom ones like "pdfs" or "survey"). Using sync makes it easier when you migrate data with a dry-run.  

Then I move through every folder, however first you need to create the subfolder using 

azcopy make

./azcopy sync "sys_master" "https://blablabla.blob.core.windows.net/sys-master-root/?${sas}" --recursive=true --delete-destination prompt --exclude-path "backofficeexcelimport;hmc;survey;backofficeconfiguration;pdfs;jasperreports;email-body;impex;catalogsync;images;cronjob"
./azcopy sync "sys_master/images" "https://blablabla.blob.core.windows.net/sys-master-images/images?${sas}" --recursive=true --delete-destination prompt --log-level ERROR
./azcopy sync "sys_master/impex" "https://blablabla.blob.core.windows.net/sys-master-impex/impex?${sas}" --recursive=true --log-level ERROR --delete-destination prompt
./azcopy sync "sys_master/cronjob" "https://blablabla.blob.core.windows.net/sys-master-cronjob/cronjob?${sas}" --recursive=true --log-level ERROR --delete-destination prompt
./azcopy sync "sys_master/catalogsync" "https://blablabla.blob.core.windows.net/sys-master-catalogsync/catalogsync?${sas}" --recursive=true --log-level ERROR --delete-destination prompt
./azcopy sync "sys_master/jasperreports" "https://blablabla.blob.core.windows.net/sys-master-jasperreports/jasperreports?${sas}" --recursive=true --log-level ERROR --delete-destination prompt
./azcopy sync "sys_master/survey" "https://blablabla.blob.core.windows.net/sys-master-survey/survey?${sas}" --recursive=true --log-level ERROR --delete-destination prompt
./azcopy sync "sys_master/pdfs" "https://blablabla.blob.core.windows.net/sys-master-pdfs/pdfs?${sas}" --recursive=true --log-level=ERROR --delete-destination prompt