M-series Apple Chip MacBooks and Abap Platform Trial containers using Docker and Podman

[dylan-drummond]

Intro 

SAP offer a container image called Abap Platform Trial which enables you to run a HANA-based SAP System on your laptop or PC:

https://hub.docker.com/r/sapse/abap-platform-trial 

That site also provides the official advice from SAP on how to use Docker container-management software to get the container up and running on host machines with an amd64 architecture (a.k.a. “x86_64” or “Intel” architecture). 

Nowadays many developers and system admins (the likely users of Abap Platform Trial) are using M-series MacBooks, i.e. MacBooks with the “Apple Chips” which have an arm64 architecture and are named according to the chip version {M1, M2, M3, …}. But Abap Platform Trial is at the time of writing only available as an amd64 image, so this means that to run it on an M-series MacBook, some form of emulation is required - so this blog will describe two paths to spin up and run Abap Platform Trial containers on an M-series MacBook - BTW here is what success looks like:

The two paths are: 

Path 1: use Docker Desktop with the feature released in mid-2023 to enable Apple Rosetta emulation - AFAIK this path is only available with Docker Desktop.

Path 2: use the UTM app with appropriate configurations to emulate via Apple QEMU an amd64 Linux distro  - this path works with Docker and Podman, and (I guess) quite possibly with other container-management tools.

Disclaimer: The demos of this blog, as well as writing the actual blog, were done in my spare time, on my own MacBook - there really is no SAP or other Helpdesk available if you get stuck trying to use this blog’s advice, it’s all about BYODS: Bring Your Own Debug Skills. But if there are errors in the text, feel free to suggest corrections. I did the demos on an M3 MacBook with macOS (Sonoma 14.2.1), but I guess the examples should work on other M-series MacBooks with a suitable macOS version. Also note that officially the host machine should have 32 GB of RAM available - maybe you can get by with less than that amount, though I haven’t tested that (as my machine has well over 32 GB RAM). 

Using Docker Desktop for M-series MacBooks

We download Docker Desktop:

https://www.docker.com/products/docker-desktop/ 

Our DMG will be the Docker Desktop for macOS (“Apple chip” i.e. M-series chip); from the DMG you install Docker Desktop to /Applications directory.

When you open Docker Desktop for the first time, you have to accept the Docker Subscription Service Agreement… Obviously if you are using Docker Desktop for work purposes of whatever organisation you belong to, you have to work out between you and your employer etc whether you need a paid subscription (not a consideration for me on my home computer doing this in my free time).

Next we choose to “Use recommended settings (requires password)” for the final setup steps (where the actual Docker Engine is installed to do the hard work behind the scenes of the Docker Desktop GUI stuff), and press “Finish”. then supply your computer user’s admin password when prompted. 

Next step is a choice between Signing Up for a Docker Hub account, or Signing In if you already have such an account, or Continue without signing in. Well, although SAP used to offer the predecessor product “Abap Trial” only via login to Docker Hub, however for the current product Abap Platform Trial, no such login to Docker Hub is required (the SAP website for that image claims that such a login is required, but at the time of writing that claim is false 😀).

So I’m not going to bother signing in to Docker Hub (even though I have had an account there since 2015); instead we choose “Continue without signing in” for now (you can always create an account and/or sign in later if you need to). Then if you like you can tell Docker about yourself in the screen “Tell us about the work you do”, though I just pressed Skip, which takes us to the main Docker Desktop window where all the icons for Images, Containers etc are situated. From there near top right we can go to the Settings screen using the cog icon, in the tab “General” we find 3 important settings, which were switched on by default, so no changes needed but it is good to review these settings a bit:

Zoom in on those 3 a bit:

The first one is “Use Virtualisation framework”, which is greyed out (unmodifiable) because it has to be switched on when using the second setting “Choose file sharing…. VirtioFS”. We keep this as is. Also we keep the third setting “Use Rosetta for x86/amd64 emulation on Apple Silicon” as is, because this is exactly the feature that appeared in 2023 and enabled people to finally succeed in using Docker Desktop for Abap Platform Trial (which is an amd64-only image).

FYI Docker, like Podman or any other Linux Container Management tools, can only work on non-Linux computers by running a Linux VM in the background - it is inside this Linux VM where the actual containers are run. We go to the Settings tab “Resources” where basically what we can specify are the main parameters of this Linux VM (for Docker the VM is based on LinuxKit); we will need a bigger and faster VM than what is provided by default, so let’s adjust the Resources settings:

CPU limit: 6 should be enough

Memory limit: 28 GB. (You might get away with less than that, I am just saying what worked for me).

Swap limit: 2 GB

Virtual disk limit: 248 GB (the nearest selectable value to 250 GB I found on the slider). Probably you can get by with a slightly smaller disk size, I am just telling yous what works for me, but remember kids, the image takes up over 50GB on disk, and then you want at least 120 GB for the container that contains a whole HANA-based SAP System, plus the Linux VM’s OS footprint takes up some space as well.

Resource Saver [not shown in screenshot below]: Keep the default settings (Enabled, etc)

Then press “Apply & restart” to save the changes and restart Docker Engine.

Enable Rosetta on MacBook

If you haven’t already enabled Apple Rosetta, or you are unsure, we can make sure that it is enabled by, well, enabling it (make sure you are connected to the Internet as the software is downloaded from Apple’s servers):

sudo softwareupdate --install-rosetta --agree-to-license

You can ignore a warning about the installKbytes attribute, instead we trust the message that says Rosetta 2 was installed successfully; you can also open the System Report, Software, Installations and scroll down till you find an entry including the Rosetta AutoUpdater package “RosettaUpdateAuto” (which was installed as part of Rosetta):

Done, now we can get the image we need and spin up the container…

Pull and spin up Abap Platform Trial

There are two ways to do this: using the command line (i.e. using Docker Engine, which you installed as part of Docker Desktop setup), or using the GUI app (Docker Desktop, which in the background will be using Docker Engine anyway). The command line method is:

sudo docker pull sapse/abap-platform-trial:1909_SP01

The GUI app method is to search in the top bar of Docker Desktop for the image, then press Pull:

The pull operation will take some time because there is over 20 GB of stuff to download, then it all has to be extracted and written to the local image repo of the Docker VM, so you just need to be patient. If the process fails, then start it again (either from GUI app or command line). Eventually you should succeed and the image will be viewable in Docker Desktop’s Images tab and via the command:

sudo docker images

Docker Desktop warns us that this is an amd64 image:

Since the image is only available as amd64, and since we have the relevant emulation settings configured via Docker Desktop, we remain confident that this is going to work out fine.

Next task is to spin up the a4h container, I am going to do this from the command line as that way we easily get to specify all the options we want and we can immediately follow the stdout info of the command’s progress… make sure Docker Desktop is running (as that starts up Docker Engine for us)… then the command I am going with is:

sudo docker run --stop-timeout 3600 -it --name a4h -h vhcala4hci -p 3200:3200 -p 3300:3300 -p 3601:3601 -p 8443:8443 -p 30213:30213 -p 50000:50000 -p 50001:50001 sapse/abap-platform-trial:1909_SP01 -skip-limits-check -agree-to-sap-license

As well as following the Terminal window output, you can also follow the progress in Docker Desktop:

Click on a4h and then the Logs tab shows the same stdout info. 

Anyway, the SAP System is soon up and running, and because we are using Rosetta instead of a fully emulated VM, the procedure goes much faster, less than 3 minutes from ‘docker run’ to the SAP System up and running.

[We note this warning that occurred at the start of the docker run operation: “The requested image’s platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested”. But as this is only a warning, not an error, and as the catch-and-translate tasks are all handled fine with no specific errors appearing in stdout, then this warning isn’t something we need to worry about.]

We do a test of opening a SAPGUI session - note that for this test, where we typically want to use “vhcala4hci” as the hostname in the SAPGUI for Java Connection Item, i.e.:

/H/vhcala4hci/S/3200

…so we need to add a line (plus optionally an explanatory comment) to MacBook /etc/hosts file, for example:

# For accessing Abap Platform Trial container on localhost:

127.0.0.1       vhcala4hci  vhcala4hci.dummy.nodomain

You can modify the file from Terminal with:

sudo pico /etc/hosts

Use <Ctrl-0>, <Enter>, <Ctrl-X> to save and exit.

[Note that in this Docker for macOS case the container’s ports are accessed from localhost (i.e. the MacBook-as-host-itself), which has the IPv4 address 127.0.0.1 - remember kids, don’t touch the existing entry for localhost in /etc/hosts, especially don’t remove it; instead it’s perfectly ok to make a separate new entry mapping vhcala4hci to 127.0.0.1 - mapping multiple names to one IPv4 address is ok].

So the SAPGUI session opens just fine:

Then exit SAPGUI, and shutdown the SAP System using <Ctrl-C> in Terminal session or using the Stop button for the container in Docker Desktop… it may take up to 10 minutes, the last steps involve telling Rosetta to let go of various HDB processes it is handling.

To restart the container, you can do that either from Docker Desktop or (assuming Docker Engine is running):

sudo docker start -ai a4h

Configuring an emulated amd64 Ubuntu Server VM for M-series MacBooks

Ubuntu Server can be downloaded from here:

https://ubuntu.com/download/server 

Choose the latest LTS (Long Term Stable) release offered there.

You can get the UTM Hypervisor-Emulator here:

https://mac.getutm.app/ 

I have the paid version from App Store, as it wasn’t that expensive (one-time payment I guess) and you then get UTM updates pushed instead of having to update it yourself. 

Create a new VM. Choose Emulation. Choose Linux. Select the Ubuntu ISO file. Architecture: x86_64 (a.k.a. amd64), System: Standard PC (Q35+ICH9+2009). 

Memory, well, let’s try giving it 28GB of RAM, which is equivalent to 28672 MB (binary) according to our calculator:

https://www.gbmb.org/gb-to-mb 

So we enter 28672 as the MB value of RAM. 

Processors (processor cores, is the more accurate term maybe): 6 should be enough.

Leave the OpenGL checkbox unchecked.

Disk size: 250 GB.

No shared directory needed. Give the VM a name. Then go to VM Settings… System tab… click on the selection icon for CPU… there are two Intel Core i7 processors available, we will take the “Nehalem-v2” option as it contains the IBRS update which mitigates the Spectre vulnerability, otherwise it is the same as the “Nehalem-v1” option (which you can also use, if you like).

We check to "on" the “Force multicore” checkbox. For the JIT Cache value, I put that to 3 GB (3072 MB), so that the VM has access to slightly less than 32 GB RAM (28+3=31). So we end up with this:

Start the VM and install Ubuntu Server, using these options… Language: English (default, i.e. English (US)). Keyboard layout: whatever matches your keyboard e.g. Finnish, Finnish (Macintosh). Update to the new installer. Choose the “Ubuntu Server” default option. Accept whatever IPv4 address is offered by Hypervisor’s NAT. Let the default mirror location get tested ok. Use the entire disk but uncheck the option to create the disk as an LVM group. 

Give the server this name: 

vhcala4hci

(According to the SAP advice, that is the only hostname of the server that the Abap Platform Trial will work with).

Skip the Ubuntu Pro option, not needed. Check to “on” the option “Install OpenSSH Server” as it’s very convenient to be able to SSH into this server. Do NOT choose any of the Server Snaps offered (especially do not choose the Docker snap). 

Ignore the “Install complete!” message in the screen that logs the progress of the installation, as we want to also let the workflow install any updates it wants e.g. security updates. Instead, wait until in the lower part of the screen the option to “Reboot now” appears:

Modify /etc/hosts of the MacBook to map the Ubuntu Server to its IPv4 address, replacing the example IPv4 address with the IPv4 address of your Ubuntu VM:

sudo pico /etc/hosts

# For the Ubuntu Server that runs Abap Platform Trial:

192.168.64.2  vhcala4hci  vhcala4hci.dummy.nodomain

<Ctrl-O>, <Enter>, <Ctrl-X> to save and exit. (On MacBook, it’s the key called “control”, not the one called “command”). Note that if you earlier already setup Docker Desktop (first part of this blog) and thus already had a mapping in /etc/hosts for vhcala4hci to 127.0.0.1, you should delete that mapping or comment it out temporarily, as you cannot simultaneously map many IPv4 addresses to the one hostname.

Reboot and either login using the UTM console, or if you prefer SSH (good for copy-pasting commands), then login from the MacBook Terminal - for example, if your Ubuntu user is called “marmot”:

ssh marmot@vhcala4hci

Then we update the repos and upgrade the packages:

sudo apt update
sudo apt upgrade -y

For Ubuntu: Comment out the “127.0.1.1 vhcala4hci” line from /etc/hosts - this is needed to prevent the SAP system getting confused.

sudo nano /etc/hosts

#Comment out as it can confuse SAP

#127.0.1.1 vhcala4hci

After that you can choose to either install Docker or install Podman - I recommend not having both utilities installed on the same VM at the same time, as that may cause problems; instead I installed them on separate VMs in the demos described below.

Abap Platform Trial with Docker on the Ubuntu Server VM

Install Docker, using the official advice here (modified that advice by replacing “apt-get” commands with “apt”, as it’s 2024 and “apt” has been the standard for many years now on Ubuntu):

https://docs.docker.com/engine/install/ubuntu/ 

# Add Docker's official GPG key:
sudo apt update
sudo apt install ca-certificates curl gnupg
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg

# Add the repository to Apt sources:
echo \
  "deb [arch="$(dpkg --print-architecture)" signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu \
  "$(. /etc/os-release && echo "$VERSION_CODENAME")" stable" | \
  sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update

sudo apt install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

sudo docker run hello-world

Assuming this all goes smoothly, the hello-world container runs, prints a greeting, and exits. You can check the version installed, if you like:

docker --version

Maybe reboot just to make sure all the processes are using the newest versions of everything. Then we can pull the image, which in late January 2024 is version 1909_SP01:

sudo docker pull sapse/abap-platform-trial:1909_SP01

Takes a while, as 20 GB of stuff needs to get downloaded, extracted and written to docker’s local repo. Once done, we can check the image is there:

sudo docker images

Ok, now we want to run it. To minimise warnings about limits (although we will include the option to prevent limits-checking from resulting in a fatal error, still the warnings will be given if something not configured right), we will first set two sysctl options statically:

sudo sysctl vm.max_map_count=2147483647
sudo sysctl fs.aio-max-nr=18446744073709551615

Then we try the docker run command, ours is this one (the --sysctl and --ulimit parameters are optional):

sudo docker run --stop-timeout 3600 -it --name a4h -h vhcala4hci -p 3200:3200 -p 3300:3300 -p 3601:3601 -p 8443:8443 -p 30213:30213 -p 50000:50000 -p 50001:50001 --sysctl kernel.shmmni=32768 --ulimit nofile=1048576:1048576 sapse/abap-platform-trial:1909_SP01 -skip-limits-check -agree-to-sap-license

Due to the full emulation, the SAP System may take about 15 minutes to get up and running. Once it is running, you can do the SAPGUI session test mentioned above in the Docker Desktop section. Then you can use the <Ctrl-C> option in the Ubuntu terminal session to shutdown the SAP System and container. To restart, use:

sudo docker start -ai a4h

Abap Platform Trial with Podman on the Ubuntu Server VM

Install Podman:

sudo apt install podman

Test with hello-podman:

podman pull quay.io/podman/hello
podman run quay.io/podman/hello

Note that by default, podman doesn’t need to be run as the sudo (root) user. Now we can pull the Abap Platform Trial image:

podman pull docker.io/sapse/abap-platform-trial:1909_SP01

Compared to docker pull, the podman pull is more prone to giving up and/or ending in error during the download of a large image. The only solution is to retry the pull operation until it works. In my testing, about 10% of the time a retry was needed. After pulling the image, it can be seen in the local repo:

podman image list

Then we set the sysctl parameters (optional) and spin up the container (the --sysctl and --ulimit parameters are also optional):

sudo sysctl vm.max_map_count=2147483647
sudo sysctl fs.aio-max-nr=18446744073709551615

podman run --stop-timeout 3600 -it --name a4h -h vhcala4hci -p 3200:3200 -p 3300:3300 -p 3601:3601 -p 8443:8443 -p 30213:30213 -p 50000:50000 -p 50001:50001 --ulimit nofile=1048576:1048576 sapse/abap-platform-trial:1909_SP01 -skip-limits-check -agree-to-sap-license

This takes around 10-15 minutes to get to the success state:

We check we can get to the SAPGUI logon screen… ok. Then exit from SAPGUI, and shutdown the container from console or SSH session using <Ctrl-C>. To restart the container, use:

podman start -ai a4h

Closing remarks

The first path (Docker Desktop) has the advantage that installing and using the SAP System on the container works faster, this is due to Apple Rosetta only emulating the calls that require emulation (whereas in the case of amd64 Ubuntu Server, we emulate the whole VM). It also involves less setup work. 

The second path (emulated Ubuntu Server) has the advantage that you are not restricted to Docker as the only viable container-manager; also, you don’t need to install any container-management software directly on your MacBook (as that is installed on the Linux VM).

In this blog we don’t talk much about what to do after spinning up the container successfully - if you need advice on those post-spinup-tasks, like logging in to the SAP System, getting an A4H license etcetera, there is the official advice from SAP on the Abap Platform Trial site: https://hub.docker.com/r/sapse/abap-platform-trial  

Also you can find similar advice in my January 2024 blog for using Podman for Abap Platform Trial with amd64 hosts (sections “Configuring the A4H System” and “Stopping and starting the container”): https://community.sap.com/t5/devops-and-system-administration-blogs/using-podman-for-abap-platform-t...

Good luck now with whichever path to Abap Platform Trial container glory you take.

[dylan-drummond]

The blog-software has a new option for code-blocks called Bash+Shell, so I edited the blog to use that, introduces a bit of colour into the command lines.

[TomHoepping]

@dylan-drummondDo you have problems with Docker Desktop 4.29 and Sonoma 14.4.1?
On my device, the docker image does not start or is always at HDB Starting.
I see various alerts in the trace files, primarily that the HDB daemon is not running.
I am currently trying to find out what could be the cause. Or whether it is a local problem on my device?

[dylan-drummond]

Hi @TomHoepping  ... well, I don't have such problems... because I have never used that combination of Docker Desktop and macOS version... when I wrote the blog, I had Docker Desktop 4.27 and macOS version Sonoma 14.2.1, on an M3 MacBook.

When I have some free time, I could upgrade Docker Desktop (macOS on my device has already been upgraded to latest Sonoma) and then try my own blog advice and see what happens. Depends on when I have the free time. Meanwhile some questions for you...

which MacBook are you using i.e. M1, M2, or M3?

Which Docker Engine version (use "docker --version" command to verify)?

Did you get this working with older Docker Desktop versions and/or older macOS versions, or have you never got this working?

How much RAM does your machine have available and what Memory Limit did you assign in Docker Desktop settings? (I used 28 GB in the blog demo).

What size of Virtual disk did you assign in Docker Desktop settings? (I used 248 GB as in the past I had issues with virtual disk not being large enough).

Then one more question - based on the error you are getting, it looks as if the needed emulation is somehow not working... can you check that Rosetta 2 is installed ok:

sudo softwareupdate --install-rosetta --agree-to-license

...and that the relevant Docker Desktop settings are set correctly ("Use Virtualization framework" checked, "VirtioFS" checked, "Use Rosetta for x86/amd64 emulation on Apple Silicon" checked? The reason I ask, is that sometime in 2023 I got similar error of "HDB daemon not running" when Docker Desktop at the time did not have the options to use Rosetta properly.

If you can provide those details, then I'll have a look sometime as to whether everything still works for me or whether I can replicate your error... and if I can replicate the error, I would then see whether I can fix it or not.

[dylan-drummond]

@TomHoepping  I posted a reply asking for more details, but then when I edited it slighly it seems to have disappeared... if it doesn't return I will make a new reply tomorrow.

[TomHoepping]

Hi Dylan @dylan-drummond . Thx for your reply and help to investigate.  

So details are the following: 

• M1 MAX / 64GB RAM -> 48GB assigned to docker, SWAP File is 4GB and vDisk-Limit is 320GB.
• OS Version is 14.4.1 -> not tested to downgrade until now, maybe during the weekend 
• Docker Version is 4.29 -> Downgrade to 4.26.1 and 4.25.1 and 4.27.2. Same behaviour. Not Starting
  • My conclusion here is, that I've either an individual problem on my device or the update to 14.4 has changed something regarding rosetta runtime.
  • Also I'm not sure, when I've used the image last time, but if I remember right, with 14.2 it was working   
• I've cleaned up rosetta, there are some tutorials, where the files are just deleted and then MAC OS requests a fresh install -> basically no changes

Error Messages, I've found in several log files of index server / name server are more or less the same: 

I've also found some discussion about errors of rosetta/sonoma 14.4 and docker in other contexts which support the hypothesis that something has changed with 14.4 and that I do not have an individual problem but not sure about that. Eg.
https://github.com/docker/for-mac/issues/7220
https://forums.developer.apple.com/forums/thread/746618
https://github.com/docker/for-mac/issues/7180
https://news.ycombinator.com/item?id=39741115

I will try a clean install of docker in the mean while, just to geht sure. 

Thx for you help again, lets see what we've find out together, Tom  

[dylan-drummond]

@TomHoepping mmm, yeh, as you mention and as the various other issue-posts hint, it sounds like Docker and Rosetta are not working together properly for the Sonoma 14.4 for emulating amd64 images in general. So then it's unlikely to be something specific to the SAP image or the hdbdaemon or otherwise to your local device.

I am not sure it is worth your time downgrading your macOS version just to get this mini-SAP instance up and running using Docker Desktop, but if you do that experiment then let us know how it goes.

Have you tried out using the Ubuntu VM approach with Qemu and Docker or Podman (i.e. not using Docker Desktop, but using the approach decribed above in this blog under the heading "Configuring an emulated amd64 Ubuntu Server VM for M-series MacBooks")? If this approach still works under Sonoma 14.4, then it's somewhat moot whether or not Docker Desktop approach is working, since the goal of the two approaches is the same - to have a usable mini-SAP instance. Anyway the VM approach would be what I would try first, before any thoughts of downgrading the macOS version.

Need to now switch attention to the football... if I get round to actually doing some tests instead of today's armchair analysis I'll keep you informed 😀

[TomHoepping]

@dylan-drummond  Yeah, you're right. I will try the approach with an UTM/Quemu VM and Podman. Did that already last-year, was painful slow during startup, but it worked. I'll give an update the next days. 

[dylan-drummond]

@TomHoepping Hi,

ok, I did two tests today. In the first test, I upgraded Docker Desktop and Docker Engine to latest versions, and pulled the new SAP Cloud Developer Trial 2022 which in effect is the latest version of Abap Platform Trial, only with a new name and download site, here is the post introducing the new version, and then the site for the actual image:

https://community.sap.com/t5/technology-blogs-by-sap/abap-cloud-developer-trial-2022-available-now/b...

https://hub.docker.com/r/sapse/abap-cloud-developer-trial/tags

Well, using Docker Desktop, and as expected on Sonoma 14.4, starting the container always hangs due to HDBdaemon not starting. So currently, we can say that we cannot use Docker Desktop path to glory until Docker and/or Apple manage to allow Docker to handle amd64 containers via Rosetta and VirtioFS etc.

The second test, was to create an emulated amd64 Linux VM in UTM as described in my blog above, and then I chose to use Podman (one of the two options, Docker being the other) as the container management software inside the Linux VM. Once podman was installed, i pulled the image (the SAP Cloud Developer Trial image):

podman pull docker.io/sapse/abap-cloud-developer-trial:ABAPTRIAL_2022

This step always takes a long time, as emulated software does slower processing including network I/O, and even after the download is complete, there is a long wait for the image to be written to the local repo by Podman. Anyway once that step was complete, I set a couple of parameters:

sudo sysctl vm.max_map_count=2147483647

sudo sysctl fs.aio-max-nr=18446744073709551615

And then I used the following command to spin up into life the a4h container:

podman run --stop-timeout 3600 -it --name a4h -h vhcala4hci -p 3200:3200 -p 3300:3300 -p 3601:3601 -p 8443:8443 -p 30213:30213 -p 50000:50000 -p 50001:50001 --ulimit nofile=1048576:1048576 sapse/abap-cloud-developer-trial:ABAPTRIAL_2022 -skip-limits-check -agree-to-sap-license

Took around 8 minutes for HDB Start to succeed, but the most important thing here is that it succeeded. Then it took another 10 minutes or so for the output telling us that "All services have started... have fun!". In other words, I have a working A4H system running on my M3 MacBook now.

So in conclusion, although using Docker Desktop on Apple Silicon MacBooks is not a viable path to get the SAP Cloud Developer Trial container working, instead we can create an emulated VM and this is still a viable path to success.

[dylan-drummond]

When restarting the a4h container using Podman on the emulated (QEMU) Ubuntu amd64 VM, startup took 5 or 6 minutes in total. So that's reasonable. SAPGUI usage etc runs a bit slower on the SAP system-side via QEMU than via Rosetta, but in the end when you run an amd64 container on an M-series MacBook, there is always going to be some kind of emulation overhead.

[TomHoepping]

Good evening Dylan @dylan-drummond  

Thx for your investigation and tests. Meanwhile I can confirm, that the new 2022 image is also running with utm/ubuntu and docker. I've the feeling that Podman is a bit faster, but never mind. 

Also there is an interesting thread on docker rosetta support, which is quite specific about the issues regarding Rosetta2. Anyway maybe it is worth trying a test with docker native and qemu in the current version. https://github.com/docker/roadmap/issues/384

Will give an update. Bests Tom 

[TomHoepping]

Short Update: 

-e EXPERIMENTAL_DOCKER_DESKTOP_FORCE_QEMU=1

Parameter will trigger execution of the image with QEMU instead of Rosetta2  in native Mac environment but leads to the old problems executing without utm. summarized, it doesn't help.

[dylan-drummond]

I tried various tricks with Apple QEMU and even with the qemu_static and qemu_system (Linux VM) parameters in December last year. None of those experiments were successful.

Although we can speak of Docker Desktop and Docker Engine as "native Mac" apps, really the Docker (or Podman) doesn't run containers natively on macOS, because basically OCI containers run on Linux, so there is always a Linux VM involved in these cases.

Which is why I don't feel it's conceptually much different from emulating a Linux VM using UTM, and then running Docker inside that VM "natively".

UTM is better at handling QEMU than me, so that's my best guess as to why things work with UTM but not via manually setting QEMU parameters...

[TomHoepping]

@dylan-drummond  Yeah, you're right.
I've a laughing and a crying eye. With docker desktop on Mac I had a much better performance energy efficiency. On the other hand, with UTM it is working stable and he we've a 2022 Image ;). 
So keep fingers crossed, that apple/docker will fix something there. 

Bests Tom 

[julieplummer20]

Hi @dylan-drummond , hi @TomHoepping ,

Many thanks for all these comments. As a non-Mac person, I have learnt a great deal.

I have already passed on this blog post to several users and will continue to do so.

Best wishes (and if it applies: Happy Holidays),

Julie.

[dylan-drummond]

Hi @julieplummer20 ,

yep, my hunch is that Apple changed something in their Virtualisation Framework in the spring, and since then neither Docker Desktop nor Podman Desktop and their underlying Engines are able to run SAP CDT instance. However the still existing Apple QEMU Framework is able to do this task, and that is why there is still a way for Apple M-series MacBook users to get SAP CDT working ok.

My plan is to check the situation after new versions of macOS are released, as that is the time when updates to Apple Virtualisation Framework are delivered, to see whether the updated Virtualisation Framework can be used with the Docker Desktop or Podman Desktop for a (faster than QEMU) runtime of SAP CDT.

I was thinking, if ARM-based Windows laptops become popular as they may well do in the coming years, will SAP CDT be runnable on those... and if not, would SAP be interested in making an arm-architecture image of SAP CDT... you can Julie always let me know if you want volunteers for participating in such a project 🙂

Regards, dylan

[TomHoepping]

Hi @julieplummer20 , Hi @dylan-drummond 

Short update from my site. I've did some short tests with new MacOs Version Sequoia. Thought it would be worth a try, as apple announced especially in the release notes of the game port tool kit that rosetta2 will now support the avs2 instruction set. In contrast, the Rosetta 2 release notes for Sequoia still contain a bug regarding Intel emulation. So it is not quite clear to me what should work and what should not. Anyways.

The old 1909 image is working again natively! with Docker Desktop and Orbstack ( Which is so fast on my MAC M1 ;), huge performance gain). The 2022 Image is still not starting with the known shared memory error during hdbutil startup. I will investigate this a bit in the next week. 

Bests Tom

PS: Also I've found some blog-post where some guys down ported the rosetta 2 version, to the version of MacOS 14.1 with full AVS2 support successfully. Maybe I'will investigate this to. 
 

[dylan-drummond]

Hi @TomHoepping ,

glad to hear you got the 1909 image working using Docker Desktop and/or Orbstack.

For me, the constraints on a solution that would count as "Better than QEMU-enabled full VM emulation" are:

- must work with latest version of macOS

- must work with latest version of Rosetta 2

- must work with Abap Cloud Developer Trial image (currently the 2022 tagged release)

For macOS Sequoia, latest Rosetta 2, and latest Abap CDT, as you mention this isn't working (tested also myself last week). Of those 3 constraints, the only one I'd be willing to break for POC purposes is the one about the version of Rosetta 2 - so if you put a link to that blog-post where someone has downgraded Rosetta 2, then I can also take a look at it. Though if you want to be the first one to test that workaround, then that's also ok, in that case looking forward to hear how your test goes...

[TomHoepping]

Hi @dylan-drummond 

I see it the same way. 1909 is not a real option for practical use, just a test for comparison and learning. 
Same for the rosetta downgrade, probably not an option for daily usage. Not sure if a down-port has to be done then with every minor patch/update. Interesting links are the following. I guess the tutorial will also work in the opposite way. 
https://www.reddit.com/r/macgaming/comments/1dd8jvs/tutorial_backport_rosetta_2_runtime_with_avx2/
https://news.ycombinator.com/item?id=25547896

[dylan-drummond]

@TomHoepping ... actually maybe I'll skip the POC, as the reddit blog seems to be all about using macOS Sonoma then getting a forward-version of Rosetta2 (Sequoia Beta), and disabling SIP, etcetera etcetera... all feasible activities, but too much hackarounding for my liking, especially the bit about running macOS Sonoma, since I already committed all my Macs to Sequoia.

But if you get into all that, then let us know the results!

[AnkanSikdar]

Hi @dylan-drummond , first of all you are a gem for writing this extensive guide. Thank you so much for taking your time out to write this.

I just wanted to share some updates from my side. I have a Macbook Pro with an M4 Pro chip with 24GB RAM with Sequoia 15.3 installed. Following your guide I have managed to install the latest version 'abap-cloud-developer-trial:ABAPTRIAL_2022_SP01' via the UTM/QEMU path. The docker path is not working as of today, unfortunately.
Here are the settings I have used for anyone interested.
CPU Cores: 6
RAM: 20GB (Yes, RAM swapping does help)
Disk: 256GB

I really hope with the growing popularity of ARM64 processors, one day we will see native support for it.

[dylan-drummond]

Hi @AnkanSikdar ,

good that you got it working with lower RAM allocation than SAP recommends, as it brings more MacBooks into the "Feasible Devices" set.

My guess is that since around macOS 14.4 (spring 2024) when Apple tightened Rosetta security to prevent e.g. emulated Java processes issuing taskkill type commands, this unfortunately for "Docker path for Abap Trial with Rosetta" means it is unlikely to ever work again. So there is only the UTM/QEMU path available, but at least that one works even though it is of course a bit slow-running (as full emulation of the VM).

It could happen, that in the future Apple actually drops Rosetta, as the idea behind Rosetta is that it is a "transition period"-technology until some critical mass of macOS app providers have switched to providing arm64-versions of their stuff... and in general this critical mass already exists... alternatively, Apple might choose to keep providing Rosetta... I have no insight into what they will actually do.

For Abap Trial, maybe if the popularity of Windows on arm64 increases (as many predict), then SAP will have more motivation to provide an arm64 Docker image of Abap Trial. @julieplummer20 might have some insight or view as to whether this idea which I raised last year, has some chance of getting onto the Abap Trial product roadmap?

[TomHoepping]

@dylan-drummond  A short update from my side. 

With Docker/Orbstack for Mac it is currently possible to run the 1909 image natively. 

The 2022 is still not working. Basically the hdbdaemon is not starting, based on the error message, I've the gut feeling that this is more related to this bug. https://github.com/docker/for-mac/issues/7058
As the error message is indicating that under pid768 something like hdbdaemon is expected and received something like rosetta/hdbdaemon. If this hypothesis is true, then not sure where to search, either docker container startup script or sapstart. Currently I've not that much time to investigate (and limited knowledge their)  But, I've the feeling that not much is missing to get the 2022 image running as well. 
@julieplummer20Do you guys have any hints? 
Bests Tom 

[dylan-drummond]

Hi @TomHoepping 

I remember you mentioned before that the (now unavailable, but you have an old copy saved) Abap Trial 1909 still runs ok on Docker. And based on the error messages that come with Abap Trial 2022, then the fail-point is indeed about hdbdaemon not being able to get started.

So far, we are in the realm of what we know. Your idea, that it's related to the /proc stuff receiving the wrong arguments with "rosetta/" pre-pended... it's possible, though my feeling is that maybe not probable, as the root cause... because:

1. If you look at the github issue, it was raised in November 2023 and discussed in the months after that. But from summer 2023 till around March 2024, the Abap Trial 2022 on Docker Desktop for Mac, was working fine (including all processes related to hdbdaemon). So the time-periods are incompatible with something in Docker or Rosetta prepending "rosetta/" as the root cause of the issue for Abap Trial 2022.

2. Although it must be something to do with the HANA DB, as that is the main difference between Abap Trial 2022 and Abap Trial 1909 (Sybase DB) in terms of getting the database started (which always comes before any ASCS or CI app server can get started), I am not sure that from the point of view of Rosetta, it makes much difference what the process is... Rosetta is only interested in catching processes, which are trying to perform actions that are x64-specific. ... On the other hand, if you can run Abap Trial 1909 and see whether the /proc is receiving e.g. "sybasedaemon" or "rosetta/sybasedaemon", then that would remove the force of this "Point 2." if it turns out that sybasedaemon is fine because not calling up Rosetta... my guess, which could be wrong, is that Sybase is also doing x64-only stuff and needs to be caught by Rosetta...

But the main reason I think that the root cause is about security tightening for Rosetta, is that a load of issues were reported when, was it, Sonoma 14.4 update rolled out (not just issues for Abap Trial, but also many Java programs in Docker started failing). And those security updates are delivered via OS updates. So whether it is tighter sandboxing of emulated java processes or emulated hdbdaemon process, the results are rather similar, in effect as well as when they started occurring.

So I'm not proud: after all, my hypothesis is also guesswork, not knowledge; if it turns out that your hypothesis is right, and there would be a way to workaround the root cause it points to, then that would be great.

However, if we now do the thought experiment, and say that we assume the root cause is because of the way Apple is parsing the argv0 to /proc... it might be possible, as some suggested in the github thread, to let's say login to a container spun up already from the Abap Trial 2022 image, and do some clever rewrites somewhere, and save them, then restart the container so that, say, in the container, some scripts or utilities or arguments are able to accomodate "rosetta/" additions... might be. I doubt Julie's team would be getting involved in that though, as the idea of the Docker image is that it runs on a Linux VM on any platform... also, as this issue is not actually a showstopper, since for the subset of Abap Trial 2022 users with arm64 MacBooks, there is the UTM/QEMU path still available... so I doubt SAP would be greatly motivated to work on a solution for such a small set of consumers of a (free) SAP product...

Hence my hope is still that in the near future, adoption of arm64 Windows could be a motivation for SAP to provide a Linux arm64 Docker image. In general, many providers of Docker images and even non-containerised apps have started to provide arm64 builds, because in Linux world arm64 is already quite popular - for example, arm64 Ubuntu distros are already quite usable i.e. you can apt install lots of arm64(aarch64) versions of software already.

Sorry this is a bit of a ramble, kinda thinking out loud here 🙂 Have a nice day now, n regards, -dylan

[AnkanSikdar]

@dylan-drummond

If your hypothesis is indeed correct, then it is very sad indeed that we will never be able to run the ABAP Trial on Docker again on M series Macbooks, as in my opinion the main reasons behind buying a laptop with AMD64 processor are amazing performance & battery life. Both of which take a massive hit when using the fully emulated approach.

I will sincerely hope SAP to consider the idea of building the image to run natively on AMD64 one day.

Best Regards,
Ankan

[MichaelFromNPB]

Hi @dylan-drummond,

Thank you as well from my side for this excellent guide! This makes my Mac ever more special to me:-). I am very impressed about its performance and compared to earlier Mac models I am not noticing it heating up as much as well. My MacBook is an M2 Max with 32 GB RAM running on Sequoia. I decided for the installation of the Abap Cloud Developer Trail 2022 SP01 with Docker on the Ubuntu Server VM. Startup time is about 8 min. 

I stumbled onto this guide during the process of installing SAP development software on my Mac. Now I have the Java SapGui 7.8 running being able to access the SAP backend server running in the VM. Currently I am not able to access the server from my Eclipse installation on my Mac but I have a workaround running with an Eclipse installation on a Windows 11 VM. 

The error message raised in Eclipse during the logon process is:
com.sap.adt.communication.exceptions.CommunicationException: partner 'vhcala4hci:3300' not reached;

Caused by: com.sap.conn.jco.JCoException: (102) JCO_ERROR_COMMUNICATION: Initialization of repository destination A4H_001_supperm_en failed: Connect to SAP gateway failed

connection parameters: TYPE=A DESTINATION=A4H_001_supperm_en ASHOST=vhcala4hci SYSNR=00 PCS=1

I installed additional java SDK’s, generated through the SAPMachine Manager but I believe it is missing the SAPJCo connector. Maybe this is the problem?? In case anybody having also Eclipse installed on the M series Mac not having a problem could confirm that this setup should work would be appreciated. 

Here now a screen print including all described software:

For me the end goal would be to play around with CDS views, RAP and Fiori apps on this server.

Thanks again for the guide and I will keep updating my experiences with it.

Best regards,

Michael

[dylan-drummond]

Hi @MichaelFromNPB ,

this question is pretty much off-topic, as this blog was only about installing/spinning-up the mini-SAP, not about how to connect to the instance using JCo or otherwise. 

Anyway... since you can connect to the SAP instance using a Windows VM - we'll assume this is on the same hypervisor subnet - then most likely the reason you cannot connect from host Mac, is that you messed up somehow with networking... e.g. the port-forwarding at creation of instance, or the hosts file resolution, or similar.

At spin-up time, it can be a good idea to include the message-server standard port 3601 as part of the docker run or podman run command. Myself I always include it, but I suppose it is not part of the standard instructions... anyway the addition is:

-p 3601:3601

Unfortunately, without a lot of trickery we won't go into here, it's not easily possible to ensure this port-forwarding works later.

That is all the advice I can really give based on your issue description. Good luck with resolving the bug.

[MichaelFromNPB]

Hi @dylan-drummond 

As a quick update, I am now able to run SAPGUI and Eclipse on my M2 Mac against the ABAP Cloud Developer Trail installed backend. I may have switched an Eclipse parameter which prohibited a successful logon (Not the place to go further into this). After a clean delete and reinstalling both tools worked as expected. I did not need to adjust anything in the SAP Java Connectors or with port-forwarding.

Thanks for all the information and best regards,

Michael   

[dylan-drummond]

Hi @MichaelFromNPB ,

good that you fixed the issue. Usually, when something works from one location, but not from another location (e.g. Abap in Eclipse working from Win11 device but not from Mac device), the issue is either with networking (as suggested by the previous error messages), or with divergent config between the devices (as turned out to be the case, with Eclipse parameters).

[TomHoepping]

@dylan-drummond 

Regarding probably why 2022 image doesn't launch natively - you're probably right. I saw the same error today regarding PID / rosetta engine.
You also mentioned that 1909 runs on Sybase? Is that really the case? In dbacockpit it is listed as HANA. Bests Tom 

DBA Cockpit 19091909 Trial2022 Trial

[dylan-drummond]

Hi @TomHoepping ,

some kind of memory-fail by me... Abap Trial 1909 did indeed run on HANA DB. For some reason when I was writing that post about Sybase, I was looking at a SAP NetWeaver Dev Edition 7.52 mini-sap (the predecessor to Abap Trial 1909) which I happened to have up and running... and that mini-sap does use Sybase... so somehow confused those two old and deprecated mini-SAPs to come up with the Sybase idea.

Sorry for the confusion.

In any case, something about the way in Abap Trial 2022 the HDB is starting up, offends macOS since 14.4 and hence the HDB just cannot somehow get started... similar to what has happened with non-SAP Rosetta-emulated x64/amd64 processes of other containers. So from the point of view of the two Paths of this blog, still only the full VM-emulation Path works i.e. the much slower one.... but no doubt SAP will soon produce an arm64 Abap Trial image and everything will be superfast again... [disclaimer: just a hopeful joke, not expecting this to happen anytime soon].

[TomHoepping]

Hello @dylan-drummond 

Just wanted to give you a quick update. Still need to investigate and test this a bit, don't understand from release notes which change of the latest release led to this change. But.... drum roll ... drum roll.... 

With latest docker version 4.41.0 and Mac OS Version 15.4.1 the 2022 Image is coming up. 

Good news ;)! Bests Tom 

[dylan-drummond]

Hi @TomHoepping ,

👍sounds good... I had already got used to running the container on an M4 (used at work) using emulated VM (as it is just about quick enough even for the SAML2 Abap WebDynpro tcode, though of course not that quick), but anyway will check this evening if suitable macOS + docker version is enough on an M3 (used at home)...

[dylan-drummond]

@TomHoepping @julieplummer20 

Based on the report from Tom earlier today, did a quick check on M3 Mac, macOS 15.4.1 (Sequoia), Docker Desktop 4.41.2, Docker Engine 28.1.1... and.... it works just fine. So that's great news. 

[Just noticed, that macOS 15.5 is available, so let's see once macOS is upgraded if everything still works... 😱 ... upgrade done, container restarted ok and can login fine, so seems like macOS 15.5 is also good for using this container].

sudo docker run --stop-timeout 3600 -it --name a4h -h vhcala4hci -p 3200:3200 -p 3300:3300 -p 3601:3601 -p 8443:8443 -p 30213:30213 -p 50000:50000 -p 50001:50001 sapse/abap-cloud-developer-trial:ABAPTRIAL_2022_SP01 -skip-limits-check -agree-to-sap-license 

Julie, you could adjust the advice at https://hub.docker.com/r/sapse/abap-cloud-developer-trial#macos to reflect the fact that macOS and Docker Desktop path is now usable again even for arm64 MacBooks.

[TomHoepping]

@dylan-drummond Really good news! I'll will upgrade to 15.5 the next days - have to do this anyways and give an update. Also I will give orbstack a try again, as their performance regarding containers is really  good and impressive. 

[TomHoepping]

@dylan-drummond, @julieplummer20   Can confirm, with 15.5 it is working as well. Orbstack I still need to test. 

[TomHoepping]

@dylan-drummond, @julieplummer20 Sorry guys for the many messages. On Orbstack the Container is starting as well ( MacOS 15.5 and Orbstack 1.10.3). Orbstack is working a much faster during startup (M1 MAX) and resource efficient regarding memory and battery-usage. But might not be the reliable solution compared to docker. Docker has a much broader user base and faster updates/reactions regarding changes of Mac OS / Rosetta. 
Anyway I will continue to work with orbstack and give it a try. Keep you updated. 

Enjoy your weekend and bests Tom 

[dylan-drummond]

@TomHoepping good that it has started working on Orbstack as well. From the timing of things starting to work, we could make the guess, that the "fix" was not somewhere in e.g. Docker Engine, or in Orbstack, but instead occurred in the Apple Virtualisation framework via some update, for example in macOS 15.4.1 ... for us, as consumers of an amd64 container, the main thing is that it's nice there is this Rosetta-based way of running the container.

As noted somewhere above, Apple doesn't guarantee there will always be a Rosetta-solution available, but let's hope it is there for another 20 years!

[julieplummer20]

Hi @TomHoepping ,  @dylan-drummond ,

 Please don't apologise! Thanks a lot for all the testing; it really helps us out.

Glad to hear this seems to be working; will try to update our guidance / readme etc where appropriate.

Very best wishes,

Julie.