Technology Blogs by Members
Explore a vibrant mix of technical expertise, industry insights, and tech buzz in member blogs covering SAP products, technology, and events. Get in the mix!
cancel
Showing results for 
Search instead for 
Did you mean: 
dylan-drummond
Contributor
3,846
Let’s say you have or will have an SAP NetWeaver instance running as a VM (virtual machine) on your laptop or desktop computer. Sometimes you might want to send out emails, programmatically or as a dialog user. So these emails, when “sent” without any setup work, they are not actually sent to anyone, instead they go to the so-called SOST Queue. For example, let’s use tcode SBWP to create and send an email:



In SOST we see our email sitting in the SOST Queue buffer:



If it is enough for you to know that the emails get to SOST Queue, then you can probably skip this blog. If you want to really send the emails to their recipients, then this blog shows you one way to set up an SMTP scenario, to reach this goal of actually sending the emails…

[Disclaimer: if you get stuck, then I’m highly unlikely to be able to debug your setup or replicate your error, so you need to be mentally prepared for BYODS (Bring Your Own Debugging Superpowers 🤣). Also I assume basic competencies in such things as installing an OS on a VM, installing SAP NetWeaver Developer Edition using the tutorials, and the Linux command line… nowadays by searching the Internet you can usually find all the advice you need to overcome all bugs and setup errors, though i’m not saying it’s always easy, it often isn’t].

Here is the town planning diagram showing the main components and “SMTP flow” - whoever drew this isn’t going to win the Neatest Diagram of 2020, but never mind, you get the idea:



The two orange boxes are VMs on your host laptop or desktop, running inside some Type 2 Hypervisor such as VMware Fusion/Player or Oracle VirtualBox. One VM is called vhcalnplci (SAP’s recommendation): SAP NetWeaver Developer Edition 7.52 SP04 is installed and running here. The other is called linux-hjno (a random hostname, if you like and know how then you can change whatever hostname openSUSE gave you to a nicer hostname): Postfix is installed and running here. Postfix is a popular, free, open-source Mail Transfer Agent:

http://www.postfix.org/

https://en.wikipedia.org/wiki/Postfix_(software)

When the SAP user (or a Send job) picks up an email in SOST Queue and sends it, it goes to the Postfix server listening on port 25 of linux-hjno. 

Now the red box represents somewhere on the Internet run by Google, specifically we are using smtp.gmail.com as our free relayhost: i.e. smtp.gmail.com receives whatever emails Postfix sends out and tries to relay those on to the email inboxes of their proper recipient(s), the inboxes are represented by the light blue box.

So let’s get started on this wee project...

Set up an SAP instance on a VM


We will use Desktop (Gnome) distros of Linux for our VMs, although that is mainly just because it’s more colourful in our grey nerdy lives to have a GUI; in effect all the steps could be done with server versions since all steps are done using the command line (except my SSH trick from the Files app, but for 100% sure the equivalent task is doable from the command line there also… probably there are some supernerds out there who think SSH should only be used from the command line). Using server (non-GUI) distros is thus left as an exercise for anyone who wants to, or e.g. who finds it tricky to run e.g. 2 GUI VMs on a laptop due to RAM constraints.

(If e.g. for reasons of laptop RAM constraints or Hypervisor NAT networking trickiness, you prefer to run the Postfix server on the same VM as your SAP instance (i.e. you only need one virtual machine then instead of two), then this setup works fine as well. Postfix will have a small footprint compared to SAP itself. Adjust the instructions in this blog to suit using one VM, e.g. in tcode SCOT you can specify the host as “localhost”, more details below).

For this example I run SAP on openSUSE on VMware Fusion, there isn’t an SAP tutorial for exactly that Hypervisor, but here are the sites for downloading the rar files and installing on VMware Player or Oracle VirtualBox:

https://blogs.sap.com/2019/07/01/as-abap-752-sp04-developer-edition-to-download/

...from there see e.g.: “Installing SAP AS ABAP 7.52 SP04 on Linux – VMWare” i.e.:

https://www.sap.com/documents/2019/09/c86f9218-687d-0010-87a3-c30de2ffd8ff.html

You can follow this VMware Player tutorial from SAP when using VMware Fusion. As Shared Folders didn’t seem to work for me at first, so instead, I copied the NetWeaver installation files into the VM using “graphical SSH” into host from the Files app (screenshot): so in this case I know the host computer is on node 192.168.108.1 and I can enter in the “Connect to server” field of Files: ssh://192.168.108.1 and press Connect… then enter host computer’s username (i.e. MacBook user) and password, then we can browse directories of the host and can transfer files etc:



In case anyone wants to set up their SAP NetWeaver Developer Edition on Ubuntu VM, I have a concise guide, as well as a somewhat older non-concise guide:

https://blogs.sap.com/2019/10/20/concise-guide-to-install-sap-netweaver-developer-edition-on-ubuntu-...

https://blogs.sap.com/2018/04/05/installing-sap-netweaver-developer-edition-on-an-ubuntu-desktop-vm-...

(From the point of view of our Postfix examples, it likely doesn’t make any difference if the SAP instance is running on Ubuntu or openSUSE, basically at the tcode level of SCOT and SOST, SAP doesn’t need to know which OS it is running on).

Get a Gmail App Password


To obtain an app password from your Google account, here is advice from Google themselves:

https://support.google.com/accounts/answer/185833?hl=en

When you get to the screen for generating a new app password, choose “Other (custom name)” from the “Select App” dropdown-listbox, and give it a name. The you press Generate to get the new 16 character password.

When this 16 character password is displayed (the spaces don’t count, they are just there to make the password look nicer as 4 blocks of 4 characters), you should copy it somewhere safe (it won’t be too great if someone else finds your password for using your Google account). Once you have got the email workflow all running nicely, you should delete your temporary copy of the 16 character password… If someday you need to know it again, don’t worry just go and generate a new app password from Google account as before. 



[There are some limits on how many emails you can relay - Web interface user of free Gmail can send 500 emails in a day, non-Web sending (such as in our example) gets a lower quota I think. However, even when I was debugging this whole topic and sending out around 80 emails in one day via Gmail, no limit reached. Also, there was some Internet chatter saying that for G Suite accounts, app passwords will not be allowed in the near future (no new ones from mid-June 2020, and old ones won’t work sometime in 2021). However, I think that for ordinary non-commercial Gmail accounts (“free Gmail”), app passwords are still allowed in future. Let’s see what happens].

Set Up a Postfix server (on openSUSE).


When installing an openSUSE VM, you can disable the Firewall in the Installation Overview step; or, you can keep the default of enabled Firewall, and later allow openSUSE VM to listen on port 25 for incoming email; whichever way you prefer, is ok: just mentioning it in case later you are wondering why Postfix is not accepting any emails sent from SAP, it could be something as simple as the Firewall.



So to recap, now we are setting up Postfix to use Gmail as relayhost, on openSUSE (by the way, there is also "Ubuntu-specific Postfix advice" section below if you want to run Postfix on Ubuntu); well there’s an excellent tutorial showing all the needed steps, including which file to put your username@gmail.com with your new app-password you recently generated, how to generate the password hash, which parameters to set with which values, and so on...

https://www.howtoforge.com/tutorial/configure-postfix-to-use-gmail-as-a-mail-relay/

The main thing I would change in that tutorial, is adding “sudo” to all the commands, since almost everything has to be done using root privileges (or you could do “sudo -i” and just “run as root”, though I prefer the "explicit sudo-prefixing" style).. So here we summarise the commands and config values for opensuse, based on that tutorial, but using the sudo prefix:

--begin [extracts for openSUSE from howtoforge-tutorial, own comments in square brackets]
sudo zypper update && sudo zypper install postfix mailx cyrus-sasl

[though probably nothing to do here for zypper, since in openSUSE Leap 15, postfix mailx and cyrus-sasl are already installed].
sudo nano /etc/postfix/sasl_passwd

Add the line:
[smtp.gmail.com]:587    username@gmail.com:password

Save and close the file. [This is the file where your gmail address and new app-password go], Your Gmail password is stored as plaintext, so make the file accessible only by root:
sudo chmod 600 /etc/postfix/sasl_passwd

[Modify config file:]
sudo nano /etc/postfix/main.cf

Add or modify the following values:
relayhost = [smtp.gmail.com]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_tls_CAfile = /etc/ssl/ca-bundle.pem

[Note that these parameters already exist in the main.cf file, and just need to be located and modified to match the values above].

Save and close the file.

OpenSUSE also requires that the Postfix master process configuration file master.cf be modified. Open it for editing:
sudo nano /etc/postfix/master.cf

Locate the line which reads:
#tlsmgr unix - - n 1000? 1 tlsmg

Uncomment it, so it reads:
tlsmgr unix - - n 1000? 1 tlsmg

Save and close the file.
sudo postmap /etc/postfix/sasl_passwd

sudo systemctl restart postfix

Test your new configuration by sending an email using the mail command. Run:
mail -s "Test subject" recipient@domain.com

You will be presented with a blank line (or a CC: field, which you can bypass by pressing Enter). Type the body of your message, pressing Enter for new lines. When you are finished composing the email, type CTRL-D to send it. To cancel the email, press CTRL-C twice.

--end [extracts for openSUSE from howtoforge-tutorial]

Couple of other remarks…


For inet_interfaces parameter of main.cf file, check that it is set to the value “all”. So to check quickly, we can do:
sudo cat /etc/postfix/main.cf | grep inet_int

If it is not set to "all", then change it in main.cf:
inet_interfaces = all

Then restart postfix to pick up the changes:
sudo systemctl restart postfix 

If you want postfix to start up every time you reboot your VM, this should work:
sudo systemctl enable postfix

Always a good idea to check after reboot whether it worked:
systemctl status postfix

...if necessary restart the postfix service.

--

It’s also a good idea to add to /etc/hosts file (in both VMs, but especially for vhcalnplci which will want to know how to resolve hostnames of any Postfix servers on the subnet) an entry allowing for easy identification of the other VM, also it can sometimes be useful to “self-identify”; so we would add to hosts files e.g. the lines:
192.168.108.143 vhcalnplci
192.168.108.145 linux-hjno

--

Before switching to SAP to complete the setup, on the vhcalnplci server (SAP instance’s server) we can use the command line program “mail” to test sending email via our Postfix server on the other VM e.g. (you would replace “linux-hjno” with the hostname of your Postfix server, and so on...):
mail -v -s "Test ext-smtp using mailx" -S smtp=smtp://linux-hjno -S from="marmotic.meister@sapinstance.marmot(MurmelGeister)" recipient@theirdomain.tld

...add some message text, such as “Hello World!!” (this becomes the message body i.e. content of the email) and then Ctrl+D to send the email. This will give you a report on how things went with the sending attempt. (If sending fails, try checking the postfix and mail logs, as suggested in the Postfix-tutorial above under “troubleshooting”. Then it’s the usual procedure: trawling the Internet trying to find what the errors mean and what is the likely way to fix them… incidentally this was how I arrived at the “Ubuntu advice on mynetworks config” described below. Good luck in all your debug quests!).

Setting up SCOT in SAP


Back in our SAP instance, in SCOT-land 🏴󠁧󠁢󠁳󠁣󠁴󠁿, we should set up an SMTP node. 

So, go to tcode SCOT. Create an SMTP node. Let’s call the node “SMTP1”. Specify the host: we can specify “linux-hjno”(hostname of Postfix server), or “localhost” / “vhcalnplci” (when Postfix is on the same VM as SAP).

Specify port number 25, and “Do not use TLS” (well, if you like you can use TLS, then you need to provide probably your gmail and app password and Default SSL Client, up to you if you think anyone can hack into the traffic between SAP and Postfix… I would say you are unlikely to get into any security troubles in this “own laptop VMs”-scenario). 



…also make sure “Node in use” flag is ticked (i.e. is in “ON”-state):



Press the green tick icon to save your changes.

Add the allowed recipient domains via the button “Set” for Internet Address Types, use the wildcard ‘*’ as prefix for your selected domains:



One other thing: in the Settings menu, you need to specify a so-called Default domain; if you leave it blank (which is the default value), no email sending will happen from SAP, so, just set this to some value... a sensible Default domain for our use case is GMAIL.COM:



Remember to press the Save icon in the Settings pane.

Test email sending using SOST:


Now we can test this, we have our test email that we created earlier from SBWP, viewable in SOST. So from SOST, we can highlight the email item then press the clock+green-tick icon to start the actual sending, when a popup-to-confirm opens, select “Yes”. Status should change to a non-error value, such as 718 (click on the Status Message Number to view text):



...



 

Then you can go to your recipient’s inbox (if you were the recipient), or if you sent it to a friend (best to agree first that you will use them as testing help), then you can ask them to check their inbox.



So that is what success looks like 🙂 .

Scheduling a Send Job from SCOT


Once your email sending is working, you can automate it by scheduling a Send Job, which at periodic intervals (default is every 10 minutes, in our example we specify every 2 minutes) picks up the emails in the SOST Queue and sends them out. In this example, we restrict it to “INT” i.e. Internet Email:



...



Hitting “Continue” takes us to the Define Job transaction, remember to Save the new Job:



We can follow the progress of the periodic job from tcode SM37:







So this job picks up unsent emails from the SOST Queue and sends them out, in effect it does the same procedure that we did manually in our smoke-testing, for all the unsent, non-error status emails.

Ubuntu Postfix advice:


Follow the same tutorial as for openSUSE, but using the Ubuntu-specific commands and parameters, obviously:

https://www.howtoforge.com/tutorial/configure-postfix-to-use-gmail-as-a-mail-relay/#-enable-less-sec...

As mentioned above, you will want to prefix the commands in that tutorial with “sudo” (or you could do “sudo -i” and just “run as root”). So here is the summary from the tutorial, with sudo prefixes (this summary is different from the openSUSE summary, because the steps and some of the parameters are different for Ubuntu, hence the similar but different summary below for Ubuntu):

--begin [extracts for Ubuntu from howtoforge-tutorial, own comments in square brackets]
sudo apt update && sudo apt install postfix mailutils

When prompted for "General type of mail configuration," choose Internet Site.

When prompted for a "Mail name," choose a hostname to be used in mail headers as the origin of your emails. A fully-qualified domain name is preferred, but using your machine's simple hostname is OK. [We can use the simple hostname that we specified when installing Ubuntu, e.g. “postfix01”]. Regardless of what you enter here, your return address will appear to recipients as your Gmail address.

You may be prompted to set the "Root and postmaster mail recipient." Enter root, or another user who should receive mail subsystem notifications.

For any other prompts, you can choose the default values.
sudo nano /etc/postfix/sasl_passwd

Add the line:
[smtp.gmail.com]:587    username@gmail.com:password

Save and close the file. [This is the file where your gmail address and new app-password go], Your Gmail password is stored as plaintext, so make the file accessible only by root:
sudo chmod 600 /etc/postfix/sasl_passwd

[Modify config file:]
sudo nano /etc/postfix/main.cf

Add or modify the following values [seems only relayhost parameter line provided initially in Ubuntu's main.cf, i.e. you can modify relayhost, and add the other new parameters]:
relayhost = [smtp.gmail.com]:587
smtp_use_tls = yes
smtp_sasl_auth_enable = yes
smtp_sasl_security_options =
smtp_sasl_password_maps = hash:/etc/postfix/sasl_passwd
smtp_tls_CAfile = /etc/ssl/certs/ca-certificates.crt

Save and close the file.
sudo postmap /etc/postfix/sasl_passwd

sudo systemctl restart postfix

Test your new configuration by sending an email using the mail command. Run:
mail -s "Test subject" recipient@domain.com

You will be presented with a blank line (or a CC: field, which you can bypass by pressing Enter). Type the body of your message, pressing Enter for new lines. When you are finished composing the email, type CTRL-D to send it. To cancel the email, press CTRL-C twice.

--end [extracts for Ubuntu from howtoforge-tutorial]

Once you have got Postfix up and running, then you can add this VM - mine has hostname “postfix01”, imaginative indeed - so yes you can add to the /etc/hosts file of the SAP VM vhcalnplci, e.g. this line:
192.168.108.144 postfix01

...and then in tcode SCOT, change the host to be “postfix01”, and press the green tick to Save changes. But don’t go to SOST just yet, because you still need to make one more change to Ubuntu Postfix that is not mentioned in the above tutorial...

When the Postfix server is on Ubuntu, you need to add the Hypervisor-owned subnet (that the SAP VM and the Postfix server are on) to the mynetworks parameter in main.cf, so the third subnet is the one added to this parameter’s default values on Ubuntu’s version of main.cf. So there are two ways to do this... you can do it this way, e.g. I added subnet 192.168.108.0/24 :
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.108.0/24

 

...or, you can comment out the existing mynetworks parameter and add the mynetworks_style parameter with value “subnet” (mynetworks_style parameter only works in case no mynetworks parameter found when restarting Postfix):
#commenting out mynetworks, adding mynetworks_style:
#mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128
mynetworks_style = subnet

(As usual after changing anything in main.cf, you need to sudo systemctl restart postfix and check it restarted ok using systemctl status postfix ).

Either of the above changes (to mynetworks or mynetworks_style) in main.cf, means that the “permit_mynetworks” criterion is met in the parameter:
smtpd_relay_restrictions = permit_mynetworks permit_sasl_authenticated defer_unauth_destination

[In Postfix on openSUSE, it is not necessary to add your subnet to mynetworks, as your subnet is already by default a trusted SMTP network, here is more detail from the main.cf file itself on openSUSE’s implementation of Postfix:
# TRUST AND RELAY CONTROL
# The mynetworks parameter specifies the list of "trusted" SMTP
# clients that have more privileges than "strangers".
#
# In particular, "trusted" SMTP clients are allowed to relay mail
# through Postfix. See the smtpd_recipient_restrictions parameter
# in postconf(5).
#
# You can specify the list of "trusted" network addresses by hand
# or you can let Postfix do it for you (which is the default).
#
# By default (mynetworks_style = subnet), Postfix "trusts" SMTP
# clients in the same IP subnetworks as the local machine.

mynetworks_style = subnet

So this is why on openSUSE there is nothing needed to do, whereas in the Ubuntu implementation of Postfix you need to make a config change to have your subnet trusted].

One other note about setting mynetworks_style to value “subnet”: if you have chosen e.g. bridged networking so that your VM is “just another node on the corporate network” or similar, then by specifying that the whole subnet is trusted for SMTP, you are allowing nodes you don’t necessarily trust, to use your Postfix server. So if not using Hypervisor NAT for your Postfix server, I suggest specifying manually the nodes that should be in the mynetworks parameter, e.g. if there would be two servers allowed to send mail via Postfix, on 192.168.0.2 and 192.168.0.3, you would have in main.cf :
mynetworks = 127.0.0.0/8 [::ffff:127.0.0.0]/104 [::1]/128 192.168.0.2 192.168.0.3

Anyway, remember to sudo systemctl restart postfix any time you have changed the main.cf or master.cf files. And check using systemctl status postfix that all is well. All is well when there is some green text saying “active (exited)”. If something went wrong expect to find “inactive” and some red text. If inactive, probably have a syntax or semantic error in the configuration files, so good luck with solving those ?.

See also “Couple of other remarks” above, for checking that inet_interfaces parameter is set to “all”, for enabling postfix to restart on reboot, and for how to test Postfix from the SAP VM vhcalnplci.

How to avoid getting Delivery Status Notifications


If you don’t want all the Delivery Status Notifications (which you receive for all successful deliveries as default behaviour), you can add an extra line in main.cf to disable them (in openSUSE or in Ubuntu):
smtpd_discard_ehlo_keywords = silent-discard, dsn

Then restart Postfix to pick up the changed config.

Sender is the gmail user from Postfix config


The relayhost (Gmail) will specify the email Sender address as the app password owner specified in the postfix file; if you need to find the original sender, it's in the message metadata e.g.:
From: Marmotic Meister <yourusername@gmail.com> 

X-Google-Original-From: Marmotic Meister <marmotic.meister@sapinstance.marmot>

 

Miscellaneous


If you notice any errors in the blog e.g. some step missed out, or something misdescribed or a parameter value missed out or a command spelt wrong etcetera, then feel free to suggest improvements via the comments section.

--EOF
3 Comments
Labels in this area