Application Development Blog Posts
Learn and share on deeper, cross technology development topics such as integration and connectivity, automation, cloud extensibility, developing at scale, and security.
cancel
Showing results for 
Search instead for 
Did you mean: 
Former Member
2,697

Hello everybody, I like to introduce myself first. My name is Patrick and I am a security consultant. Today, I like to talk about our first project “connecting non SAP systems to SAP ETD”. We and SAP Deutschland SE & Co.  KG (SAP) commonly have worked successfully together in order to attach non SAP systems of the city Wolfsburg to their ETD instance. The responsibility assignment between my company and SAP was the following:

My company connected several systems to the ETD. We used Syslog-NG for the log file transport on Unix systems and our Windows agent for transportation of Windows Event Logs. In addition, we created patterns and charts to visualize the behavior or to trigger alarms.


In the case of issues, we contacted our SAP partner and supported to reproduce these issues in both ETD instances, ours and SAPs. The team around SAP ETD created an update or fix within a few hours. An example has been not yet supported timestamp formats.


A schematic workflow of connecting non-SAP systems to the ETD is visualized in the following picture. In ETD SP3 version is it not necessary to create new attributes anymore. An almost complete list of attributes is available, which can be used in the log learning process.


1. ETD Configuration

Once the basic installation had been done, new log attributes were created in ETD. After creation of new log attributes, the system is ready to perform the log learning process.


Please note: Log attribute configuration has been included into the common ETD setup with SP3. Therefore no manual extra steps are necessary anymore.


1.1. Create necessary Attributes

Creation of log dependent attributes used to be necessary in SP2. In this version the ETD provided a few basic attributes only that could be taken into account for specific log characteristics (e. g. “ip address” or “port number”). These attributes provide an ETD-wide consistent semantic. Therefore an extraction and mapping of log file entries with ip address information to the ETD attribute “ip address” is necessary. It allows comparison and correlation of different log entries and source types. Since SP3 the ETD provides plenty more attributes out of the box.

For example, we added the following attributes during the ETD SP2 integration:

  • “Is Computer” (to decide is the log from a computer or a server)
  • “Log Level” (see the log level of a specific system)
  • “Rule ID” (to see which ID was taken of the system)
  • “Message Text” (for explanations of log files)



1.2. Start Log Learning

The final step of the ETD configuration is the log learning process. Log learning can be started from the main page of the ETD.


Log learning needs a small representative log file which contains a lot of different log events. It simplifies the learning process, because you do not have to deal with a bunch of trash messages. Therefore you can see just one entry in the mark-up table below. You typically retrieve more entries when using a regular syslog file. Then a deeper understanding of the log file and its structure is needed.


The windows agent log output meanwhile consists of well-structured key-value pairs. Matching of those to new or standard attributes to the values is a straight forward procedure.


When log file entries appear to be less structured a deeper experience in value-mapping is needed. In those cases a “value mapping” function is available. It allows usage of regular expressions for extraction of the corresponding values out of the log message text.

Final step in the log learning process is a test import. As soon as this import terminates successfully, the ETD is ready to import incoming online log streams.


2. Connecting non-SAP systems to the ETD

We connected the following systems to the ETD:

  • Unix systems and appliances
    • Email-Gateways
    • Firewall
    • Anti-Virus-Scanner
  • Windows systems
    • Windows Domain Controllers 2012 R2

2.1. UNIX preparation

Almost each Unix system runs a syslog daemon (e. g. Syslog-NG) that is able to send syslog files to a log management server. We used this feature to transfer log files of the various systems to the ETD. The configuration was quickly done: Just edit the Syslog-NG configuration file, add the destination IP address or host name and the destination port and restart the syslog daemon.

Please note: If one of the systems is part of demilitarized zone (DMZ) or attached to separated network segment, firewall policies, in terms rules for used ip addresses and ports, need to be adapted.


2.2. Windows preparation

Logging in Windows operating system is based on the Microsoft event log protocol. This and that we like to get more system information is why we developed our own agent. The agent transfers Windows event log messages to the ETD instance. The installation is quite easy: Just copy the installer to the target system, run the installer and configure the destination host. After restart, our agent transfers new log entries to the ETD instance.

Please note: If one of the systems is part of demilitarized zone (DMZ) or attached to separated network segment, firewall policies, in terms rules for used ip addresses and ports, need to be adapted.



3. Creating Patterns and Charts

Next step is the creation of specific patterns and charts. In the pilot project, data collection could be started just after one day of log learning process. One day of collected data is a good period of time to start with pattern configuration. It usually includes a complete work cycle of users, including logons, logoffs, etc. It allows to drill into the data within the “Forensic Lab”. The Forensic Lab is used to analyse log data from the past. It also allows the creation of patterns and charts.




We created several charts in the context of the project in the Forensic Lab (see the picture below) .



Here is a little overview with general charts.


  • Emails
    • Emails incoming vs outgoing
    • Emails per user
    • Emails per gateway
    • Attachment kind
    • Attachment count
    • Incoming threat emails
    • Outgoing threat emails
    • Threat emails per user
  • Anti-Virus-Scanner
    • Virus found
    • Virus per user
    • Heuristic
    • Heuristic per user
    • Updates errors
    • Update errors per system
  • Windows Events
    • Logon successful
    • Logon fail
    • New object created
    • Object deleted
    • Object modified
    • Events per domain controller

All these charts and some more are presented on monitor pages. Monitor pages are configurable in terms of scale and refresh interval. For example below you can see the picture that provides an overview of the email gateway.

2 Comments
Labels in this area