- SAP Managed Tags:
SAP SMP 3.0 MBO Log Correlation Command Line Utility consolidate-log.bat
Purpose
SAP SMP 3.0 includes utility to help collect the error message from all the logs that the SMP 3.0 server users. It can be a daunting task to go line by line in each log file. It can often be a very time consuming task. With the consolidate-log utility can make the collection of the error message from the very fast and very simple In this write up we will walk thru the necessary steps to collect all the error from all the logs at the same time. this is also a very importaint part of the product support tools to analize customers logs.
Required Software
Assumption
Issue:
SAP SMP 3.0 server is throw error message in the logs. The administrators need a quick way to collect all the message from all the logs in order to isolate the issue
Solution
Run the consolidate-logs.bat utility.
Syntax
consolidate-logs.bat [options]
Where [options] can be
[-h|-help]
[-v |-verbose]
[-w|-windows app|smp|sys|none app-name]
[-a|-application app-name]
[-c|-connection con-name]
[-d|-domain domain-name]
[-u|-user user-name]
[-f|-from yyyy-mm-ddTthh:mm:ss ]
[-t|-to yyyy-mm-ddTthh:mm:ss]
[-q|-query jms-selector]
[-m|-match "count op N"]
[-e|-error jms-selector]
[-e|-error tool-warnings]
[-n|-network shared-path]
[-x:rs| rs-host-logs-dir]
[-x:sa| sa-host-logs-dir]
[-i|-input input-file]
[-o|-output output-file]
[-r|-rows maximum-rows]
[-s|-size maximum-size]
Steps
Usage: consolidate-logs [options]
Options:
[-h|-help] (display this usage information)
[-v|-verbose] (display names of all opened input sources)
[-w|-windows app|smp|sys|none] (default is all system + application events)
[-a|-application <app-name>] (exclude entries for different applications)
[-c|-connection <con-name>] (exclude entries for different connections)
[-d|-domain <domain-name>] (exclude entries for different domains)
[-u|-user <user-name>] (exclude entries for different users)
[-f|-from <from-timestamp>] (exclude entries before this time)
[-t|-to <to-timestamp>] (exclude entries after this time)
[-q|-query <jms-selector>] (exclude entries that do not match query)
[-m|-match "count <op> N"] (exit status is 1 if count not as expected)
[-e|-error <jms-selector>] (exit status is 1 if any errors are found)
[-e|-error tool-warnings] (exit status is 1 if this tool has warnings)
[-n|-network <shared-path>] (shared platform path for cluster members)
[-x:rs <rs-host-logs-dir>] (directory path for Relay Server log files)
[-x:sa <sa-host-logs-dir>] (directory path for SQL Anywhere log files)
[-i|-input <input-file>] (input from a file generated by this tool)
[-o|-output <output-file>] (use .zip file extension for zipped output)
[-r|-rows <maximum-rows>] (multiple output files if limit is exceeded)
[-s|-size <maximum-size>] (multiple output files if limit is exceeded)
Timestamp formats:
yyyy-mm-ddTthh:mm:ss, e.g. 2000-03-22T12:34:56
yyyy-mm-ddTthh:mm, e.g. 2000-03-22T12:34
yyyy-mm-ddTthh, e.g. 2000-03-22T12
yyyy-mm-dd, e.g. 2000-03-22
yyyy-mm, e.g. 2000-03
yyyy, e.g. 2000
Message selectors:
A subset of SQL92 expression syntax, see JMS "Message Selectors" in:
http://docs.oracle.com/javaee/6/api/javax/jms/Message.html
Valid property names are the column header names found in the output file
Output sizes:
number[unit], where optional unit is k/kb/m/mb/g/gb
(if unit is omitted, then size is measured in bytes)
e.g. 1gb = 1000mb = 1000000kb = 1000000000
Examples:
-user fred@example.com
-from 2000-01-01
-to 2000-01-01T12:34:56
-query "(level='WARN' and subsystem='SCC'") or level='DEBUG'"
-match "count = 0" (any JMS selector with 'count' property)
-error "level in ('ERROR', 'FATAL')"
-network C$\SAP\MobilePlatform (all cluster mebers must share the path)
-x:rs \\my-rs-host\C$\inetpub\wwwroot\ias_relay_server\server
-input old-consolidated.log (regular input sources will not be read)
-output new-consolidated.zip (use .log extension for unzipped output)
-rows 1000000 (recommended if using Excel 2010+ to view logs)
-size 100mb (recommended if using text editor to view logs)
-windows smp (exclude system events and non-SMP applicaton events)
As we can see from the out the utility has a lot of option to consider here and and is not a simple utility to combine errors from logs
Note: Since the Windows shell has special treatment for the percent character, you need to double or escape each ‘%’ in the JMS selector based “-query” option.
For example, to find all the log entries containing the word “system”, use the following syntax:
-query “message like ‘%%system%%’”
Examples
Output all the server logs in a non-clustered environment to the standard output
C:\>consolidate-logs.bat
Save the consolidated logs in a zip file excluding entries containing users different to “wsmith@example.com”
C:\>consolidate-logs.bat -output new-consolidated.zip -user wsmith@example.com
Output logs entries that happened between January 1st, 2016 and February 1st, 2016 12:34:56
C:\>consolidate-logs.bat -from 2016-01-01 -to 2016-02-01T12:34:56
Output logs for a given level/subsystem, and include logs from a relay server
C:\>consolidate-logs.bat -query "(level='WARN' and subsystem='SCC'") or level='DEBUG'"
-x:rs \\my-rs-host\C$\inetpub\wwwroot\ias_relay_server\server
Invoke tool programmatically to get server logs older than 2016 and expect exit status is 1 if no log entry was returned
C:\>consolidate-logs.bat –to 2016-12-31 -match "count = 0"
Invoke tool programmatically and expect exit status is 1 if any errors are found
C:\>consolidate-logs.bat -error "level in ('ERROR', 'FATAL')"
Output logs from a cluster environment where all cluster members share the path C:\SAP\MobilePlatform
C:\>consolidate-logs.bat -network C$\SAP\MobilePlatform (all cluster members must share the path)
Use existing file as input (regular input sources are not read) to create a new consolidated log file no larger than 100mb
– Limiting the output size is recommended if using a text editor to view the logs.
Note that potentially multiple output files are generated, with “new-consolidated.log” being used as a base name.
For example, “new-consolidated-001.log”, “new-consolidated-002.log", etc.
C:\>consolidate-logs.bat -input old-consolidated.log -size 100mb
Save output unzipped (use .log extension for unzipped output) and retrieve up to 1 million rows Limiting the number of rows is recommended if using a spreadsheet program to view logs:
C:\>consolidate-logs.bat -output consolidated.log -rows 1000000
Output server logs excluding system events and non-SMP application events
C:\>consolidate-logs.bat -windows smp.
Here is an exampled of test run using the command line
C:\SAP\MobilePlatform3\MR30\Servers\UnwiredServer\bin>consolidate-logs.bat -output old-consolidated.log
Figure 1.
If you note the amount of entries is very large after collecting the data you may find the file too large to edit.
This happens a lot when the logs are very large and have not been purged roll rolled over.
We can then try and limit the amount of row we collect in file and we can create mutable output file
to easily manage and edit. I changed the command line to the follow note the –rows 10000 even
thou that is a small number it does help when editing the several files that are created.
for example
C:\SAP\MobilePlatform3\MR30\Servers\UnwiredServer\bin>consolidate-logs.bat -output consolidated.log -rows 10000
Here is an example of the output file from the above command.
Figure 2.
Keep in mind that you can command many option to fit the output of your liking.
And this is KEY to this utility you can create a combination to suite your needed to collect the exact data you are looking for ERROR WARRNING and so on how many rows in each file customize and look a particular user
Summary
Command line utility to consolidate, correlate, and filter the content of the server logs. The utility is limited to the SAP Mobile Server and cluster logs. The consolidate-logs.bat utility is typically used by the SAP Mobile Platform support team, but can be used by administrators to collect data for analysis or reporting to the support team. The command options allow administrator to customize the out for reporting or reviewing issues they may encounter. in addition the output is very useful when reporting issue to SAP product support as this will help speed up the support process.
Purpose
SAP SMP 3.0 includes utility to help collect the error message from all the logs that the SMP 3.0 server users. It can be a daunting task to go line by line in each log file. It can often be a very time consuming task. With the consolidate-log utility can make the collection of the error message from the very fast and very simple In this write up we will walk thru the necessary steps to collect all the error from all the logs at the same time. this is also a very importaint part of the product support tools to analize customers logs.
Required Software
- SAP SMP 3.0.
Assumption
- Local administrator account.
- SMP administrator account
Issue:
SAP SMP 3.0 server is throw error message in the logs. The administrators need a quick way to collect all the message from all the logs in order to isolate the issue
Solution
Run the consolidate-logs.bat utility.
Syntax
consolidate-logs.bat [options]
Where [options] can be
[-h|-help]
[-v |-verbose]
[-w|-windows app|smp|sys|none app-name]
[-a|-application app-name]
[-c|-connection con-name]
[-d|-domain domain-name]
[-u|-user user-name]
[-f|-from yyyy-mm-ddTthh:mm:ss ]
[-t|-to yyyy-mm-ddTthh:mm:ss]
[-q|-query jms-selector]
[-m|-match "count op N"]
[-e|-error jms-selector]
[-e|-error tool-warnings]
[-n|-network shared-path]
[-x:rs| rs-host-logs-dir]
[-x:sa| sa-host-logs-dir]
[-i|-input input-file]
[-o|-output output-file]
[-r|-rows maximum-rows]
[-s|-size maximum-size]
Steps
- Open a command prompt
- cd to C:\SAP\MobilePlatform3\MR30\Servers\UnwiredServer\bin
- From the command prompt execute the following command
- C:\SAP\MobilePlatform3\MR30\Servers\UnwiredServer\bin>consolidate-logs.bat -h
Usage: consolidate-logs [options]
Options:
[-h|-help] (display this usage information)
[-v|-verbose] (display names of all opened input sources)
[-w|-windows app|smp|sys|none] (default is all system + application events)
[-a|-application <app-name>] (exclude entries for different applications)
[-c|-connection <con-name>] (exclude entries for different connections)
[-d|-domain <domain-name>] (exclude entries for different domains)
[-u|-user <user-name>] (exclude entries for different users)
[-f|-from <from-timestamp>] (exclude entries before this time)
[-t|-to <to-timestamp>] (exclude entries after this time)
[-q|-query <jms-selector>] (exclude entries that do not match query)
[-m|-match "count <op> N"] (exit status is 1 if count not as expected)
[-e|-error <jms-selector>] (exit status is 1 if any errors are found)
[-e|-error tool-warnings] (exit status is 1 if this tool has warnings)
[-n|-network <shared-path>] (shared platform path for cluster members)
[-x:rs <rs-host-logs-dir>] (directory path for Relay Server log files)
[-x:sa <sa-host-logs-dir>] (directory path for SQL Anywhere log files)
[-i|-input <input-file>] (input from a file generated by this tool)
[-o|-output <output-file>] (use .zip file extension for zipped output)
[-r|-rows <maximum-rows>] (multiple output files if limit is exceeded)
[-s|-size <maximum-size>] (multiple output files if limit is exceeded)
Timestamp formats:
yyyy-mm-ddTthh:mm:ss, e.g. 2000-03-22T12:34:56
yyyy-mm-ddTthh:mm, e.g. 2000-03-22T12:34
yyyy-mm-ddTthh, e.g. 2000-03-22T12
yyyy-mm-dd, e.g. 2000-03-22
yyyy-mm, e.g. 2000-03
yyyy, e.g. 2000
Message selectors:
A subset of SQL92 expression syntax, see JMS "Message Selectors" in:
http://docs.oracle.com/javaee/6/api/javax/jms/Message.html
Valid property names are the column header names found in the output file
Output sizes:
number[unit], where optional unit is k/kb/m/mb/g/gb
(if unit is omitted, then size is measured in bytes)
e.g. 1gb = 1000mb = 1000000kb = 1000000000
Examples:
-user fred@example.com
-from 2000-01-01
-to 2000-01-01T12:34:56
-query "(level='WARN' and subsystem='SCC'") or level='DEBUG'"
-match "count = 0" (any JMS selector with 'count' property)
-error "level in ('ERROR', 'FATAL')"
-network C$\SAP\MobilePlatform (all cluster mebers must share the path)
-x:rs \\my-rs-host\C$\inetpub\wwwroot\ias_relay_server\server
-input old-consolidated.log (regular input sources will not be read)
-output new-consolidated.zip (use .log extension for unzipped output)
-rows 1000000 (recommended if using Excel 2010+ to view logs)
-size 100mb (recommended if using text editor to view logs)
-windows smp (exclude system events and non-SMP applicaton events)
As we can see from the out the utility has a lot of option to consider here and and is not a simple utility to combine errors from logs
Note: Since the Windows shell has special treatment for the percent character, you need to double or escape each ‘%’ in the JMS selector based “-query” option.
For example, to find all the log entries containing the word “system”, use the following syntax:
-query “message like ‘%%system%%’”
Examples
Output all the server logs in a non-clustered environment to the standard output
C:\>consolidate-logs.bat
Save the consolidated logs in a zip file excluding entries containing users different to “wsmith@example.com”
C:\>consolidate-logs.bat -output new-consolidated.zip -user wsmith@example.com
Output logs entries that happened between January 1st, 2016 and February 1st, 2016 12:34:56
C:\>consolidate-logs.bat -from 2016-01-01 -to 2016-02-01T12:34:56
Output logs for a given level/subsystem, and include logs from a relay server
C:\>consolidate-logs.bat -query "(level='WARN' and subsystem='SCC'") or level='DEBUG'"
-x:rs \\my-rs-host\C$\inetpub\wwwroot\ias_relay_server\server
Invoke tool programmatically to get server logs older than 2016 and expect exit status is 1 if no log entry was returned
C:\>consolidate-logs.bat –to 2016-12-31 -match "count = 0"
Invoke tool programmatically and expect exit status is 1 if any errors are found
C:\>consolidate-logs.bat -error "level in ('ERROR', 'FATAL')"
Output logs from a cluster environment where all cluster members share the path C:\SAP\MobilePlatform
C:\>consolidate-logs.bat -network C$\SAP\MobilePlatform (all cluster members must share the path)
Use existing file as input (regular input sources are not read) to create a new consolidated log file no larger than 100mb
– Limiting the output size is recommended if using a text editor to view the logs.
Note that potentially multiple output files are generated, with “new-consolidated.log” being used as a base name.
For example, “new-consolidated-001.log”, “new-consolidated-002.log", etc.
C:\>consolidate-logs.bat -input old-consolidated.log -size 100mb
Save output unzipped (use .log extension for unzipped output) and retrieve up to 1 million rows Limiting the number of rows is recommended if using a spreadsheet program to view logs:
C:\>consolidate-logs.bat -output consolidated.log -rows 1000000
Output server logs excluding system events and non-SMP application events
C:\>consolidate-logs.bat -windows smp.
Here is an exampled of test run using the command line
C:\SAP\MobilePlatform3\MR30\Servers\UnwiredServer\bin>consolidate-logs.bat -output old-consolidated.log
Figure 1.
If you note the amount of entries is very large after collecting the data you may find the file too large to edit.
This happens a lot when the logs are very large and have not been purged roll rolled over.
We can then try and limit the amount of row we collect in file and we can create mutable output file
to easily manage and edit. I changed the command line to the follow note the –rows 10000 even
thou that is a small number it does help when editing the several files that are created.
for example
C:\SAP\MobilePlatform3\MR30\Servers\UnwiredServer\bin>consolidate-logs.bat -output consolidated.log -rows 10000
Here is an example of the output file from the above command.
Figure 2.
Keep in mind that you can command many option to fit the output of your liking.
And this is KEY to this utility you can create a combination to suite your needed to collect the exact data you are looking for ERROR WARRNING and so on how many rows in each file customize and look a particular user
Summary
Command line utility to consolidate, correlate, and filter the content of the server logs. The utility is limited to the SAP Mobile Server and cluster logs. The consolidate-logs.bat utility is typically used by the SAP Mobile Platform support team, but can be used by administrators to collect data for analysis or reporting to the support team. The command options allow administrator to customize the out for reporting or reviewing issues they may encounter. in addition the output is very useful when reporting issue to SAP product support as this will help speed up the support process.
