Question Re: Slow performance from BOBJ and Web Intelligence in Technology Q&A

Slow performance from BOBJ and Web Intelligence

keithfisherdsc — Thu, 22 Aug 2019 11:18:19 GMT

Hello,

I have an issue with a client where performance for the Launchpad, opening Web Intelligence reports and opening queries in WebI is slow. It can take 30 seconds or more from the authentication screen to the documents view at logon, over 3 minutes to open a large report for editing, and almost a minute to open the query panel and view WebI queries.

The problem system is 4.2 SP6 running on a Windows Server 2016 virtual server, with dedicated resource. It has 8 cores of AMD Opteron Processor 6172 at 2.1 GHz and 24GB RAM. I am using Chrome v76 on the desktop of the server itself.

At first logon of a day the logon process takes up to a minute, with this output in the Chrome console:

This 'Waiting (TTFB)' (Time to first byte) is consistent for the first logon on a day - subsequent logins are quicker.

Then when I come to open a large report, albeit one where the data has been purged from I get these timing from the Chrome console:

52 seconds to download the file

27 seconds wait time again - after which the report first page has come to the screen and the structure/formatting is visible.

then the left hand pane to display the universe available objects takes another 3.7 minutes to become visible.

I accept that this report is very large - 19 queries, 21 report tabs, dozens of variables. However I loaded the same report into another system and had these comparative timings:

logon wait time is 5 seconds vs 48 in client system
opening the document: viewDocument.jsp total time is 15.78 seconds vs 52.46 on client's machine - the major difference is that on the client's server this is mainly in download time, on the test machine it is all in TTFB.
opening the document: report.jsp total time is 7.49 seconds vs 27.48 on client's server
waiting for left panel: ajaxIncObjects total time 39.21 seconds vs 3.7 minutes on client's server.

For the test machine timings I am using Chrome v76 not on the server desktop but on my own laptop. The server is Windows 2012 R2 running Intel Core i7-4790 @ 3.6GHz (8 cores) with 32 GB RAM on a cloud hosted VM.

These timings show some obvious findings - that a faster server returns data more quickly even over a wider network. However the logon wait time far exceeds anything I've seen at other clients, and the general slow timings for things like opening of query panels, moving between queries, opening smaller reports etc etc suggests something is not right.

Can anyone help with suggestions of what might be wrong, where the extreme latency is coming from?

thanks

Keith

Re: Slow performance from BOBJ and Web Intelligence

denis_konovalov — Thu, 22 Aug 2019 11:53:11 GMT

1. 24gb ram is below minimum required for BI4.x
2. disk I/O is often the first and major bottleneck and contributor to slow systems in virtual environments. And slow disk i/o will make BI crawl.
3. You can run end to end trace of the affected workflows and see where time is being spent within BOE process flows (KBA 1861180)

Re: Slow performance from BOBJ and Web Intelligence

Former Member — Thu, 22 Aug 2019 12:56:52 GMT

Is BO installed only in one server? Tomcat, BO services etc? How many users in the system when you are seeing the latency?

What is the Tomcat Configuration? Allocated memory for Tomcat ?

Is your APS split? Is this issue happening now or since the beginning? Can you check if there is bottle neck in the DB side to refreshing the reports?

Re: Slow performance from BOBJ and Web Intelligence

keithfisherdsc — Thu, 22 Aug 2019 14:16:06 GMT

Hi Denis,

thanks for the reply 🙂

The PAM suggests 16GB is the minimum RAM required for BI4.2 and we do have some clients with 16-24GB whose systems run quicker than this.

I've asked the questions about disk IO and am waiting for a reply from IT on that - also to see what other VMs might exist on the host machine

I'll check that SAP note and see what I can glean from it, thanks

Keith

Re: Slow performance from BOBJ and Web Intelligence

keithfisherdsc — Thu, 22 Aug 2019 14:20:34 GMT

Hi Jawahar,

thanks for the reply 🙂

it is a single server environment, yes. The number of users fluctuates but is normally around 10-20 max. The latency does not seem to change depending on the users in the system though,it is consistent.

Tomcat has a max memory pool of 6144, and a MaxMetaspaceSize of 2048M - upping these settings does seem to have reduced the need to restart Tomcat so often, but has not sped up the performance

The APS is split in accordance with our normal configs for a server this size. There is no problem with the database size as data refreshes are actually quite quick, it is the opening and editing of reports that takes so much longer than it should

thanks

Keith

Re: Slow performance from BOBJ and Web Intelligence

Joe_Peters1 — Mon, 26 Aug 2019 12:55:13 GMT

I'd recommend checking the CMS database. BO generates hundreds of queries, and high latency there can cause significant delays. If that's not a problem, you can do an end-to-end trace and submit the results to tech support for analysis.

Re: Slow performance from BOBJ and Web Intelligence

former_member806888 — Tue, 28 Mar 2023 19:36:24 GMT

Hi Keith,

We are experiencing the same issue in BOBJ 4.3 SP2. Please let me know if you are able to identify the root cause for it. Appreciate any pointers to resolve this issue.

Thanks,

Das

Re: Slow performance from BOBJ and Web Intelligence

keithfisherdsc — Thu, 30 Mar 2023 19:07:27 GMT

Hi Das,

it was a while ago now but I think the client eventually solved their own problem when they moved datacenters, which implies it was down to their infrastructure - possibly disc IO as Denis first suggested, or network latency/bandwidth.

Sorry I don't have anything more concrete to suggest

Keith