Apache Tomcat service failing every minute

[Former Member]

Hi there,

We have been running Business Objects Edge with WebIntelligence for several months now without issue. The server runs Windows 2003 and SQL2000, with Tomcat 5.0.27. There are also 2 other seperate Tomcat 4.0 instances running for other applications. The server configuration has not changed since Edge was implemented, no other software has been installed.

The server is scheduled to reboot every weekend, which doesn't normally cause an issue. However, after last weekend, I am unable to access WebIntelligence. The windows system log shows this every minute:

"The Apache Tomcat 5.0.27 service terminated unexpectedly. It has done this 514 time(s). The following corrective action will be taken in 1000 milliseconds: Restart the service."

I can't see anything at all in the windows application log, and I don't think there's anything relevant in the Tomcat logs. Task manager shows the Tomcat5.exe service consuming a large amount of CPU for a minute or so, then being assigned a new PID as the service fails.

I have tried restarting the server again, but with no luck. Please help as it's a critical business system that we've been without for a day already!

[Former Member · Accepted solution]

When you say that you can't access Webi, do you mean Infoview? or WebI reports? Can you log into infoview? Can you view a report if it doesn't have refresh on open? I'm curious if it fails at a particular point, or if tomcat just didn't start correctly and therefore it keeps trying.

These are just stabs in the dark, but I wonder if this has to do with any of the following:

1. disk space usage. (is tracing turned on? logs will accumulate over time, so it eats up all of your HD and doesn't leave room for swap and cache)

2. a large scheduled report that never ran to completion? (trace the webi job server, or turn it off. to see if you can at least access your webi reports and refresh ad-hoc.)

3. orphaned java process? Tomcat restarting is a strange symptom. May be a stupid question, but have you already tried bouncing your server? Orphaned processes can clog up the port, and cause it to never start up properly.

4. Is another BOBJ deployment conflicting or clustering with yours? Your production server may not have had any changes, but if you duplicated your deployment onto another system in the same network segment, it may try to cluster.

Maybe try turning off all BusinessObjects services such as CMS, WebiReportServer, etc just to see if Tomcat will stay running. Can you access the tomcat homepage and the jsp examples?

[Former Member · Accepted solution]

Hi, thanks for your reply. Infoview was 404ing.

We have however found the problem - the other Tomcat instances (Older BO and VMWare web access) were sharing the same shutdown port. Changing it on the v5 instance sorted the problem - although why this didnt happen before is a mystery!