TREX does not index documents

[Former Member]

Now this one is a bit strange. We have created an index on a file system repository that contains a number of different types of files including plain text, MS-Word, Excel and Acrobat PDFs. The queue now has a red status showing a sizeable number of documents failed the preparation, but a few are ready to be transmitted.

We found that none of the MS-Word, Excel and Acrobat documents passed the preparation stage. Anything that passes preparation is either a directory or a plain text file, though this is not consistent either since a number of files that did not pass preparation where als plain text files.

What gives? Am I missing something here? Any hints would be very much appreciated.

[Former Member]

What is your EP/KM/TREX version? The issue with the doc size >10K was resolved with SPS10 I believe.

Nick

[Former Member]

hi Nicholas,

Thanks for your time. The version details are as below:

EP - EP6SP9

KM - 6.0.9.3.0

TREX - 6.1.09.00

But my doubt is that from the above thread it seems like others are able to solve before SP10???

Can you please confirm this information whether this >10K problem is solved in SP10?

Thanks & best regds,

alagammai.

[Former Member]

Hello,

Were you able to solve this problem? If not go to

System Adminsitration>System Configuration>Knowledge Mangement>Configuration>Content Management>Global Services>URL Generator and make sure correct parameters are set in Host field. If you did something else please post it here so that community has access to the solution.

Thanks

Kiran

[Former Member]

Finally - solved the problem. Some valuable clues came from the community so here is the rundown for those that may have a similar issue:

The problem was an authentication issue. First I had to figure out that TREX is using the system user "index_service". A couple of very obscure SAP Notes finally made it click - Notes 671568 and 800895. Though we don't use SiteMinder, both Notes gave me the vital clue:

1. CM is requesting a cookie for the user "index_service" when you create an index.

2. This cookie is passed to the TREX Webserver and stored in the topology.ini file.

3. TREX uses the credentials of the user "index_service" by using the cookie to retrieve the documents for indexing.

Now it comes - If you enable password expiry in the Portal, the password of the user "index_service" will expire like any other users' password, rendering indexing useless until you update the password of the user "index_service" (see Note 800895).

- Nice going SAP -

Hope this helps those that suffer from the same issue.

Best regards,

Wolfgang

[Former Member]

Hi Wolfgang,

I am also facing the problem in indexing of documents on TREX.

As per ur suggestions after referring to SAP Notes i have changed the password of the user index_service. I could not get the second point. i.e setting the Alternate Host address for URL Generator service. I am not getting what has to be the input in that field. In my case we have the Database,Portal Platform,CM as well as TREX on the same Windows Box. The URL for our Portal goes as http://sapnw2:50000/irj/portal. This is also the input which is given in the field "Host" in URL Generator service.

Please if u could guide me.Please send me the details at [email protected]

Thanx

Yatin

[Former Member]

Hi Yatin,

Sorry for a late response - The notes only gave me the clue what may have been wrong. Does your TREX index any documents at all? Try creating a small plain ascii file in the repository (< 10k), the reindex and see if TREX is creating the index.

Also have a look at the default trace file of the portal for errors.

Cheers,

Wolfgang

[Former Member]

Hi this is Jay,

I too face the same problem. I am able to search documents less than 10 k size but not greater than 10k.Above mentioned sapnotes are checked they seem to fine.

Can anybody let me know the exact steps to configure so that it indexes documents >10KB successfully.

Thanks in Advance

Jay

[HarshC]

Hi,

I'm also facing the 10kb problem. I went through the notes and checked the index_service user. The user is active(how do i know if the pwd has expired?). My repository is a file system repository on my local machine(a shared folder) that I'm trying to index. The authentication is the microsoft windows network user/pwd that I've configured in the "network path".

1. What's the next step? Do I need to edit the Alternative Host in URL Generator?

2. If yes, What should I enter there?

3. Where can i find the TREXPrepocessorAlert.trc file?

Looking forward to a reply,

Harsh

PS: The repository is working. The indexing is working for documents under 10kb.

[Former Member]

Hi,

I tried editing the password of the user "index_service". As I don't know the old password for "index_service", I gave a new password for that account. As you told, When I logged in using "index_service", it asked me to change the password and I have done that too. After that I re-indexed my repository. Still only the documents < 10 KB are getting indexed. Any other clues please?

Note:

=====

I am using the company's proxy server. I am not using any other proxy server specifically for the portal. Should I still give that "Alternative URL" entry under the URL Generator Service?

As you are already experienced and have solved this problem, I am greatly expecting your reply in this regard, as it is an urgent requirement for me.

[Former Member]

Hello Wolfgang,

we faced exactly the same problem. To be more precisely, documents with more than 10kb were not indexed. The reason for this is, that doc´s smaller than 10kb are automatically transmitted to TREX by the crawler. For documents bigger than 10kb the crawlers only transmit the URI and then the TREX fetches them by itself.

The URL´s that are sent to TREX are normally built by the URL generator using the host entry. If you have a clustered landscape this might not work, because the Firewall, Load-Balancers, WebServers, or external authentication Systems might cause problems when they are directly called by TREX via the normal portal URL.

In this case you must specify the name of a specific server (e.g. http://server.com:50000) as Alternative Host in the URL Generator Service. But be careful with that setting. A wrong setting might crash your system (happened on our productive layer unfortunately).

If you use SSL for the communication between TREX and portal it even gets more complicated. Fortunately I did not have to deal with that.

Thus, if you have clustered system, be sure that the communication TREX to Portal can take place without problems. Take a look at the trace files of the TREX preprocessor, this should give you a hint on this.

Hope this helps.

Kind regards,

Christopher Höfele

[Former Member]

Thanks for the info. It prompted me to have a look at the TREXPrepocessorAlert.trc (Trace) file.

There are a number of entries that look like:

HTTPHEAD failed for URL http://server:58600/irj/servlet/prt/por

tal/prtroot/com.sap.km.cm.docs/documents/Manual.pdf with Httpstatus 401

This seems to indicate the preprocessor being unable to open the document.

What login is the preprocessor using (if any at all) - I tried to use the link in a browser and get a login prompt username/password.

Wolfgang

[Former Member]

Thanks Christopher,

your response gave me a good clou what is happening. Indeed I find that trex indexes plain text documents of < 10 K but fails on the ones that are larger.

You mentioned to specify an Alternative Host in the URL Generator Service ... could you maybe point me to where I find this parameter? Is it on the TREX server or in the Portal?

Thanks so far for your great help.

Wolfgang

[Former Member]

Hi Wolfgang,

the alternative host is a parameter of the global service "URL Generator". In the portal just browse to System Adminsitration>System Configuration>Knowledge Mangement>Configuration>Content Management>Global Services>URL Generator

There you can specify the alternative host, but as mentioned, please be careful with this setting.

Regards,

Christopher