Resolving Issues with IBM WebSphere Portal 8.0 Default Search Service, Crawlers, and Collections

After installing, clustering, and securing WP 8.0, the search services and collections may need to be recreated, but this process seems fragile. Multiple restarts and multiple attempts to recreate the default search service and the default Portal / WCM collections result in an array of errors that can be difficult to isolate.

Some of the error messages seen in the logs and in the UI when the Search Services are mis-configured:

• Unable to start Portal Search Service

• Unable to create WebScanner

• Failed to parse CrawlerStatus xml

To completely reset the search services, I eventually had to take the following steps:

• Delete the Default Search Service via the Portal Admin console

1. Stop server(s)
2. Delete all files in /usr/wps/WebSphere/wp_profile/PortalServer/CollectionsConfig
3. Start the server(s)

• Update Search Service Credentials

1. Manage Search -> Search Services -> Default Search Service -> Default Search Collection
2. Edit "Portal Content Source", "Security" tab to manually update credentials
   
   Note: I had a problem with my Portal Source due to incorrect protocol in the generated seedlist URI. This was causing a failure with a 401 HTTP code. To fix it I had to switch the URI from HTTPS to HTTP and restart the crawler. 
3. Edit "WCM Content Source", "Security" tab to manually update credentials
4. Run both crawlers
5. Win.

Snippets of the stack traces I encountered in my portal logs during this process:

PortalCollect E com.ibm.hrl.portlets.WsPse.PortalCollectionsService createWebScannerLite EJPJO0032E: Unable to create Webscanner
com.ibm.lotus.search.engine.SearchAdminException: Failed to get crawler status from persistence layer.
...
(the following line was a big clue!)
Caused by: com.ibm.lotus.search.engine.PersistenceException: Failed to parse CrawlerStatus xml.
at com.ibm.lotus.search.engine.PersistenceService.getCrawlerStatus(PersistenceService.java:313)
at com.ibm.lotus.search.engine.SearchAdminService.init(SearchAdminService.java:686)
... 47 more
Caused by: org.xml.sax.SAXParseException: Content is not allowed in trailing section.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at com.ibm.lotus.search.engine.xml.SearchEngineInternalContentHandler.parseCrawlerStatus(SearchEngineInternalContentHandler.java:75)
at com.ibm.lotus.search.engine.PersistenceService.getCrawlerStatus(PersistenceService.java:311)
... 48 more
PortalCollect E com.ibm.hrl.portlets.WsPse.PortalCollectionsService PortalCollectionsService EJPJO0119E: Failed to initialize portal collections services.
com.ibm.hrl.portlets.WsPse.PortalWebScannerException: EJPJO0032E: Unable to create Webscanner
...
Caused by: com.ibm.lotus.search.engine.SearchAdminException: Failed to get crawler status from persistence layer.
at com.ibm.lotus.search.engine.SearchAdminService.init(SearchAdminService.java:688)
at com.ibm.lotus.search.engine.SearchEngineWebScannerLiteImp.init(SearchEngineWebScannerLiteImp.java:180)
at com.ibm.lotus.search.engine.SearchEngineWebScannerLiteImp.setProp(SearchEngineWebScannerLiteImp.java:291)
at com.ibm.hrl.portlets.WsPse.PortalCollectionsService.createWebScannerLite(PortalCollectionsService.java:3669)
... 44 more
Caused by: com.ibm.lotus.search.engine.PersistenceException: Failed to parse CrawlerStatus xml.
at com.ibm.lotus.search.engine.PersistenceService.getCrawlerStatus(PersistenceService.java:313)
at com.ibm.lotus.search.engine.SearchAdminService.init(SearchAdminService.java:686)
... 47 more
Caused by: org.xml.sax.SAXParseException: Content is not allowed in trailing section.
at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
at com.ibm.lotus.search.engine.xml.SearchEngineInternalContentHandler.parseCrawlerStatus(SearchEngineInternalContentHandler.java:75)
at com.ibm.lotus.search.engine.PersistenceService.getCrawlerStatus(PersistenceService.java:311)
... 48 more

Happy Portalling!

IBM Lotus Connections Media Gallery Video Playback Fails with Error #2032

In my Connections 4.0 environment, any attempt to play a video file through the embedded media player would fail with the mysterious code "Error #2032".

Error #2032 on Video Playback

Digging a little deeper with Fiddler uncovered a 404 error for a file called "textLayout_1.0.0.595.swz".  This file does not exist on my Connections server.

Apparently it is common for the Adobe Flash plugin to reach out to Adobe's website for certain supporting localization and support files, but in our environment that wasn't possible so it attempted to find the file on the internal server instead..  Why these files weren't included with Connections is a good question that I never did get an answer to.

I was able to locate the missing file (textLayout_1.0.0.595.swz) on the Adobe web site at this URL: http://fpdownload.adobe.com/pub/swz/tlf/1.0.0.595/textLayout_1.0.0.595.swz

Once I had the file, I was able to follow the procedure documented in the following Connections Customization article to customize my Connections installation with the missing file:  http://infolib.lotus.com/resources/connections/4.0/doc/en_us/ic4_p4.html#t_admin_common_customize_main

Then I tried again... and the problem persisted.  Another fiddler trace revealed that the 404 had moved on to a different .swz file, though, so I repeated the process again for osmf_flex.4.0.0.13495.swz, then again for another .swz... and so on until I had added each of the following to my Connections installation:

• framework_4.0.0.14159.swz
• osmf_flex.4.0.0.13495.swz
• rpc_4.0.0.14159.swz
• sparkskins_4.0.0.14159.swz
• spark_4.0.0.14159.swz
• textLayout_1.0.0.595.swz

Once all of these .swz files were hosted locally within my Connections installation, the Media Gallery started playing video normally!

Happy Portalling!

Note:  Your particular Connections Media Gallery Video Player (sn_media_video_player) may require different versions of these .swz files, but a little searching on the Adobe site should help you find the ones you need.

Installing OmniFind 9.1 to AIX 7.1 - Workaround for IBM XL C++ for AIX, V10.1.02 Error

When attempting to install Omnifind 9.1 to an AIX 7.1 server, I encountered an error stating: "Required AIX Library is Missing" with details telling me that "xlC.aix.rte version must be 10.1.0.1 or later."  I check the AIX machine and found that my xlC.aix.rte  version was actually 12.1.0.0... should be good to go, right?  Not so much!

Digging a little deeper, I found that a script named "getxlcversion61.sh" was being placed in my temporary folder during the installation process.  This script ran and generated a text file containing the xlc version, but the text file was empty and the version check failed.  Turns out that there's a bug in this script causing the problem, and here's how you get around it:

1. Locate or define the temp directory for your installation.  You may define it by setting the IATEMPDIR environment variable to whatever works best for you:
1. export IATEMPDIR=/usr/omnifind/tmp
2. Start Omnifind Installer: 
1. cd <installer dir>/Main
2. ./install.bin
3. Choose Language & click “OK”
3. STOP! DO NOT ACCEPT LICENSE AGREEMENT!
4. Move to the OmniFind 9.1 temp directory on your system and find the getxlcversion61.sh file:
1. find . -name "getxlcversion61.sh"
2. This file is only available while the installer is running.  If you quit the install, it cleans up after itself and removes this file.
3. This file is stored in a temporary, numbered directory within your temp folder.  
5. Update the following line to remove "C/":
1. VERSION=`lslpp -lq xlC.aix61.rte 2>/dev/null | grep "XL C/C++ Runtime" | cut -f16 -d" "` [BAD!]
2. VERSION=`lslpp -lq xlC.aix61.rte 2>/dev/null | grep "XL C++ Runtime" | cut -f16 -d" "` [GOOD!]
6. Save and close getxlcversion61.sh.
7. Accept the license agreement.
8. Ignore the “unsupported OS” warning if installing to AIX 7.1.
9. Continue with the installation... and good luck!

Happy Portalling!