Tork Wrench » tip

Workaround for Solaris 10 slow boot on VMware ESX 4.0

Graham — Thu, 01 Jul 2010 22:32:32 +0000

When building a Solaris x64 guest on VMware , the boot sequence when booting from the Solaris DVD seems to take forever. About 5 minutes in fact. Which is really annoying if you are trying to automate installing Solaris and you need to restart 50 times a day. The part I am talking about is after you select to boot Solaris from the grub menu – there is a sequence of dots that comes up until the next Solaris kernel seems to load. Incidentally the guest’s CPU goes to 100% during this sequence, which could be an issue if you are running on a loaded system.

Luckily there is a workaround. Go into the ‘edit settings’ screen for the Solaris guest and click the options tab. Change the Guest Operating System Version to Solaris 10 32 Bit, instead of 64 bit. Then the boot goes more like this:

I made a screen cast of the slow boot as well, but it literally is the same thing as above, except it just goes on for 5 minutes. It could be the most boring video on the internet. The fast boot video above is probably the second most boring video on the internet! It’s just a hard problem to explain in words.

This setting doesn’t mean that you’ll be running a 32 bit OS or anything either – as far as I can tell it doesn’t do anything besides fix the slow boot problem!

Why doesn’t the WCM authoring portlet come up after installing Portal?

Graham — Wed, 30 Jun 2010 01:29:46 +0000

Hey a Q and A post! Let’s hope the Q gets ‘A-ed’.

Peter commented on my post about installing Portal on Ubuntu, and my response got a bit long, so I thought it might might a good post by itself.

Peter writes:

got thru the install but WCM doesnt appear in the admin (for libraries) or UI. can you confirm you can see the WCM stuff in yr portal? and maybe share wpinstalllog.txt with me?

Hey Peter,

When WCM doesn’t come up, it could be a couple of things :

The simplest explanation is that you’ve selected the ‘admin’ install in the setup wizard, so you get a blank Portal out of the box. You can validate this in the PortalServer/wps.properties file, the property WPInstallType will tell you what sort of install you have done. To add WCM to an admin install, run the task:

wp_profile/ConfigEngine/ConfigEngine.sh configure-wcm-authoring -DPortalAdminPwd= -DWasPassword= .

Log into Portal and you should see the WCM stuff under the content tab.

It could also be that you have installed the server version of Portal. Check the file PortalServer/wps.properties and check the value of WPFamilyName . It should read WPFamilyName=content. If it says something else you’ve installed the ‘server’ version of the software – which doesn’t include WCM. This is the worst problem because you basically have to reinstall again – there’s no way to add WCM to a server version. You can see which downloaded files make up each version of Portal in the download documents for each release. The download document for the server version is here, and the content version is here. I hope this isn’t your problem

With 6.0 (not valid for 6.1, but I’ll include it anyway), on Linux you can get a problem where the WCM authoring page and portlet are there, but don’t render properly – the inside of the portlet is just blank. I’ve covered this before in this post.

Let us know how you go Peter!

mod_was_ap22_http.so wrong ELF class: ELFCLASS32

Graham — Tue, 08 Jun 2010 22:39:42 +0000

I’m experimenting with running Apache on our servers instead of IBM HTTP Server. This could be an advantage in terms of security updates – if a particular security vulnerability is fixed in Apache, it is going to be much easier to apply it by typing yum update than going to the IBM site and downloading the latest update and then struggling through the WAS Update Installer.

Anyway, when trying this out, I got this error when starting Apache. This error is pretty straightforward :

Starting httpd: httpd: Syntax error on line 993 of /etc/httpd/conf/httpd.conf: Cannot load /opt/WebSphere70/Plugin/bin/mod_was_ap22_http.so into server: /opt/WebSphere70/Plugin/bin/mod_was_ap22_http.so: wrong ELF class: ELFCLASS32

This simply means that the wrong version of the plugin is installed. I’m using 64 bit linux and 64 bit Apache, but the 32 bit version of the WAS Plugin. Reinstalling the 64 bit version made it work fine.

CWUPI0033E on Solaris 10 when installing WAS

Graham — Thu, 13 May 2010 02:57:58 +0000

Here’s a weird one for you. We were trying to install 6.1.0.3 on a Solaris 10 system to do some tests. The Portal install would fail after about 10 minutes. In the /tmp/wpinstalllog.txt file, it was clear that the problem was due to a failure in the internal WebSphere Application Server install. (When you install Portal, the Portal installer will kick off it’s own silent install of WAS).

The first thing to do when debugging a WAS install problem is to look at the logs in ~/waslogs . These indicated the following problem:

CWUPI0033E:
There is insufficient free disk space on the system:

/opt/WebSphere/AppServer:
Required: 1403 MB
Available: 0 MB

/var/tmp/:
Required: 1403 MB
Available: 0 MB

/opt/.ibm/.nif:
Required: 2 MB
Available: 0 MB

Please ensure that there is enough free disk space
on all required filesystems and restart the installation.

If /var/tmp/ , /opt/WebSphere/AppServer
and /opt/.ibm/.nif are on the same partition,
then the amount of space required is the sum of the space
required on /var/tmp/ , /opt/WebSphere/AppServer and
/opt/.ibm/.nif.

My system had heaps of space on it! Surely the installer wouldn’t even run if there was 0 MB free! The method that the installer used to determine how much disk space was free was failing. But how does the installer figure out how much disk space is free? After lots of poking and prodding around I stumbled on dtrace. I had heard of it before, but never had the opportunity to use it. Dtrace is a mechanism to instrument and probe the tiniest little interactions on a Solaris/BSD/OSX machine. Being so powerful, it has a steep learning curve. This collection of handy dtrace oneliners was really helpful.

I kicked off the WAS install portion of the Portal install and ran this dtrace command in another window.

dtrace -n 'syscall::open*:entry { printf("%s %s",execname,copyinstr(arg0)); }' -o trace.log

It captured each file interaction that occurred when running the install. Luckily the WAS install failed after about 30 seconds, so there wasn’t too much data to wade through.

Here is the dtrace log (trace.log from the command above). Something called gushellsupport.sh is calling df (standard unix disk free command). This must be how the installer determines how much disk space is free. The column on the left is the pid of the install process (which is java) . The library files on the far right are what is being called by each executable; the next column over to the left.

0 44056 open64:entry gushellsupport.s /var/tmp/ismp003/gushellsupport.sh
0 43668 open:entry df /var/ld/ld.config
0 43668 open:entry df /lib/libcmd.so.1
0 43668 open:entry df /lib/libc.so.1
0 43668 open:entry df /usr/dt/lib/nls/msg/C/SUNW_OST_OSCMD.cat
0 43668 open:entry df /usr/lib/locale/C/LC_MESSAGES/SUNW_OST_OSCMD.mo
0 43668 open:entry df /var/ld/ld.config
0 43668 open:entry df /lib/libcmd.so.1
0 43668 open:entry df /lib/libc.so.1
0 43668 open:entry df /etc/mnttab
0 43668 open:entry df /usr/dt/lib/nls/msg/C/SUNW_OST_OSCMD.cat
0 43668 open:entry df /usr/lib/locale/C/LC_MESSAGES/SUNW_OST_OSCMD.mo

This script, gushellsupport.sh, is owned by InstallShield so I can’t publish the contents of it. But it has a diskcheck function in it that relies on ‘/usr/xpg4/bin/df’ which I didn’t have installed. Solaris has many different versions of the same tools that are left behind for backwards compatibility. When installing this system initially, I used the “Core System Support” option in the Solaris install to build a lean, quick machine. Unfortunately it didn’t come with this legacy version of df.

df belongs in a package called SUNWxcu4. To install it, mount your Solaris CD and go to the directory Solaris_10/Product/ . In there, copy the subdirectory ‘SUNWxcu4′ to /var/spool/pkg and run

pkgadd SUNWxcu4

If you rerun the install again it’ll work since gushellsupport.sh is calling the correct version of df. Talk about obscure huh?

Find ConfigEngine tasks in WebSphere Portal 6.1

Graham — Mon, 10 May 2010 00:42:09 +0000

Sometimes when a ConfigEngine task fails, it can be handy to be able to look up in the code to see more information about which task failed. And since all the tasks are written in plain old xml files it’s quite easy to just go in there and look at them. In Portal 6.0 (and before) this was really easy to do, since all the configuration scripts were located in the same place (or two places – WebSphere/PortalServer/config/actions and WebSphere/PortalServer/config/includes) . You could simply grep over all the xml files in these two directories for the task name that was failing and it would return the xml file the task was in.

From 6.1 and on this is more difficult because the configuration scripts are located with each component of Portal, in form like this: /config/includes/.xml . Here’s a oneliner than you can use to search for task names in just these directories. Run it from the PortalServer directory.

#!/bin/bash
find . -wholename */config/includes/*.xml -print0 -type f | xargs -0 grep -l "task-name"

Here’s the same thing in a script.
config-engine-search

If you make any changes to the ConfigEngine scripts, make sure you take a backup first!

CNTR0019E: Initialization failed due to invalid property “supportedMemberTypes”.

Graham — Mon, 14 Dec 2009 06:47:44 +0000

Any problem where WMM (WebSphere Member Manager) doesn’t start properly is a deal breaker for Portal. Once WMM fails, every other component will fail, and you’ll be faced with a nasty 404 message.

After installing 6.0.1.4-WP-IFPK83731.zip and 6.0.1.4-WP-IFPK70263.zip on a machine here I got this error message, and a 404 when I tried to hit portal. These fixes update the version of WMM to the (current) latest version. Here’s what was in my SystemOut.log

[12/14/09 17:31:09:618 EST] 0000002a SystemOut O WMM Implementation Version: WMM5.6_PK83731 (April 1 2009)
[12/14/09 17:31:09:618 EST] 0000002a WSMM Message E com.ibm.ws.wmm.MemberRepositoryManager init() Initialization failed due to invalid property “supportedMemberTypes”.
[12/14/09 17:31:09:653 EST] 0000002a WSMM Message E com.ibm.ws.wmm.objectimpl.MemberServiceBeanBase ejbCreate() com.ibm.websphere.wmm.exception.InitializationException: Initialization failed due to invalid property “supportedMemberTypes”.
[12/14/09 17:31:09:655 EST] 0000002a ExceptionUtil E CNTR0019E: EJB threw an unexpected (non-declared) exception during invocation of method “getConfigurationData”. Exception data: com.ibm.ejs.container.CreateFailureException: ; nested exception is:
java.lang.reflect.InvocationTargetException
at com.ibm.ejs.container.StatelessBeanO.(StatelessBeanO.java:172)
at com.ibm.ejs.container.CMStatelessBeanO.(CMStatelessBeanO.java:58)
at com.ibm.ejs.container.CMStatelessBeanOFactory.create(CMStatelessBeanOFactory.java:40)
at com.ibm.ejs.container.EJSHome.createBeanO(EJSHome.java:913)
at com.ibm.ejs.container.EJSHome.createBeanO(EJSHome.java:1016)
at com.ibm.ejs.container.activator.UncachedActivationStrategy.atActivate(UncachedActivationStrategy.java:83)
at com.ibm.ejs.container.activator.Activator.activateBean(Activator.java:595)
at com.ibm.ejs.container.EJSContainer.preInvokeActivate(EJSContainer.java:3439)
at com.ibm.ejs.container.EJSContainer.preInvoke(EJSContainer.java:2836)
at com.ibm.ejs.container.EJSContainer.preInvoke(EJSContainer.java:2745)
at com.ibm.websphere.wmm.objects.EJSRemoteStatelessMemberService_14d751a3.getConfigurationData(Unknown Source)
at com.ibm.websphere.wmm.objects._MemberService_Stub.getConfigurationData(_MemberService_Stub.java:2292)
at com.ibm.wps.services.puma.SystemWMMAccessBean$39.run(SystemWMMAccessBean.java:906)
at com.ibm.ws.security.auth.distContextManagerImpl.runAs(distContextManagerImpl.java:2771)
at com.ibm.ws.security.auth.distContextManagerImpl.runAsSystem(distContextManagerImpl.java:2651)
at com.ibm.wps.services.puma.SystemWMMAccessBean.getConfigurationData(SystemWMMAccessBean.java:912)
at com.ibm.wps.services.puma.RealmAwareURManager.initRealms(RealmAwareURManager.java:117)
at com.ibm.wps.services.puma.RealmAwareURManager.(RealmAwareURManager.java:103)
at com.ibm.wps.services.puma.PumaServiceImpl.init(PumaServiceImpl.java:215)
at com.ibm.wps.services.Service.init(Service.java:107)
at com.ibm.wps.services.Service.init(Service.java:83)
at com.ibm.wps.services.ServiceManager.createService(ServiceManager.java:400)
at com.ibm.wps.services.ServiceManager.getService(ServiceManager.java:527)
at com.ibm.wps.services.ServiceManager.getService(ServiceManager.java:553)
at com.ibm.wps.services.puma.Puma.(Puma.java:52)
at com.ibm.wps.ac.impl.AccessControlDataManagementServiceImpl.initializeDomainConfig(AccessControlDataManagementServiceImpl.java:885)
at com.ibm.wps.ac.impl.AccessControlDataManagementServiceImpl.reinit(AccessControlDataManagementServiceImpl.java:792)
at com.ibm.wps.ac.impl.AccessControlDataManagementServiceImpl.init(AccessControlDataManagementServiceImpl.java:439)
at com.ibm.wps.services.ServiceManager.createService(ServiceManager.java:400)
at com.ibm.wps.services.ServiceManager.getService(ServiceManager.java:527)
at com.ibm.wps.ac.impl.AccessControlDataManagement.(AccessControlDataManagement.java:41)
at com.ibm.wps.ac.impl.AccessControlServiceImpl.initializeServices(AccessControlServiceImpl.java:138)
at com.ibm.wps.ac.impl.AccessControlServiceImpl.init(AccessControlServiceImpl.java:114)
at com.ibm.wps.services.ServiceManager.createService(ServiceManager.java:400)
at com.ibm.wps.services.ServiceManager.getService(ServiceManager.java:527)
at com.ibm.wps.ac.internal.AccessControlLookupManager.getAccessControlLookup(AccessControlLookupManager.java:37)
at com.ibm.wps.ac.ACManager.getAccessControl(ACManager.java:132)
at com.ibm.hrl.siapi.search.admin.utils.PortletUtils.getPortalAdminUserID(PortletUtils.java:321)
at com.ibm.hrl.siapi.search.admin.portlets.manage.SearchAdminPortletManager.(SearchAdminPortletManager.java:64)
at com.ibm.hrl.siapi.search.admin.portlets.SiapiSearchAdminPortlet.init(SiapiSearchAdminPortlet.java:532)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.init(ServletWrapper.java:320)
at com.ibm.ws.webcontainer.servlet.ServletWrapper.initialize(ServletWrapper.java:1821)
at com.ibm.wsspi.webcontainer.extension.WebExtensionProcessor.createServletWrapper(WebExtensionProcessor.java:141)
at com.ibm.ws.webcontainer.webapp.WebApp.getServletWrapper(WebApp.java:885)
at com.ibm.ws.webcontainer.webapp.WebApp.initializeTargetMappings(WebApp.java:612)
at com.ibm.ws.webcontainer.webapp.WebApp.initialize(WebApp.java:479)
at com.ibm.ws.webcontainer.webapp.WebGroup.addWebApplication(WebGroup.java:123)
at com.ibm.ws.webcontainer.VirtualHost.addWebApplication(VirtualHost.java:146)
at com.ibm.ws.webcontainer.WebContainer.addWebApp(WebContainer.java:940)
at com.ibm.ws.webcontainer.WebContainer.addWebApplication(WebContainer.java:893)

I’m not sure what was updated in WMM for this one, but the solution is simple: install WAS 6.0.2.37 to fix it.

Tips for changing your Portal datasources

Graham — Tue, 04 Aug 2009 05:33:09 +0000

For some reason, we’ve got a test system that is hooked up to a db2 database that is using DHCP. This would be ok if Portal was configured to talk to the hostname of the database server, but it’s using the IP. So the inevitable happened and the IP of the database changed and the Portal server went down.

So the obvious fix is to change the datasource so it’s using the hostname of the database server, rather than the ip (or set a static IP for the database server, but that wouldn’t really require a whole blog post now would it!).

To do this, fire up the WAS admin console and navigate to datasources.

Your list of datasources might be different to mine, depending on your database, but the principles are the same.

Go into each datasource and at the bottom you will see a properties dialog. Change the old IP to a hostname.

change the IP

All of them need to be changed, so go back into each one and change it. Now run the test, by selecting each one and clicking the test connection button.

You should get a message like this:

successful test connection message

Now start Portal. Hmm, I get this when I try to go to the site….

” Error 404: Initialization of one or more services failed. ”

Check the SystemOut.log :

Caused by: com.ibm.websphere.ce.cm.StaleConnectionException: [ibm][db2][jcc][t4][2043][11550] Exception java.net.SocketException: Error opening socket to server /9.185.226.121 on port 50,000 with message: Operation timed out: connect:could be due to invalid address.DSRA0010E: SQL State = null, Error Code = -4,499

at sun.reflect.GeneratedConstructorAccessor64.newInstance(Unknown Source)

at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

at java.lang.reflect.Constructor.newInstance(Constructor.java:522)

at com.ibm.websphere.rsadapter.GenericDataStoreHelper.mapExceptionHelper(GenericDataStoreHelper.java:523)

at com.ibm.websphere.rsadapter.GenericDataStoreHelper.mapException(GenericDataStoreHelper.java:578)

at com.ibm.ws.rsadapter.AdapterUtil.mapException(AdapterUtil.java:2159)

... 11 more

Hang on, the datasource test worked? Why won’t the server start? It looks like it’s using the old database server IP still!

The answer lies in the transaction logs that WAS uses. These binary log files contain XA transactions that might not have been executed yet. You should take care when messing about with these files – they are there for a reason. But on the other hand, the server won’t start now, so we can hardly make things worse, can we ) . Stop the server1 and WebSphere_Portal application servers and go to your wp_profile directory. Delete the contents of the tranlog and recoverylogs directories and try to start the server again.

This message should appear in SystemOut :

[4/08/09 15:23:16:825 EST] 00000012 LogHandle I CWRLS0007I: No existing recovery log files found in C:/WebSphere/wp_profile\tranlog\win61\win61\WebSphere_Portal\transaction\tranlog. Cold starting the recovery log.

[4/08/09 15:23:16:825 EST] 00000012 LogFileHandle I CWRLS0006I: Creating new recovery log file C:/WebSphere/wp_profile\tranlog\win61\win61\WebSphere_Portal\transaction\tranlog\log1.

[4/08/09 15:23:16:841 EST] 00000012 LogFileHandle I CWRLS0006I: Creating new recovery log file C:/WebSphere/wp_profile\tranlog\win61\win61\WebSphere_Portal\transaction\tranlog\log2.

[4/08/09 15:23:16:856 EST] 00000012 LogHandle I CWRLS0007I: No existing recovery log files found in C:/WebSphere/wp_profile\tranlog\win61\win61\WebSphere_Portal\transaction\partnerlog. Cold starting the recovery log.

[4/08/09 15:23:16:856 EST] 00000012 LogFileHandle I CWRLS0006I: Creating new recovery log file C:/WebSphere/wp_profile\tranlog\win61\win61\WebSphere_Portal\transaction\partnerlog\log1.

[4/08/09 15:23:16:856 EST] 00000012 LogFileHandle I CWRLS0006I: Creating new recovery log file C:/WebSphere/wp_profile\tranlog\win61\win61\WebSphere_Portal\transaction\partnerlog\log2.

[4/08/09 15:23:16:903 EST] 00000012 RecoveryManag A WTRN0028I: Transaction service recovering 0 transactions.

Success, now the server will start up!

Eclipse + flash drive = IDE anywhere

Graham — Tue, 03 Mar 2009 10:39:16 +0000

I’m taking a introduction to Java course at night school at the moment. I’ve bought so many ‘Learn Java in 15 minutes’ books and they just sit on the shelf and get dusty. So the class is supposed to make me work at it.

In the class we all are giving nice little windows PCs, and they all have NetBeans 6.5 installed on them. Nothing against NetBeans, but everyone at work uses Eclipse, and I’ve played with Eclipse quite a bit so I didn’t really want to learn NetBeans.

Since the course is held in one of the most treacherous IT environments imaginable (a school!) with every wiseass trying to hack the machines, they are locked down pretty tight. So I didn’t imagine I’d be able to install Eclipse. Also, since you weren’t assigned a specific computer for each session, installing it each time on a different machine wasn’t going to be an option.

I guess you read the title, so know what’s going to come next. I dumped a build of Ganymede onto a crappy flash drive at home and took it to class. Eclipse ran just great off the drive. The particular flash drive has terrible r/w speeds too. I really thought it wouldn’t work well at all. Just make sure you create your workspace on the flash drive too (duh). Now I can work from the same development environment where ever I am. The only caveat I guess is that it has to be the same OS (Wndows, in my case). I’m sure there is some way to launch a Windows build of Eclipse on Linux, but it would be hard to figure out. Too much mucking around with the classpath.