Sun x4450

With a range of options for the Intel Tigerton Quad-core Xeon processors, the Sun x4450 fitted the recommendations for our new postgres database servers. Solaris 10 was chosen by default as the operating system - why complicate the setup by mixing vendors, right?

Well, long story made short: I've ditched Solaris 10 x86_64 in favour of Red Hat Enterprise 5.1. Why? For one thing the Solaris 10 installation was frustratingly complicated.

Performing an installation via serial console requires you to redirect output from the BNC to the System in the BIOS settings. Then the manual states you should choose grub menu's option "ttb". Of course the manual meant "ttyb". But it is actually a typo AND an error: you must choose ttya or all you'll see after boot is "Sun Solaris 5.10" and nothing else.



Once the serial console installation is finished and the system reboots you have another obstacle. The boot process stops at this error message:

Bad PBR sig

Reboot and Select proper Boot device or
Insert Boot Media in selected Boot device and press a key

Googling around (Sun's KB was useless) I found someone with the same problem reporting that a GUI installation didn't show that error. Very well, I configured the firewall to allow Sun's network KVM:

8890 TCP Remote Console
9000 TCP Remote Console
9001 TCP Remote Console
9002 TCP Remote Console
9003 TCP Remote Console
69 UDP TFTP (for firmware upgrades)
161 UDP SNMP (for monitoring)

As the anonymous poster reported, this did solve the problem. Now to install postgres for x86_64. Hmm. Doesn't run - complains about missing libraries. Probably just need to run crle to set-up the location of the 64bit libraries. Where are they, where are they... WTF? There are no x86_64 libraries installed in my Solaris 10 for x86_64. I check around with `file' and yeah: all bloody system libraries are 32 bits.

I confess I didn't try too hard after this. A day later and we had postgres running on RHEL 5.1. And yes, with a 64bit binary and libraries.

To Sun's credit, the hardware looks a beauty, even though you have to assemble the whole thing by yourself: memory cards, SAS card, Fibre Channel card, hard disks and whatever extras you bought. What a pain in the arse. Compared to the other Dell servers with pre-installed RHEL 5.1 we recently bought the Sun experience is pathetic: the Dells where up on the same day versus 1 week for Sun.

Teamsite 6.7.1 SP1 won't start

After installing Teamsite 6.7.1 SP1 on a Solaris 10 machine I tried starting the service to check out the new version. Sadly, the service wouldn't start, dumping a core and printing this message:

[crit] file vhost.c, line 190, assertion "rv == APR_SUCCESS" failed Abort - core dumped /app/teamsite/iw-home/iw-webd/bin/iw.webd start: iwwebd could not be started

In my case, adding "dns" to the "hosts:" line on /etc/nsswitch.conf solved the problem:

hosts: files dns

A little bee tells me that you can also edit /iw-webd/conf/iwwebd.conf.template and change "_default_" on the VirtualHost entry to "*":

<VirtualHost _default_:__IWWEBD_HTTPS_PORT__>

to

<VirtualHost *:__IWWEBD_HTTPS_PORT__>

I didn't try it but that's also the recommendation from an Interwoven's KB article (support account needed).

TCP wrappers: refused connect from ...

I've used inetd + tcp wrappers + netcat a number of times for migration of TCP-based services to a new server. It goes something like this:

1. Get the service running on the new box
2. Point the DNS entry (or IP address of the server on clients) to the new server
3. Stop the service on the old box
4. Enable the redirection using inetd
   

For number 4 and HTTP redirection, an entry like the one below in your /etc/inetd.conf is usually enough:

http stream tcp nowait nobody /usr/bin/tcpd /usr/bin/netcat new-server 80

You then leave the old server running until no more clients connect to it. I do that by inspecting the syslog entries and looking for the netcat redirections. Last time, however, I was seeing these:

Jan 30 14:20:04 old-box netcat[16769]: [ID 947420 mail.warning] refused connect from 189.201.77.65

And sure enough, I started to get complaints that some clients were no longer able to connect to the service. I had left /etc/hosts.allow empty on purpose since there was no need to restrict the service to specific hosts.

After some digging through the tcp wrappers readme, I suspected that the version of tcpd on this SunOS 5.8 (Solaris 8) had been compiled with -DPARANOID. If defined, PARANOID will cause tcpd to reject hosts whose IP address don't resolve to a name (using reverse DNS).

I downloaded the tcp_wrappers source, recompiled without -DPARANOID and installed the newly compiled binary. The refused connection entries were gone from the log and the clients confirmed they were able to reach the server once again.

"Bad user" on Solaris 10 crontab

An account used for an application could not run its cron jobs. In /var/cron/log all I could see was:

! bad user (wondapp) Tue Nov 13 03:23:00 2007

I checked /etc/cron.allow (which didn't exist) and the user's shell in /etc/passwd but the problem turned out to be in /etc/shadow. The user was listed as:

wondapp:*LK*:::::::

This was because a password was never set for it. I just edited it to read:

wondapp:NP:::::::

Which still doesn't make a valid password but doesn't lock-out the account either. Cron jobs for wondapp work now.

Mongrel on Solaris 10, no C compiler

I've installed Coolstack's Ruby package and some gems on a T2000 box, including Mongrel and its dependencies. However, when I tried to start Mongrel it failed with an error:

/opt/csw/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:21:
in `require__': no such file to load -- http11 (LoadError)

It took me a while to figure-out what was wrong because `gem install mongrel' shows a false success message:

Successfully installed mongrel, version 1.0.1

Searching around, some threads pointed to a problem with rbconfig.rb, but this was not my problem; there was a library missing:

# ls -al
/opt/coolstack/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/http11.so
/opt/coolstack/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib/http11.so: No such file or directory

This box doesn't have make, a C compiler or other build tools so, obviously, the installation wasn't able to create http11.so. I just wish it had failed instead of giving me false hope.

I got around it by building it on another machine with the same architecture (sparc, in my case) and compiling tools:

1. Copy to your build host the directory /opt/coolstack/lib/ruby/gems/1.8/gems/mongrel-1.0.1/ext/http11
2. To the same destination, copy /opt/coolstack/lib/ruby/1.8/sparc-solaris2.10/*.h and /opt/coolstack/lib/libruby.so
   
3. On the build host, under the http11 directory, run `make'.
4. You should now have a http11.so on the build host. Copy it to the original machine under /opt/coolstack/lib/ruby/gems/1.8/gems/mongrel-1.0.1/lib

Another option, if you have root access to the build host, is simpy installing mongrel over there and then copying the http11.so library over to the other host. YMMV.

This guy has a neat guide to installing Mongrel on a shared host. It may also help you. G'luck.