Category Archives: System Administration

PHP 5.2 and Drupal 4.7.4 don’t work together

Posted on by
Reply

While doing a software upgrade on a server today I ran into some problems. Apparently Drupal 4.7.4 and PHP 5.2.1 don’t work together.

After upgrading PHP, Apache and APC cache I couldn’t get Drupal to work on all 7 sites on one machine. I could login to any drupal site as the admin and get shown my user page, but when trying to do anything at all as an authenticated user it considered me logged out.

I initially thought APC had something to do with it and began to disable it. That didn’t fix it. On a haunch and after a lot of reading with no answer I decided to install a lesser version of PHP and that fixed it. PHP 5.1.1 was laying around and did the trick. Unfortunately I needed 5.2 to fix a mem leak!

I decided to update the Drupal sites to the latest version first to see if that helps. Lo and behold when I visit the Drupal download page I am greeted with:

PHP 5.2 compatibility is only available from Drupal 4.6.11 / 4.7.5 / 5.x.

That pretty much cinches it. I need PHP 5.2 to fix a memory leak so I’ll take time to upgrade to Drupal 5 now.

Be warned, if you are running Drupal 4.7.4 sites and attempt to upgrade to PHP 5.2 you may be faced with being unable to login or navigate as an authenticated user in Drupal. Downgrade to PHP 5.1, upgrade to > Drupal 4.7.4 and then continue your upgrade to PHP 5.2 and you should be alright.

Apache, Postgres and Drupal on a VPS

Posted on by
Reply

I really would prefer to have my own server but sticking a box in colo is expensive. Where I live, getting access to the physical colo space would be nearly impossible too. As a result I run on a VPS. Unfortunately VPS has some horrible limitations depending on who is on the box with you.

Recently I decided to move my personal blog off of b2evolution and stick it on Drupal. Too bad drupal is such a resource hog. Most CMS and blog software is though and it is really hard to find a minimalized, optimized blog software that uses any form of caching. Today, it hit the skids and my patience hit the wall. Argh!

I was converting my personal blog by hand because I only have about 30 entries so it didn’t pay to write a conversion script. Everytime I hit the ‘post’ button in Drupal I wound up with a blank screen, could not connect or worse, the results of any command in SSH terminal window showed a “could not allocate memory”. As a result, I had to do some fast tuning of something because I had to reboot the server after every blog post!

I chose to tackle Apache first because they have an Apache Performance Tuning Guide that helps a bunch. Postgres, well I’m running an old version that I really need to upgrade before I tackle configuration and optimization of it. That’s not a 30 minute job.

VPS, Apache and Low Memory

VPS has a low memory for sure. Even though you can sometimes burst more, in general it is pretty low. The very first thing in the Apache performance tuning guide is tackling memory.


You can, and should, control the MaxClients setting so that your server does not spawn so many children it starts swapping. This procedure for doing this is simple: determine the size of your average Apache process, by looking at your process list via a tool such as top, and divide this into your total available memory, leaving some room for other processes.

Using top and sorting by thread using ‘h’ I am able to see that the average Apache processes is using WAY TOO MUCH memory at 25M a piece – and 10 processes running. I don’t have time to tune the size now so I’ll tune the number of servers using very simple configuration parameters. Since we are using MPM Prefork, the directives can be found in extra/httpd-mpm.conf file under mpm_prefork_module.

Since I am supposed to be guaranteed 256M memory burstable to 1G I’ll optimize to the lower number. 256M / 25M is 10. Not including room for other processes. The current setting is 150!

From:


StartServers 5
MinSpareServers 5
MaxSpareServers 10
MaxClients 150
MaxRequestsPerChild 0

To:


StartServers 2
MinSpareServers 2
MaxSpareServers 5
MaxClients 10
MaxRequestsPerChild 0

I also changed it to only start 2 servers, only keep 2 spare servers instead of 5, and only allow 10 clients instead of 150. This will essentially create a queue if anyone is waiting but it shouldn’t dip into swap space and it will save a bunch of memory usage. This will of course be monitored. Once time permits and I am able to minimize the size of each Apache2 process (the other factor) then I will revisit this and likely increase the MaxClients accordingly.

Upgrading Postgres in parallel with your old version

Posted on by
Reply

Today I got caught up in PostgreSQL. A feature didn’t work as planned and when the mailing list responded to my plea for help it turned out the feature didn’t work fully in my version of Postgres, v.8.1.4.

Sad to say, I haven’t compiled a pgsql version for 13 months. I got so lazy that the last time I installed on my dev machine I just used the package that came with Kubuntu. Now I’m in a bind. I have a production machine and a dev machine to backup, upgrade and restore.

I learned one important lesson in this and that is to name your postgres installation directory the same as the current version. That kind of goes against my best judgement but it means that you can run two Postgres instances beside each other. Why would you want to do that? Take this line from the upgrade instructions:
Today I got caught up in PostgreSQL. A feature didn’t work as planned and when the mailing list responded to my plea for help it turned out the feature didn’t work fully in my version of Postgres, v.8.1.4.

Sad to say, I haven’t compiled a pgsql version for 13 months. I got so lazy that the last time I installed on my dev machine I just used the package that came with Kubuntu. Now I’m in a bind. I have a production machine and a dev machine to backup, upgrade and restore.

I learned one important lesson and that is to name your postgres installation directory the same as the current version. That kind of goes against my best judgement but it means that you can run two Postgres instances beside each other. Why would you want to do that? Take this line from the upgrade instructions:

To back up your database installation, type:

pg_dumpall > outputfile

…

To make the backup, you can use the pg_dumpall command from the version you are currently running. For best results … try to use the pg_dumpall command from [your new version of] PostgreSQL … since this version contains bug fixes and improvements over older versions. While this advice might seem idiosyncratic since you haven’t installed the new version yet, it is advisable to follow it if you plan to install the new version in parallel with the old version. In that case you can complete the installation normally and transfer the data later. This will also decrease the downtime.

Try to backup the existing database from the new version, which hasn’t been installed and can’t be installed until the old is backed up. That is, use the binary from your newly compiled version and run it on the old backend.

Seems like a problem, but if you run the new version in parallel you have a couple of benefits as I see it.

1. You don’t have to take your database down during upgrade! Once you are sure things are working on your new version then you simply flick the switch and it is nearly transparent. Don’t be like the company down the block who took their site down for 2 days to perform a db upgrade :)

2. If things go horribly wrong on your upgrade then you should still have the existing data, database and files to work from and find out the problem.

Keep these things in mind:

1. Change the install path by specifying it with the –prefix switch during ./configure Normally it installs to /usr/local/pgsql but because I already have a server installed and running there I chose to specify –prefix=/usr/local/pgsql-8.2.3

2. You can literally run it in parallel but change the port on the new server. In this fashion you can actually pipe the output of pg_dumpall to the input of the new server eliminating the SQL plain text file usually produced.

Make sure you block write access to the old version while performing the dump. When you are sure that everything dumped correctly you can shutdown the old server and start the new one on the old port.

There you have it. A nearly seamless database upgrade. A couple good reasons for running your new Postgres database upgrade in parallel with your current version.

DDoS DNS leads to User Interface Woes

Posted on by
Reply

When I moved my server a little more than a year ago I decided that I would try to get away from System Admin tasks by offloading services where possible. The first one to go was DNS. DNS is simple to manage, but just one more thing to look after. I had bought into the marketing hype about having off-site DNS so I decided to try a third party DNS. Unfortunately, having a service that many people rely on for ALL their services is just a target for attackers. After several DDos attacks with at least two providers I decided it was time to manage my own DNS again.

After my DNS was all setup I decided to retain those other third party people for my DNS secondary. After nearly a week of frustration I discovered one very important thing – not everyone that CAN program a computer SHOULD program a computer.
When I moved my server a little more than a year ago I decided that I would try to get away from System Administration tasks by offloading services where possible. The first one to go was DNS. DNS is simple to manage, but just one more thing to look after. I had bought into the marketing hype about having off-site DNS so I decided to try a third party DNS. Unfortunately, having a service that many people rely on for ALL their services is just a target for attackers. After several DDoS attacks with at least two providers I decided it was time to manage my own DNS again.

After my DNS was all setup I decided to retain those other third party people for my DNS secondary. After nearly a week of frustration I discovered one very important thing – not everyone that CAN program a computer SHOULD program a computer.

Sure, you know your network services, as much as anyone really can understand them, and you’ve administered a box for a while. You’ve got a great idea for a free online service so you set out at hacking away a user interface to allow your potential users to connect with your good idea. The problem is, you probably aren’t a programmer. Sure, you can figure out the semantics of a language pretty easy because you already know Shell script, but that certainly doesn’t make you an accomplished application developer.

How do I know? Because I’ve used your applications. Here’s some things that frustrated me:

- inaccurate error messages
- no error messages on an error
- the WRONG error message for an error
- a blank screen instead of an error message
- your PHP path and MySQL info echoed to screen on an error
- assuming the error was MY fault

Of course that is just a mild sampling that kept me from using a service properly in the past week. We really don’t know what goes on behind the scenes. When you built your application was there a white board involved? Did you have developer meetings? Is there a printout of a schema? Or did you just sit down and start typing until it looked good? Whatever way you chose, it didn’t work, it didn’t work well, or it just plain failed.

Either way, not everyone is a programmer. Too bad that scripting for the Internet is so easy that everyone has had a crack at it. Most of those people would probably get queasy if they were asked to rewrite that application in the C programming language.

We are taught to examine and verify user input but that makes us assume that all problems are caused by the user. Make sure you put in checks on your own code. Check for database connects, rows returned, function returns, etc. This way you can debug your own code as well. I used to put secret checks into my code that would either log or email me on an unexpected problem that didn’t throw an error.

As you can guess, I quit using this service. It was too frustrating to setup and I could never be sure that the values I input in the user interface were accepted. Trying to bypass the inconsistencies was remnant of hacking on a TRS-80. I think we can all learn a good lesson about user interface design when faced with problems we see on other sites. The key to a better programmer is remembering what that was, and then applying it to your sites. Sometimes you can be too close to your own project to see them.

Drupal multisite configuration problems

Posted on by
Reply

Drupal multisite setup configuration

I’ve adopted Drupal for a majority of my online activities in the past year. It has enough of the things I like and is much faster and more stable than previous CMS or forum software I’ve used. However, some documentation seems to be lacking. Multisite configuration with Apache using a single Drupal codebase is one area.

I finally decided to try a project in which I would use a common codebase for Drupal across all of my websites. That is, rather than having a directory for each website that has a Drupal installation in it, I thought I would take advantage of Drupal’s multisite functionality by having one drupal installation (codebase) in a central directory and have all the project websites point to it.

Drupal multisite advantages

The advantages of Drupal’s multisite feature are easy to spot.

First, a common codebase means only one codebase to change during an update – even though drupal requires you to update each websites database separately.
Second, using an intermediary cache like APC for PHP means that you can use less server memory caching files across multiple websites because the core files are the same.

Despite this great feature and its advantages, there is much confusion at the Drupal.org website from patrons as to what is the correct way to set this up. The Drupal install file goes into it pretty good, but leaves out one important detail – the web server configuration. Some purists say that is an Apache issue and we should leave that to the Apache mailing list but hey, PHP does a pretty good job of detailing the Apache configuration in their instructions so Drupal could too.

Most people who administer their own websites would follow a ritual in creating a new website. Create a directory for the new website, create the www subdirectory and then configure the Apache DirectoryRoot directive. With a single codebase used by multiple domains and websites you need a different approach.

The missing multisite ‘key’

The most important thing here is to make sure each website’s DocumentRoot in the Apache configuration points to the common drupal directory. This is an unusual configuration for most people but it works well.

The idea is that Drupal receives information from PHP (and Apache) to tell it which website it is supposed to serve. When it determines that, it will retrieve the correct configuration file (database passwords and URL base) from the Drupal ‘sites/’ subdirectory. That’s how it can determine which website to display.

Here is a quick breakdown.

Install Drupal in a common directory. Here I chose /var/lib/drupal-4.7.4

Caveats

Some people use a symbolic link to point Drupal to the correct distribution. I’ve done this but I suggest not doing it. Instead install your Drupal installation in a single directory titled according to it’s version number. That is, use the same directory structure and names that Drupal gave when you extracted the zip or tarball.

Why? If you have a couple sites and it comes time to upgrade you can run into trouble. It is easy to just recreate a symbolic link pointing ‘drupal/’ to ‘drupal-4.7.4′ but unfortunately that affects ALL your websites instantly. Not good on a production server. If you have 50 websites this means that you have 50 sites using the new codebase and the same 50 websites awaiting your hand to manually update their databases using Drupal’s supplied script. If any of those 50 are accessed during this wait period you’ll be in trouble.

The other disadvantage of the symbolic link comes if you are using third-party supplied modules. You forgot that 5 out of your 50 websites are using a module that isn’t being maintained on the same schedule as the Drupal project and guess what? It breaks your website. I’ve found that frequently a misbehaving module or even an errant ‘files’ path in the Drupal settings will disable all other Drupal modules. Best to avoid this altogether.

You can save yourself from both of these negative scenarios by simply putting Drupal in the supplied directory name and then adjusting your Apache DocumentRoot directive as you update them. It is an extra step but very easy.

The final advantage of not using a symbolic link is that you can hold a stable website. That is, you can have a couple different versions of Drupal on your server used by different websites. Several hosting providers do this with PHP and that is a very good example. If you upgrade to PHP 6 for one project you’ll find it doesn’t work with Drupal so you need to keep an old version of PHP installed for non-compliant websites. With Drupal I’ve found that once in a while a website gets working just right and I don’t want to update it. Or I quit monitoring it. Or it is an informational content only website with no subscriptions allowed. Or a million other reasons. Basically I will ‘freeze’ that website and not allow anymore code updates. Or, say you have a module that you use but the author abandoned it long ago. It won’t work with Drupal 6 for example. Simply freeze the website and only update your other websites with the new Drupal codebase. Keeping a couple distributions around can be handy but it means that you can’t point them to the codebase with a symbolic link.

Unzip the new codebase in parallel to the old.

# tar zxvf drupal-4.7.x.tar.gz

Backup the database for the website you are about to update.
Exact commands vary with which database you are using.

Edit Apache config file to change one website at a time.

# vi extra/httpd-vhosts.conf

Change:


DocumentRoot /var/www/example.com/www
ServerName www.example.com

To:


DocumentRoot /var/lib/drupal-4.7.4
ServerName www.example.com

Restart Apache so the config takes effect

# apachectl graceful

Visit the Drupal supplied database update URL for your website.

http://www.example.com/update.php

Watch for errors and check your logs. Visit the website and check the settings to make sure all modules still show up under the admin -> settings menu.

If successful, continue on with the next website on your server.