Scripts for a WordPress Weblog Farm [en]

A first step to WordPress-farming: a shell script and a PHP script which allow you to easily install a whole lot of WordPress weblogs in only a few minutes (I installed over 30 in less than 5 minutes). Scripts require adapting to your environment, of course.

Update 03.11.06: Batiste made me realise I should point the many people landing here in the search of multi-user WordPress to WordPress MU. All that I describe in this post is very pretty, but nowadays completely obsolete.

Here is the best solution I’ve managed to come up with in half a day to finally install over 30 WordPress weblogs in under 5 minutes (once the preparation work was done).

A shell script copies the image of a WordPress install to multiple directories and installs them. A PHP script then changes a certain number of options and settings in each weblog. It can be used later to run as a “patch” on all installed weblog if a setting needs modifying globally.

Here are the details of what I did.

I first downloaded and unzipped WordPress into a directory.

wget http://wordpress.org/latest.tar.gz
tar -xzvf latest.tar.gz
mv wordpress wp-farm-image

I cleaned up the install (removing wp-comments-popup.php and the import*.php files, for example), added a language directory (as I’m wp-farming in French) and modified index.php to my liking; in particular, I edited the import statement for the stylesheet so that it looked like this:

@import url( http://edublogs.net/styles/toni/style.css );

The styles directory is a directory in which I place a bunch of WordPress styles. I don’t need the style switcher capability, but I do need to styles. Later, users will be able to change styles simply by editing that line in their index.php (or I can do it for them).

Another very important thing I did was rename wp-config-sample.php to config-sample and fill in the database and language information. I replaced wp_ by xxx_ so that I had $table_prefix = 'xxx_';.

To make it easier to install plugins for everyone, correct the language files, and edit whatever may be in wp-images, I moved these three directories out of the image install and replaced them with symbolic links, taking inspiration from Shelley’s method for installing multiple WordPress weblogs.

mv image/wp-content common
mv image/wp-images common
mv image/wp-includes/languages common
ln -s common/wp-content image/wp-content
ln -s common/wp-images image/wp-images
ln -s common/languages image/wp-includes/languages

I also added an .htaccess file (after some painful tweaking on a test install).

Once my image was ready, I compiled a list of all the users I had to open weblogs for (one username per line) in a file named names.txt, which I placed in the root directory all the weblog subfolders were going to go in.

I then ran this shell script (many thanks to all those of you who helped me with it — you saved my life):

for x in `cat names.txt`
do
cp -rv /home/edublogs/wp-farm/image/ $x
cat $x/wp-config.php | sed "s/xxx/${x}/" > config.tmp
mv config.tmp $x/wp-config.php
wget http://st-prex.edublogs.net/$x/wp-admin/install.php?step=1
wget http://st-prex.edublogs.net/$x/wp-admin/install.php?step=2
wget http://st-prex.edublogs.net/$x/wp-admin/install.php?step=3
done

This assumes that my WordPress install image was located in /home/edublogs/wp-farm/image/ and that the weblog addresses were of the form http://st-prex.edublogs.net/username/.

This script copies the image to a directory named after the user, edits wp-config to set the table prefix to the username, and then successively wgets the install URLs to save me from loading them all in my browser.

After this step, I had a bunch of installed but badly configured weblogs (amongst other things, as I short-circuited the form before the third install step, they all think their siteurl is example.com).

Entered the PHP patch which tweaks settings directly in the database. I spent some time with a test install and PHPMyAdmin to figure out which fields I wanted to change and which values I wanted to give them, but overall it wasn’t too complicated to do. You’ll certainly need to heavily edit this file before using it if you try and duplicate what I did, but the basic structure and queries should remain the same.

I edited the user list at the top of the file, loaded it in my browser, and within less than a few seconds all my weblogs were correctly configured. I’ll use modified versions of this script later on when I need to change settings for all the weblogs in one go (for example, if I want to quickly install a plugin for everyone).

In summary:

  1. compile list of users
  2. prepare image install
  3. run shell script
  4. run PHP script

If you try to do this, I suggest you start by putting only two users in your user list, and checking thoroughly that everything installs and works correctly before doing it for 30 users. I had to tweak the PHP script quite a bit until I had all my settings correctly configured.

Hope this can be useful to some!

Update 29.09.2005: WARNING! Hacking WordPress installs to build a farm like this one is neat, but it gets much less neat when your weblog farm is spammed with animal porn comments. You then realise (oh, horror!) that none of the anti-spam plugins work on your beautiful construction, so you weed them out by hand as you can, armed with many a MySQL query. And then the journalist steps in — because, frankly, “sex with dogs” on a school website is just too good to be true. And then you can spend half a day writing an angry reaction to the shitty badly-researched article.

My apologies for the bad language. Think of how you’re going to deal with spam beforehand when you’re setting up a school blog project.

Musings on a Multiblog WordPress [en]

Thinking about a solution to make WordPress MultiBlog. Comments, criticism and other ideas welcome — please join the fun. In particular, I bump into a hairy PHP include problem.

[fr] Je réfléchis à comment on pourrait donner à WordPress la capacité de gérer plusieurs blogs avec une installation. Je me heurte à un problème concernant les includes PHP. Feedback et autres idées bienvenues!

Update June 2007: Try WordPress Multi-User now.

I’ve used Shelley’s instructions using soft links. I tried Rubén’s proof-of-concept, but got stuck somewhere in the middle.

So I started thinking: how can we go about making WordPress MultiBlog-capable? Here is a rough transcript of my thoughts (I’ve removed some of the dead ends and hesitations) in the hope that they might contribute to the general resolution of the problem. I have to point out my position here: somebody with a dedicated server who’s thinking of setting up a “WordPress weblog-farm” (for my pupils, mainly). So I’m aware that I’m not the “standard user” and that my solution is going to be impractical to many. But hey, let’s see where it leads, all the same. Actually, I think I probably reconstructed most of Rubén’s strategy here — but I’m not sure to what extent what I suggest differs from what he has done.

From a system point of view, we want to have a unique installation of WordPress, and duplication of only the files which are different from one blog to another (index.php, wp-config.php, wp-comments.php, wp-layout.css, to name a few obvious ones). The whole point being that when the isntall needs to be upgraded, it only has to be upgraded in one place. When a plugin is downloaded and installed, it only has to be done once for all weblogs — though it can of course be activated individually for each weblog.

From the point of view of the weblogs themselves, they need to appear to be in different domains/subdomains/folders/whatever. What I’m most interested in is different subdomains, so I’ll stick to that in my thinking. (Then somebody can come and tell me that my “solution” doesn’t work for subfolders, and here’s one that works for subfolders and subdomains, and we’ll all be happy, thankyouverymuch.) So, when I’m working with blog1.example.com all the addresses need to refer to that subdomain (blog1.example.com/wp-admin/, etc); ditto for blog2.example.com, blog3.example.com, blogn.example.com (I used to like maths in High School a lot).

As Rubén puts it, the problem with symbolic links (“soft links”) is called “soft link hell”: think of a great number of rubber bands stretched all over your server. Ugh. So let’s try to go in his direction, for a while. First, map all the subdomains to the same folder on the server. Let’s say blog1.example.com, blog2.example.com (etc.) all point to /home/bunny/www/wordpress/. Neat, huh? Not so. They will all use the same wp-config.php file, and hence all be the same weblog.

This is where Rubén’s idea comes in: include a file at the top of wp-config.php which:

  1. identifies which blog we are working with (in my case, by parsing $HTTP_HOST, for example — there might be a more elegant solution)
  2. “replaces” the files in the master installation directory by the files in a special “blog” directory, if they exist

The second point is the tricky one, of course. We’d probably have a subfolder per blog in wordpress/blogs: wordpress/blogs/blog1, wordpress/blogs/blog2, etc. The included file would match the subdomain string with the equivalent folder, check if the page it’s trying to retrieve exists in the folder, and if it does, include that one and stop processing the initial script after that. Another (maybe more elegant) option would be to do some Apache magic (I’m dreaming, no idea if it’s possible) to systematically check if a file is available in the subdirectory matching the subdomain before using the one in the master directory. Anybody know if this is feasible?

The problem I see is with includes. We have (at least) three types of include calls:

  • include (ABSPATH . 'wp-comments.php');
  • require ('./wp-blog-header.php');
  • require_once(dirname(__FILE__).'/' . '/wp-config.php');

As far as I see it, they’ll all break if the calling include is in /home/bunny/www/wordpress/blogs/blog1 and the file to be called is in /home/bunny/www/wordpress. What is wrong with relative includes? Oh, they would break too. Dammit.

We would need some intelligence to determine if the file to be included or called exists in the subdirectory or not, and magically adapt the include call to point to the “right” file. I suspect this could be done, but would require modifying all (at least, a lot of) the include/requires in WordPress.

Maybe another path to explore would be to create a table in the database to keep track of existing blogs, and of the files that need to be “overridden” for each blog. But again, I suspect that would mean recoding all the includes in WordPress.

Another problem would be .htaccess. Apache would be retrieving the same .htaccess for all subdomains, and that happens before PHP comes into play, if I’m not mistaken.

Any bright ideas to get us out of this fix? Alternate solutions? Comments? Things I missed or got wrong? The comments and trackbacks are yours. Thanks for your attention.

Life and Trials of a Multilingual Weblog [en]

Here is an explanation of how I set up WordPress to manage my bilingual weblog. I give all the code I used to do it, and announce some of the things I’d like to implement. A “Multilingual blogging” TopicExchange channel is now open.

[fr] J'explique ici quelles sont les modifications que j'ai faites à WordPress pour gérer le bilinguisme de mon weblog -- code php et css à l'appui. Je mentionne également quelques innovations que j'ai en tête pour rendre ce weblog plus sympathique à mes lecteurs monolingues (ce résumé en est une!) Un canal pour le weblogging multilingue a été ouvert sur TopicExchange, et vous y trouverez peut-être d'autres écrits sur le même sujet. Utilisez-le (en envoyant un trackback) si vous écrivez des billets sur le multinguisme dans les weblogs!

My weblog is bilingual, and has been since November 2000. Already then, I knew that I wouldn’t be capable of producing a site which duplicates every entry in two languages.

I think this would defeat the whole idea of weblogging: lowering the “publication barrier”. I feel like writing something, I quickly type it out, press “Publish”, and there we are. Imposing upon myself to translate everything just pushes it back up again. I have seen people try this, but I have never seen somebody keep it up for anything nearing four years (this weblog is turning four on July 13).

This weblog is therefore happily bilingual, as I am — sometimes in English, sometimes in French. This post is about how I have adapted the blogging tools I use to my bilingualism, and more importantly, how I can accommodate my monolingual readers so that they also feel comfortable here.

First thing to note: although weblogging tools are now ready to be used by people speaking a variety of languages (thanks to a process named “localization”), they remain monolingual. Language is determined at weblog-level.

With Movable Type, I used categories to emulate post-level language awareness. This wasn’t satisfying at all: I ended up with to monstrous categories, Français and English, which didn’t help keep rebuild times down.

With WordPress, the solution is far more satisfying: I store the language information as Post Meta, or “custom field”. No more category exploitation for something they shouldn’t be used for.

Before I really got started doing the exciting stuff, I made a quick change to the WordPress admin interface. If I was going to be adding a “language” custom field to each and every post of mine, I didn’t want to be doing it with the (imho) rather clumsy “Custom Fields” form.

In edit.php, just after the categorydiv fieldset, I inserted the following:

<fieldset id="languagediv">
      <legend>< ?php _e('Language') ?></legend>
	  <div><input type="text" name="language" size="7"
                     tabindex="2" value="en" id="language" /></div>
</fieldset>

(You’ll probably have to move around your tabindex values so that the tabbing order makes sense to you.)

I also tweaked the wp-admin.css file a bit to keep it looking reasonably pretty, adding the rule below:

#languagediv {
	height: 3.5em;
	width: 5em;
}

and adding #languagediv everywhere I could see #poststatusdiv, so that they obeyed the same rules.

In this way, I have a small text field to edit to set the language. I pre-set it to “en”, and have just to change it to “fr” if I am writing in French.

We just need to add a little piece of code in the form processing script, post.php, just after the line that says add_meta($post_ID):

 // add language
	if(isset($_POST['language']))
	{
	$_POST['metakeyselect'] = 'language';
        $_POST['metavalue'] = $_POST['language'];
        add_meta($post_ID);
        }

The first thing I do with this language information is styling posts differently depending on the language. I do this by adding a lang attribute to my post <div>:

<div class="post" lang="<?php $post_language=get_post_custom_values("language"); $the_language=$post_language['0']; print($the_language); ?>">

In the CSS, I add these rules:

div.post:lang(fr) h2.post-title:before {
  content: " [fr] ";
  font-weight: normal;
}
div.post:lang(en) h2.post-title:before {
  content: " [en] ";
  font-weight: normal;
}
div.post:lang(fr)
{
background-color: #FAECE7;
}

I also make sure the language of the date matches the language of the post. For this, I added a new function, the_time_lg(), to my-hacks.php. I then use the following code to print the date: <?php the_time_lg($the_language); ?>.

Can more be done? Yes! I know I have readers who are not bilingual in the two languages I use. I know that at times I write a lot in one language and less in another, and my “monolingual” readers can get frustrated about this. During a between-session conversation at BlogTalk, I suddenly had an idea: I would provide an “other language” excerpt for each of my posts.

I’ve been writing excerpts for each of my posts for the last six months now, and it’s not something that raises the publishing barrier for me. Quickly writing a sentence or two about my post in the “other language” is something I can easily do, and it will at least give my readers an indication about what is said in the posts they can’t understand. This is the first post I’m trying this with.

So, as I did for language above, I added another “custom field” to my admin interface (in edit-form.php). Actually, I didn’t stop there. I also added the field for the excerpt to the “simple controls” posting page that I use (set that in Options > Writing), and another field for keywords, which I also store for each post as meta data. Use at your convenience:

<!-- BEGIN BUNNY HACK -->
<fieldset style="clear:both">
<legend><a href="http://wordpress.org/docs/reference/post/#excerpt"
title="<?php _e('Help with excerpts') ?>"><?php _e('Excerpt') ?></a></legend>
<div><textarea rows="1" cols="40" name="excerpt" tabindex="5" id="excerpt">
<?php echo $excerpt ?></textarea></div>
</fieldset>
<fieldset style="clear:both">
<legend><?php _e('Other Language Excerpt') ?></legend>
<div><textarea rows="1" cols="40" name="other-excerpt"
tabindex="6" id="other-excerpt"></textarea></div>
</fieldset>
<fieldset style="clear:both">
<legend><?php _e('Keywords') ?></legend>
<div><textarea rows="1" cols="40" name="keywords" tabindex="7" id="keywords">
<?php echo $keywords ?></textarea></div>
</fieldset>
<!-- I moved around some tabindex values too -->
<!-- END BUNNY HACK -->

I inserted these fields just below the “content” fieldset, and styled the #keywords and #other-excerpt textarea fields in exactly the same way as #excerpt. Practical translation: open wp-admin.css, search for “excerpt”, and modify the rules so that they look like this:

#excerpt, #keywords, #other-excerpt {
	height: 1.8em;
	width: 98%;
}

instead of simply this:

#excerpt {
	height: 1.8em;
	width: 98%;
}

I’m sure by now you’re curious about what my posting screen looks like!

To make sure the data in these fields is processed, we need to add the following code to post.php (as we did for the “language” field above):

// add keywords
	if(isset($_POST['keywords']))
	{
	$_POST['metakeyselect'] = 'keywords';
        $_POST['metavalue'] = $_POST['keywords'];
        add_meta($post_ID);
        }
   // add other excerpt
	if(isset($_POST['other-excerpt']))
	{
	$_POST['metakeyselect'] = 'other-excerpt';
        $_POST['metavalue'] = $_POST['other-excerpt'];
        add_meta($post_ID);
        }

Displaying the “other language excerpt” is done in this simple-but-not-too-elegant way:

<?php
$post_other_excerpt=get_post_custom_values("other-excerpt");
$the_other_excerpt=$post_other_excerpt['0'];
if($the_other_excerpt!="")
{
	if($the_language=="fr")
	{
	$the_other_language="en";
	}

	if($the_language=="en")
	{
	$the_other_language="fr";
	}
?>
    <div class="other-excerpt" lang="<?php print($the_other_language); ?>">
    <?php print($the_other_excerpt); ?>
    </div>
  <?php
  }
  ?>

accompanied by the following CSS:

div.other-excerpt:lang(fr)
{
background-color: #FAECE7;
}
div.other-excerpt:lang(en)
{
background-color: #FFF;
}
div.other-excerpt:before {
  content: " [" attr(lang) "] ";
  font-weight: normal;
}

Now that we’ve got the basics covered, what else can be done? Well, I’ve got some ideas. Mainly, I’d like visitors to be able to add “en” or “fr” at the end of any url to my weblog, and that would automatically filter out all the content which is not in that language — maybe using the trick Daniel describes? In addition to that, it would also change the language of what I call the “page furniture” — titles, footer, and even (let’s by ambitious) category names. Adding language sensitivity to trackbacks and comments could also be interesting.

A last thing I’ll mention in the multilingual department for this weblog is my styling of outgoing links if they are written in a language which is not my post language, using the hreflang attribute. It’s easy, and you should do it too!

Suw (who has just resumed blogging in Welsh) and I have just set up a “Multilingual blogging” channel on TopicExchange — please trackback it if you write about blogging in more than one language!