In the previous episodes of this series of posts, we have built a Magnolia CMS-based project, and learnt about how to make it self-configurable. It’s deployable as-is, in your container of choice, but it is not production-ready: for example, you probably want to use an external database instead of the built-in Derby. In this article, we will see how to make this possible, without having to configure anything manually before each deployment.

A little side note, first: you might notice that this article has the same title as the previous. Well, I renamed the previous article, since the title wasn’t really matching its content. It’s always a little confusing though, as there are really two (mostly independent) configuration mechanisms in Magnolia. On one hand, there’s everything you configure via the repository (covered in the previous article), on the other there’s everything you configure in magnolia.properties files (covered in this article).

The big plan

What we’ll try to achieve with this article, is a what I’ll call a “1-file deployment”. Ideally, an update on the production server should not mean more than copying one file over. We’ll see that there will be a couple more steps in general, but nothing dramatic.

How will we do this ? We will use a technique sometimes referred to as one .war multiple configurations. If you’ve read the documentation article, the below might be a little redundant, but hang on, and we might uncover a few unexpected tricks.

Typically, one will want to change a few things before putting a Magnolia project in production:

• The location of the repository (outside the webapp!)
• The Jackrabbit configuration (i.e use MySQL instead of Derby)
• Whether the instance should automatically apply updates
• Whether the instance should run in “development mode” (no it shouldn’t)
• Whether samples should be bootstrapped (no it shouldn’t)

Custom magnolia.properties locations and files

All the above configuration can be done by modifying the magnolia.properties file. We want to avoid having to do these modifications when deploying; not once, not ever.

Magnolia ships with a few magnolia.properties files, among which WEB-INF/config/default/magnolia.properties, config/magnoliaAuthor/magnolia.properties and config/magnoliaPublic/magnolia.properties. It might be tempting to modify those, but you’ll see that it’s not exactly the preferred approach.

By default, Magnolia finds magnolia.properties files using the “webapp name” and the “server name” substitution variables. The former really only corresponds to the folder name into which the Magnolia instance was deployed, so I would recommend not relying on it¹. The “server name” property might be of interest if you have several instances sharing most of their configuration: it corresponds to the resolved name of the current host.

There are other, perhaps lesser known, substitution variables which can be used, namely context parameters and context attributes. To make use of these, we’ll need a custom web.xml; so far, we’ve relied on the one shipped with magnolia-empty-webapp. Because of this, you might have to modify that web.xml when upgrading to newer Magnolia versions – if necessary, such changes will be mentioned in the release notes, so don’t worry.

Let’s go ahead and create our custom web.xml; we’ll simply copy it from our working copy, since Maven already pulled it from magnolia-empty-webapp:

cp ./acme-project-webapp/target/war/work/info.magnolia/magnolia-empty-webapp/WEB-INF/web.xml ./acme-project-webapp/src/main/webapp/WEB-INF/

Open it in your editor, uncomment the following section and replace its <param-value> as below:

<context-param>
  <param-name>magnolia.initialization.file</param-name>
  <param-value>
    WEB-INF/config/${servername}-${contextParam/magnoliaInstance}-magnolia.properties,
    WEB-INF/config/${servername}-magnolia.properties,
    WEB-INF/config/${contextParam/magnoliaInstance}-magnolia.properties,
    WEB-INF/config/default-magnolia.properties
  </param-value>
</context-param>

Notice the difference with the default value: essentially, we have replaced ${webapp} with ${contextParam/magnoliaInstance}. The ${contextParam/XXX} placeholder will be substituted with a context parameter of the same name. magnoliaInstance is an arbitrary context parameter which we will declare in our container. This could be shipped with the default web.xml of Magnolia, since they won’t get substituted if the value does not exist. If you find a good, legible, meaningful and unequivocal name for this parameter, we’d love to include it in the default web.xml. Share your idea in the comments or on our Jira !

As from Magnolia 4.5, the contextPath will also be available as a substitution variable, as well as system properties (${systemProperty/xxx}) and environment variables (${env/xxx}). Unfortunately, we can’t really ship a default web.xml that would use the ${contextPath} variable, as a deployment in the root context would result in it being substituted with an empty string, thus changing the order in which the properties files are loaded. ²

You’ll also notice we replaced a bunch of /s with -s: nothing mandates the use of magnolia.properties as a file name, and I personally find it more convenient to keep these files in one single directory.

Upon startup, Magnolia will substitute these ${...} placeholders with the appropriate values, and load any file it finds matching the resulting paths. It will then overlay the files in reverse order. What this means is that the last value (WEB-INF/config/default-magnolia.properties) should contain all your most generic and common configuration values, while the first values should contain the more specific values.

For now, we’ll settle with 4 imaginary values for our magnoliaInstance context parameter: prodAuthor, prodPublic, testAuthor and testPublic. I suppose you’re starting to see how one could use this value to differentiate different instances, where the configuration only varies slightly. Let’s create the corresponding configuration files.

Since we also replaced the value of default/magnolia.properties by default-magnolia.properties, we will recreate the file. We’ll use this opportunity to introduce a custom magnolia.home property³. Let’s create this file in acme-project-webapp/src/main/webapp/WEB-INF/config and name it default-magnolia.properties:

magnolia.home=${magnolia.app.rootdir}

magnolia.logs.dir=${magnolia.home}/logs
magnolia.cache.startdir=${magnolia.home}/cache
magnolia.upload.tmpdir=${magnolia.home}/tmp
magnolia.exchange.history=${magnolia.home}/history
magnolia.repositories.home=${magnolia.home}/repositories
magnolia.repositories.config=WEB-INF/config/default/repositories.xml
magnolia.repositories.jackrabbit.config=WEB-INF/config/repo-conf/jackrabbit-bundle-derby-search.xml
magnolia.connection.jcr.userId = admin
magnolia.connection.jcr.password = admin
log4j.config=WEB-INF/config/default/log4j.xml
magnolia.bootstrap.dir=WEB-INF/bootstrap/author WEB-INF/bootstrap/common
magnolia.bootstrap.authorInstance=true
magnolia.bootstrap.samples=true
magnolia.update.auto=false
magnolia.develop=true

This is a cleaned up copy of the default magnolia.properties file shipped with Magnolia. Notice the new magnolia.home property, and how it’s referenced by many other properties below ? This is how we’ll be able to locate our repository, cache, log and temporary files outside the web application’s own folder, by changing the value of a single property. Keep in mind Magnolia substitutes the variables after it has loaded all configuration files. The value of ${magnolia.app.rootdir} corresponds to the root of the webapp’s deployment directory, so it’s a “good enough” default value while developing.

Now all we have to do is create our “specialized” properties files:
testAuthor-magnolia.properties:

magnolia.repositories.jackrabbit.config=WEB-INF/config/repo-conf/jackrabbit-memory-search.xml
magnolia.bootstrap.authorInstance=true

testPublic-magnolia.properties:

magnolia.repositories.jackrabbit.config=WEB-INF/config/repo-conf/jackrabbit-memory-search.xml
magnolia.bootstrap.authorInstance=false

prodAuthor-magnolia.properties:

magnolia.home=/opt/acme-project/data/author/
magnolia.repositories.jackrabbit.config=WEB-INF/config/jackrabbit-acme-prod.xml
magnolia.bootstrap.authorInstance=true
magnolia.bootstrap.dir=WEB-INF/bootstrap/author WEB-INF/bootstrap/common
magnolia.bootstrap.samples=false
magnolia.develop=false
magnolia.update.auto=false

prodPublic-magnolia.properties:

magnolia.home=/opt/acme-project/data/public/
magnolia.repositories.jackrabbit.config=WEB-INF/config/jackrabbit-acme-prod.xml
magnolia.bootstrap.authorInstance=false
magnolia.bootstrap.dir=WEB-INF/bootstrap/public WEB-INF/bootstrap/common
magnolia.bootstrap.samples=false
magnolia.develop=false
magnolia.update.auto=true

So what do we have here ? The “test” instances use one of the jackrabbit configuration files that are shipped with Magnolia by default, which configures JackRabbit so as to store its data in memory. This allows fast startup time, but also means no data is kept between restarts. One could even use this in the default file, but I like my dev or local instance using Derby, so I can still stop/restart my IDE instance without losing all data. They also set the magnolia.bootstrap.authorInstance property, to indicate to Magnolia whether they should be considered an author or public instance when first installing. They “inherit” all other properties from the default configuration file.

In addition, the “prod” configuration files do the following:

• Set magnolia.home to /opt/acme-project/data/<author-or-public>. This value will of course need to be adapted to your deployment scheme, and its value will need to be set in accordance with your system engineers. As a result, all files created by Magnolia (with the exception of JSP files, which need to reside in the webapp’s folder, per spec) will reside in this directory. Note: one could think about using the ${contextParam/magnoliaInstance} variable substitution in the properties file. Not a bad idea, but currently a limitation of the Magnolia 4.4.x branch: the ${contextParam} substitutions are only available when resolving the value of the magnolia.initialization.file parameter set in the web.xml file.
• Set the Jackrabbit configuration file to a custom file which we will get to in detail later.
• Set the magnolia.bootstrap.samples flag to false: we don’t need samples on the production sites.
• Set the magnolia.develop flag to false, which improves the performance somewhat.
• Set the magnolia.update.auto flag to true on the public instance, so that “Magnolia install/update” screens don’t show up, even accidentally, on a public instance.

I have sometimes been tempted to push this even further, and introduce several files and corresponding context parameter: one for the instance “type” (dev, prod, …, which would control properties such as the Jackrabbit configuration) and one for the instance “role” (author or public, which would control the magnolia.bootstrap.authorInstance, magnolia.bootstrap.dir and magnolia.update.auto properties). Tempting, but might make finding the missing or wrong property a little more hairy. What do you think ?

Note: we have seen some folks take another approach: instead of defining a custom magnolia.initialization.file in their web.xml and use several files overriding certain properties of other magnolia.properties files, they chose to define the value of magnolia.initialization.file directly in the container configuration (similarly to how we will declare our magnoliaInstance below), giving it the value of an absolute path outside the webapp, and thus have the one magnolia.properties outside the webapp. This solution is quite clever, and gives maximum flexibility to system engineers (they’ll set these property themselves as they please, without having to report changes back to developers for inclusion in further releases), but however has the slight disadvantage that all properties need to be redeclared.
A middle-ground approach might be to set the web.xml value to something like /opt/${contextParam/magnolia.home}/magnolia.properties, WEB-INF/config/magnolia-default.properties which should provide the best of both worlds (defaults in webapp, overrides outside). Let me know in comments if you’ve used this or any other approach !

Custom Jackrabbit configuration file

Now, we’ve referred to a Jackrabbit configuration file above, so let’s just finish that. Hang on, it will all make sense in a moment.

Create a file in acme-project-webapp/src/main/webapp/WEB-INF/config/, name it jackrabbit-acme-prod.xml, copy the content from this gist:

The only difference with the regular Jackrabbit configuration files shipped with Magnolia is the driver and url parameters of the <PersistenceManager> elements; this is how Jackrabbit knows to use a JNDI DataSource instead of a regular JDBC driver.

Using a datasource has a couple of advantages:

• No username, password, or even database URL in the Jackrabbit configuration file.
• The same file can be re-used on several instances, since the url, username and password are defined outside the webapp.

Tying it all together – container configuration

… and finally, we can make sense of all the above configuration. For this example, we’re going to use Tomcat and MySQL. Feel free to adapt for your container and database of choice.

We’re going to pretend we’re installing a production server … and do that on our development machine. Oh well. In this example, we’ll be deploying both author and public instances on the same Tomcat instance. In a real production environment, you’ll want a different server altogether, and you’ll probably want a second public instance, but let’s put that aside for now.

Start off by extracting a fresh Tomcat instance in a location of your choice… wait, no. Remember how we set the magnolia.home properties to /opt/acme-project/data/<author or public>/ earlier ? Why don’t we use this same location for our Tomcat ? ⁴

sudo mkdir /opt/acme-project
sudo chown $USER /opt/acme-project/
tar xfvz ~/Downloads/apache-tomcat-6.0.33.tar.gz
# delete sample Tomcat webapps
rm -rf tomcat/webapps/ROOT/ tomcat/webapps/docs/ tomcat/webapps/examples/
# this directory is not there before the first start. The `Catalina` folder name corresponds to the `<Service>` name from `server.xml`
mkdir -pv tomcat/conf/Catalina/localhost
ln -s apache-tomcat-6.0.33 tomcat
cd tomcat/bin/ && curl -O http://svn.magnolia-cms.com/svn/community/bundle/trunk/magnolia-tomcat-bundle/src/release/tomcat/bin/setenv.sh && cd -
mkdir wars
mkdir data

So we installed our Tomcat in there. We removed the sample Tomcat applications (allowing us to deploy in ROOT, btw), created a missing configuration directory (so we can configure our contexts before the first startup), we created a data directory, which will host our instances’ magnolia.home contents, and a wars directory, where we’ll drop our .war files. We also grabbed a setenv.sh from Magnolia’s svn server, which just sets a few JAVA_OPTS options.

Next up, we’ll create our contexts. Create the following files:

tomcat/conf/Catalina/localhost/author.xml:

tomcat/conf/Catalina/localhost/ROOT.xml:

So we finally declared and gave a value to our magnoliaInstance context parameter (which we referred to in the web.xml above). We also declared a datasource which is not quite like the examples in Tomcat’s documentation. Instead, it uses the MySQL classes directly, which gives us better control on pooling (or lack thereof, in this case). Tip of the hat to Richard Hunger for letting me know about this trick. ⁵

We have declared two contexts: "" (aka ROOT, aka “empty context path) and /author, for the public and author instance, respectively. In a production system, these would most likely be declared in a different Tomcat instance, possibly using virtual hosts (thus each instance could be deployed in the ROOT context). In our scenario, it is the context’s filename that determines the contextPath. See the Tomcat Configuration Reference for more info.

You’ll also notice that we refer directly to our .war file (via the docBase attribute), and that both contexts refer to the exact same file. This will greatly facilitate updates, as you’ll see below.

The last missing link …

Hopefully, you have MySQL on your machine⁶; if not, you can still edit the datasource definitions urls above to point to some other server. Let’s create the appropriate (empty) databases: ⁷

mysql -u root -p
mysql> CREATE DATABASE acme_repo_author CHARACTER SET utf8;
mysql> CREATE DATABASE acme_repo_public CHARACTER SET utf8;
mysql> CREATE USER 'acme_magnolia'@'localhost' IDENTIFIED BY 'secret';
mysql> GRANT ALL ON acme_repo_author.* TO 'acme_magnolia'@'localhost';
mysql> GRANT ALL ON acme_repo_public.* TO 'acme_magnolia'@'localhost';

Lastly, let’s get the MySQL driver and push it in Tomcat’s shared lib folder: ⁸

cd tomcat/lib && curl -O http://repo1.maven.org/maven2/mysql/mysql-connector-java/5.1.18/mysql-connector-java-5.1.18.jar && cd -

Finally !

We are now ready to deploy and start our app. Let’s do this !

Go back to the folder where the sources of the project are, and build it. (Just imagine the last step (‘cp’) is done by your sysadmin, by downloading the .war from your company’s repository, for example)

cd ~/tmp/blog/acme-project
mvn clean install
cp acme-project-webapp/target/acme-project-webapp-1.0-SNAPSHOT.war /opt/acme-project/wars/

Well that’s it ! We can start !

tomcat/bin/startup.sh && tail -f tomcat/logs/catalina.out

The above command should start Tomcat, and 2 instances of our Magnolia project. If successful, it will immediately start tailing the logs, so you should keep your eyes peeled for the following bits:

2011-10-25 19:30:38,896 DEBUG o.magnolia.cms.servlets.MgnlServletContextListener: magnolia.initialization.file value in web.xml is :'WEB-INF/config/${servername}-${contextParam/magnoliaInstance}-magnolia.properties,
      WEB-INF/config/${contextParam/magnoliaInstance}-magnolia.properties,
      WEB-INF/config/${servername}-magnolia.properties,
      WEB-INF/config/default-magnolia.properties'
2011-10-25 19:30:40,215 INFO  fo.magnolia.cms.beans.config.PropertiesInitializer: Loading configuration at /opt/acme-project/tomcat/webapps/acmeAuthor/WEB-INF/config/default-magnolia.properties
2011-10-25 19:30:40,216 INFO  fo.magnolia.cms.beans.config.PropertiesInitializer: Loading configuration at /opt/acme-project/tomcat/webapps/acmeAuthor/WEB-INF/config/prodAuthor-magnolia.properties

This confirms two things: one, our custom web.xml was indeed loaded, and, more importantly, the properties files we have been working on since the beginning of this article are loaded, and in the expected order.

This will occur twice, as the second instance gets deployed, you’ll notice it loads the public configuration files.

Try to hit your author instance, where you should get the Magnolia installation screens. Start the installation. Try to hit your public instance: the installation should haven go through immediately, so you should see.. a 404 error page. Yeah, we don’t have any content yet, do we ? To be sure, try to hit the public instance’s AdminCentral.

You’ll notice you can’t activate pages between those two instances. That’s because the default Magnolia configuration sets up one subscriber (the public instance to which an author activates) at http://localhost:8080/magnoliaPublic. You’ll need to configure that (to http://localhost:8080, simply). And yes, you should do this via your version handler, as per the previous article. I’ll leave that as an exercise for the reader.

Another exercise, to make sure you’ve understood everything that was going on: how would you deploy the project on a test server ?

A note on updates

If you’ve made a mistake, or want to try something new in the webapp, simply rebuild the webapp and replace the war file in /opt/acme-project/wars. Stop Tomcat, delete the corresponding webapp directory in Tomcat: rm -rf tomcat/webapps/acme* and restart Tomcat. Since we’ve used a magnolia.home outside the webapp, you will not lose any data !

In case of a real, non-snapshot, update, you’ll want to keep your 1.0 .war file, and get the new 1.0.1 release next to it. You’ll need to update the context file to point to the new .war, delete the extracted webapps and restart Tomcat.

Anything else ?

I hope you’ve picked up a few things along the (admittedly long and dense) way of these three articles. We now have a project based on Magnolia which:

• Builds upon Magnolia with minimal duplication, and is deployable as-is for both development and production,
• Installs itself and configures Magnolia without human intervention,
• Provides one single artifact for deployment of several instances and different servers,
• Can easily be updated, by deploying a new .war file.

Some parts of this dense article might be unclear. Try it out, and let me know what you think. I’ll also be happy to extend or cover areas I haven’t yet.

I’m a sucker for good ideas, so there are probably things that can be improved in Magnolia, or in our build chain, to make this whole process easier. And surely, there are ideas, tools and processes out there that could help as well. Looking forward to your comments !