Don’t configure Magnolia: let your projects configure it.

In the previous installment of this series of posts, we have created a new Magnolia-based project, including a reproducible build. We have a webapp that can be started in an IDE, or outside of it.
You might remember we added an STK theme to our project. And you might remember that it didn’t go all that well when we tried to view the first page we created. As mentioned in the previous post, this is due to MGNLSTK-788; that doesn’t mean you need to wait for a bugfix, however. The bug is due to the fact that no theme was configured for your project. Either way, you will need to tell Magnolia what theme you want to use for your specific website, right ?

Since we’re using STK (as opposed to ETK and the Enterprise Edition), configuring which theme to use can be done via the following menu entries in the Admin Central: Templating Kit > Site Configuration, then opening the theme node, and setting the name property. It currently refers to the pop theme, which we did not install in our project, and that’s why you’re seeing the aforementioned bug. As you might know, the Site Configuration tree we are looking at is a shortcut to the modules/standard-templating-kit/config/site node in the configuration workspace, so the canonical path to the property we’re interested in is modules/standard-templating-kit/config/site/theme/name. For now, let’s change this property to blue, and verify our test page shows up.

So… that works. But, once your project goes to production, do you really want to go and find that item in the tree and modify it ? When there’s just so much traffic to your site that you need to deploy a second public instance, you probably would rather avoid doing that too (or worse, forgetting to do it). Closer to our developer hearts, you don’t want to have to write documentation for your fellow developers and administrators for them to go and change that property either, do you ? You probably don’t even want to remember to do this yourself every time you clean up your development environment.

Enter the module mechanism

If you know everything about Magnolia’s module mechanism, you’ll be tempted to skip to the next section. However, I’d like to introduce a kind a module that isn’t very much spoken about. Magnolia “functional” modules are a well-know part of the Magnolia eco-system, and Magnolia itself is built as a collection of base modules (there’s a ton on our repositories and the forge). But there’s also a lesser-known, perhaps less popular kind of module (for a simple reason: there’s no sense in publishing them), which I’d call “project module”. The purpose of such a module is to simply install, or configure, your project.

Since you can read about the module mechanism in our documentation, I’ll just point out the few things we’re going to do here:

• Declare dependencies. By declaring that our project module depends on STK and the “Blue” theme, for instance, we’ll ensure our module is installed after these, thus ensuring we can configure them.
• Use a ModuleVersionHandler to automate all installation or configuration tasks our project requires. (i.e setting the theme to “blue”)

Before we go further, a quick note on terminology: I will further refer to “module” or “project module” and that will mean “The files that compose our Project-specific Magnolia module”. It’s unfortunate that Magnolia and Maven use the same “module” term for different things. When I mean “Maven module”, I will write “Maven module”.

Typically, in a Maven-centric environment, you’ll write a Magnolia module as a separate Maven module. When it comes to project modules however, I’m fine with the module being intertwined with the webapp Maven-module itself; such a project module will consist of just about 2 files, and they belong closer to all other configuration files (Jackrabbit, web.xml, …) than to other modules.

You might also have noticed there is an archetype for Magnolia modules, so we could use that. It creates the necessary infrastructure for more complex modules though, so it would create more noise than necessary for this tutorial.

With that said, we will now go ahead and do stuff. First off, let’s create our module descriptor. Let’s create the directory structure. From the root of the project, type:

mkdir -p acme-project-webapp/src/main/resources/META-INF/magnolia/

… or clickedi-click in your IDE if you are so inclined.

In this newly created directory, let’s create our module descriptor, in a file called acme-project.xml:

<!DOCTYPE module SYSTEM "module.dtd">
<module>
  <name>acme-project</name>
  <displayName>${project.name}</displayName>
  <description>${project.description}</description>
  <versionHandler>com.acme.myproject.Setup</versionHandler>
  <version>${project.version}</version>

  <dependencies>
    <dependency>
      <name>standard-templating-kit</name>
      <version>1.4.4/*</version>
    </dependency>
    <dependency>
      <name>dms</name>
      <version>1.5.2/*</version>
    </dependency>
    <dependency>
      <name>theme-blue</name>
      <version>1.0/*</version>
    </dependency>
  </dependencies>
</module>

What’s relevant here ? We rely on Maven’s variable substitution to fill in most values, and we more-or-less replicate the pom‘s dependencies (except we’re using version ranges). Most of this could indeed be completely generated, and there are plans for that. The one interesting bit is the versionHandler element, and we’ll get to that right away. Eh, it refers to an non-existing class, right ? We’ll create it. You can read more about Module version handlers, then come back here:

mkdir -p acme-project-webapp/src/main/java/com/acme/myproject

In this newly created directory¹, let’s create our Setup class², with the following code:

package com.acme.myproject;

import info.magnolia.module.DefaultModuleVersionHandler;
import info.magnolia.module.InstallContext;
import info.magnolia.module.delta.CheckAndModifyPropertyValueTask;
import info.magnolia.module.delta.Task;

import java.util.ArrayList;
import java.util.List;

/**
 * Sets up our great ACME project !
 */
public class Setup extends DefaultModuleVersionHandler {
    @Override
    protected List<Task> getExtraInstallTasks(InstallContext ctx) {
        final List<Task> install = new ArrayList<Task>();
        install.add(new CheckAndModifyPropertyValueTask(
            "Setup theme",
            "Sets up the 'blue' theme for our project.",
            "config",
            "/modules/standard-templating-kit/config/site/theme",
            "name", "pop", "blue"));
        return install;
    }
}

Let’s have a closer look at what this class does³: we extend the DefaultModuleVersionHandler which provides a reasonable set of default tasks. I have pondered about this while writing this article, though. Extending AbstractModuleVersionHandler would have made sense, since we don’t really need or want to use any of the other regular module specificities. However, I wouldn’t want to introduce more confusion than necessary, so we’ll stick to the basics for now. If you know the difference between the two classes, and are familiar with the version handling mechanism, I’d indeed recommend you extend AbstractModuleVersionHandler for such project modules. If there’s interest, and good ideas pop up, there might even be a case for exposing a base class for such project modules in future versions of Magnolia.

The only method we currently implement is getExtraInstallTasks, which, in its default implementation, returns an empty list. This is where a module can tell Magnolia what to do when installing the module. There is an important differentiation between installs and updates with the module mechanism, and we’ll get back to that when we talk about releasing our project.

In getExtraInstallTasks, we currently register a single task, called CheckAndModifyPropertyValueTask.

Essentially, this instance of the task is modifying the name property of the /modules/standard-templating-kit/config/site/theme node of the config workspace (which is the node we manually modified earlier); the complete configuration of the task itself might seem a little overwhelming, but there are several points to keep in mind:

• A “good-citizen” task will name and describe itself properly, so users can read what’s going to happen when they go through the install process. (We’ve tried to apply a convention where the name of the task is a simple sentence providing some context, and its description is a third person, complete and meaningful sentence ending with a period symbol, so as to have a sense of uniformity when displaying the tasks in the install process.)
• A “good-citizen” task will attempt, as much as possible, to be safe and reliable. In our case, we’re ensuring that the current value of the property is pop before we modify it. This level of safety helps ensuring your module(s) are deployed as expected. When writing install and update tasks, developers should give a long thought about what the expected state of the system should be. In this case, we wouldn’t modify to modify the property if, for any reason, it had already been modified: if by accident another module does the same sort of change, we want to cancel the installation before we overwrite those unexpected changes.

What now ? Well, two choices: if your Magnolia instance is still running, go back to the theme configuration and set it back to pop manually – we just wrote code that ensures it is set to pop when our project is first installed – then shut it down. Or, if it’s not running anymore, you can safely drop your repository:

rm -r ./acme-project-webapp/target/acme-project-webapp-1.0-SNAPSHOT/repositories

Assuming your IDE was able to re-build the project properly, you can now restart the server. You’ll have to go through the install process again, because we just added a new module ! (And if you wiped your repository, you’ll also have to reinstall all other modules, of course). So if all went well, Magnolia should tell you it needs to install “My ACME Project: webapp (version 1.0.0-SNAPSHOT)”, and if you “show details”, you should find our own custom install tasks which “Sets up the ‘blue’ theme for our project.”.

Go ahead and proceed with the installation.

Recreate a sample page if needed, and shazam!, no more FreeMarker error, nor manual configuration of the theme !

Install and update tasks

We have registered an install task above. This means the task will be executed only when the module is added, for the first time, to a given instance.

If a module is already deployed, we need to look into update tasks. Here’s how this could look:

public Setup() {
    register(DeltaBuilder.update("1.1", "ACME 1.1 has a bunch of cool new features.")
            .addTask(new CheckAndModifyPropertyValueTask("Mail config", "Sets up the mail module to use our mail server.", "config", "/modules/mail/config", "smtpServer", "localhost", "mail.acme.org"))
    );
}

What we do here is tell Magnolia: “Hey, for version 1.1 of this module, we want these tasks to be executed: <…>”.

Magnolia knows what version of a module it currently installed, by keeping track of the version property under eachmodule’s node. And it knows what version of a module is being deployed by looking at the module descriptor. (And because we use Maven variable substitution in our module descriptor, we know that the version number Magnolia will read is the same as our Maven project’s version.)

Now, you need to think about how and when your module is deployed and installed. In this example, it is quite likely that you’ll want to also apply this task when installing the module. So you should probably register the same task in getExtraInstallTasks. (Since tasks are stateless descriptors, you can easily extract them into class fields, for example.) If you look at Magnolia’s own module version handlers, you’ll find that there are not that many install tasks, because most of the installation work is done via bootstrap files (which are extracted on installation by one of the default tasks provided by DefaultModuleVersionHandler).

When installing a module, only install tasks are executed; when updating a module, only the update tasks registered for versions between the current and the version we’re updating to are executed.

The logic above only applies to subclasses of AbstractModuleVersionHandler, and, if needed, you could completely modify this logic by re-implementing the getDeltas method of your ModuleVersionHandler.

The future of your project

As the above might have suggested, there can be a lot to do in terms of management of updates and releases. If thoroughly used and applied though, Magnolia’s update mechanism can be a powerful tool.

Using a stable version numbering scheme for your modules, as well as a proper release process (Maven’s release plugin, if using Maven) will help greatly for updates of integration and production systems. It might however be a little tedious to update your development environment. I don’t think there’s any other recipe than discipline here (i.e remember to write update tasks for all needed changes, and test them regularly). Writing update tasks can be a difficult exercise, and even more so if you don’t write them immediately. We wrote a ModuleVersionHandlerTestCase class to help writing unit tests for your module version handlers. It’s far from perfect, so we’ll welcome any improvements. ⁴

This is also an exercise in dependency management. You’ll need to figure out when to update to Magnolia’s newest and latest release, and figure out what that means for your module. In most cases however, Magnolia itself will handle the update nicely. Say, if the way a theme was registered for a project was to be changed in a future release of STK, STK’s own ModuleVersionHandler will take care of updating your configuration. Of course, our example module’s install task would still need to be modified to take this change into account.

A note about bootstrap files

I’ve only slightly mentioned “bootstrap files” so far. Before the introduction of the update mechanism in Magnolia, this was the only way to install or configure a module or project. Typically, what happens is this:

• files in a module’s mgnl-boostrap/<module-name>/ are bootstrapped on install, provided one is using DefaultModuleVersionHandler or a subclass of it.
• files in a webapp’s WEB-INF/bootstrap/author or WEB-INF/bootstrap/public, as well as WEB-INF/bootstrap/common are bootstrapped when the corresponding workspace(s) are empty; one can change which folders are used by modifying the magnolia.bootstrap.dir property.

What’s a bootstrap file ? It’s a simple JCR export file, i.e an xml representation a node’s content, as exported by Magnolia. Magnolia’s naming scheme for such file is pretty simple: <workspace-name>.<path>.<to>.<node>.xml.

It’s an easy way to get module installed, and can be useful to push “large” or default content in the repository. I’d recommend using this to bootstrap sample or base content in the website, dms, data workspaces, for example. I’d advise against it for configuration of a project (since you’re likely to modify existing content that was installed by another module, you’re better off using the more strict mechanisms described above.

Sean wrote an interesting piece with some interesting ideas and practices for the management of your modules and content.

What’s next ?

We now have a project which not only has a stable and reproducible build, but which can also be installed and deployed right away by your peers. Hopefully you’ll be able to reuse ideas to better manage your project’s lifecycle by now. If not, feel free to comment on what doesn’t work for you, or what could be clarified or expanded upon.

Next up, we’ll look into what to do to make this project deployable as-is in production.