Quartz Where!

One week ago we released the first beta of our Fremantle release, which includes Ehcache 2.4, Terracotta Enterprise Suite 3.5 and at last but not least Quartz 2.0. You have been able to cluster your Quartz Scheduler instances using Terracotta for a while already. Yet, as with a JDBC backed storage, you had no control over what node your job would be executed on. The only guarantee was that your job would be executed once within the cluster. Quartz Where aims at addressing exactly that, and is one of the many new features that are part of this new major release of our product line. A popular demand from clustered Quartz Scheduler users was to be able to specify where a Job would be executed: because data for the job is known to be present on some machine (like using NFS-like file sharing) or because the Job requires much processing and memory. Controlling the locality of execution is now feasible. We have tried to make this a seamless addition to Quartz: you can configure jobs to be dispatched to node groups using a simple configuration file; or programmatically schedule LocalityJob or LocalityTrigger instances. Let’s first cover the configuration based approach, which doesn’t require any code changes to an existing Quartz 2.0 application.

Configuration based locality of execution

Before getting started with this new feature, you will have to configure your Quartz scheduler to use the Terracotta Enterprise Store by setting the property org.quartz.jobStore.class to org.terracotta.quartz.EnterpriseTerracottaJobStore. If you were not using Terracotta to cluster Quartz, you will also have to set the org.quartz.jobStore.tcConfigUrl property to point to the Terracotta server. Here is a small example of a quartz.properties

org.quartz.scheduler.instanceName = QuartzWhereScheduler

org.quartz.scheduler.instanceId = AUTO
org.quartz.scheduler.instanceIdGenerator.class = org.terracotta.quartz.demo.locality.SystemPropertyIdGenerator

org.quartz.threadPool.class = org.quartz.simpl.SimpleThreadPool
org.quartz.threadPool.threadCount = 10
org.quartz.threadPool.threadPriority = 5
org.quartz.threadPool.threadsInheritContextClassLoaderOfInitializingThread = true

org.quartz.jobStore.class = org.terracotta.quartz.EnterpriseTerracottaJobStore
org.quartz.jobStore.tcConfigUrl = localhost:9510

Using the quartzLocality.properties configuration file, you can define node groups. A node group is composed of one or more Quartz Scheduler instance nodes (generally one per machine within your cluster). You define them as such:

org.quartz.locality.nodeGroup.slowNodes = tortoise, snail
org.quartz.locality.nodeGroup.fastNodes = hare, leopard
org.quartz.locality.nodeGroup.linuxNodes = tortoise

We have now defined three groups: the slowNodes, fastNodes and linuxNodes. We can now use the node groups to have jobs or triggers being executed by them, depending on their group. Quartz Jobs and Triggers were and still are uniquely identified by a name and group pair. We can now have all jobs (or triggers) of a certain group only get executed on a node of a given group through the same configuration file:

org.quartz.locality.nodeGroup.fastNodes.triggerGroups = bigJobGroup
org.quartz.locality.nodeGroup.linuxNodes.triggerGroups = reporting

Now all triggers from the group bigJobGroup will be executed by a Scheduler from the group fastNodes, either the hare or leopard scheduler. These scheduler nodes receive unique ids as before by providing an org.quartz.spi.InstanceIdGenerator implementation to the scheduler at configuration time (don’t mix this with the instanceName, which needs to be the same for all nodes from them to be a single clustered Scheduler). Triggers from the group reporting will always be executed on tortoise, as this is the only scheduler in the linuxNodes group.

Programmatic locality of execution

Using the new locality API for Quartz that is part of our Terracotta Enterprise Suite 3.5 you can achieve even finer grained control and express more complex constraints about where a job should be executed. The example below uses the new DSL like builder API introduced with Quartz 2.0. Let’s see how that looks:

LocalityJobDetail jobDetail =
    localJob(
        newJob(ImportantJob.class)
            .withIdentity("importantJob")
            .build())
        .where(
            node()
                .is(partOfNodeGroup("fastNodes")))
        .build();

On line 3, we create a new JobDetail for the Job implementation ImportantJob. We then wrap it on line 2 as a localJob that needs to be executed on a node that is part of the group fastNodes. You might have noticed that creating the JobDetail is pretty straight forward with the new API. Adding the locality information isn’t much more work neither. You can be much more precise on where the job should be executed though. Let’s have a look at this example:

scheduler.scheduleJob(
    localTrigger(
        newTrigger()
            .forJob("importantJob"))
        .where(node()
            .has(atLeastAvailable(512, MemoryConstraint.Unit.MB)
            .is(OsConstraint.LINUX)))
        .build());

Here we schedule an immediate trigger for the importantJob we’ve registered in the previous example. Line 2 is creating the locality aware trigger, defining it to require a node that is running Linux and has at least 512 MB of heap available. Using these constraints, here memory and OS, you can be much more explicit about what the characteristics of the node executing the Job should be.

Locality constraints

The new Terracotta clustered JobStore we’ve introduced will evaluate the constraint expressed on a Trigger and/or a Job to decide where to dispatch the Job for execution. We plan on providing implementation for expressing constraints on the CPU, Memory and Operating System characteristics of the node. I am still heavily working on these, but what is being shipped as part of this first beta should give you a good feel for where we are headed. To test it yourself today, go fetch the Fremantle beta 1 from the Terracotta website now!