Using Hazelcast With CFML

Recently I’ve been experimenting with Hazelcast, trying to wrap my head around various uses cases and how I might take advantage of the technology in some of our projects. If you’re unfamiliar, Hazelcast provides a very high speed in-memory storage and processing system that can be connected to many different types of systems. It’s commonly used in things like fraud detection for credit card transactions, when a lot of data and calculations have to be processed as quickly as possible.

Hazelcast is a Java app, and with CFML running on top of Java as well, we can use Hazelcast from within a CFML application by calling the Java code directly, and making a few modifications to the Java examples in the documentation. Given that the Java examples are based on a desktop-style application and not an HTTP request/response model, there are a few other items to account for along the way, but it’s not terribly difficult to get most of Hazelcast configured and running inside CFML.

(You can download all of the sample code referenced in this article from this GitHub repo: https://github.com/nolanerck/hazelcast-cfml-demo)

Installing and Running Hazelcast Locally

Installing Hazelcast on your web server is really straight forward.

1. Go to Hazelcast.com
2. Click “Get Hazelcast”
3. Scroll down and click the “Platform Open Source” button, which takes you to the “Download” page: https://hazelcast.com/open-source-projects/downloads/
4. Download the Hazelcast ZIP file. (For this demo I’m using version 5.0.2.)

Unzip this file into a folder that’s accessible to your CFML server. For simplicity, I’ve created a folder called “/hazelcast” under the webroot. Now we need to load the Hazelcast .jar file so it’s available in our CFML application. The easiest way to do that is via the “this.javaSettings” parameter in Application.cfc, like so:

this.javaSettings = {
loadPaths: [ expandPath( "/hazelcast/lib/hazelcast-5.0.2.jar" ) ]
};

Now you should have the core Hazelcast things available for use. To start Hazelcast, you’ll need to create an instance of the Hazelcast Config object, and an instance of Hazelcast engine itself, and connect them together, like so:

config = createObject( "java", "com.hazelcast.config.Config" ).init();
hazelcast = createObject( "java", "com.hazelcast.core.Hazelcast" );
hz1 = hazelcast.newHazelcastInstance( config );

Now we have an instance of Hazelcast saved in the variable “hz1” that we can start filling with data:

_map = hz1.getMap( "my-map" );
_map.put( "1", "John" );
_map.put( "2", "Paul" );
_map.put( "3", "George" );

Here we've created an IMap inside Hazelcast called “my-map” and added 3 items into it. This is a simple map and works the same way any other key-value data structure works. I can output these 3 items using the .get() method:

<cfoutput>
<div>Item 1: #_map.get( "1" )#</div>
<div>Item 2: #_map.get( "2" )#</div>
<div>Item 3: #_map.get( "3" )#</div>
</cfoutput>

Which will give you the output:

This works the same as any other “Map” data structure you might be familiar with. In CFML, this is basically the same as a Struct (except the data is being stored in Hazelcast directly). So if I were to re-use the key “1”:

_map.put( "1", "Ringo" );

And output the contents of my map again, you’ll see that “John” was replaced with “Ringo”:

(You can find a working example of the code above in the “1_connect_local” folder of the Git repo.)

Starting Hazelcast in Application.cfc

That last demo worked, but you might have noticed that the code in newInstance.cfm took several seconds to run, which is a pretty long time for a single .cfm file that’s not really doing much. The reason for that is, we’re initializing the Hazelcast engine every time we run that file, and the initializing takes a while. Those first few lines really should be moved into the Application.cfc so that they only happen once when our app starts up, instead of on every page request:

public any function onApplicationStart()
{
var config = createObject( "java", "com.hazelcast.config.Config" ).init();
var hazelcast = createObject( "java", "com.hazelcast.core.Hazelcast" );
application.hz1 = hazelcast.newHazelcastInstance( config );
return true;
}

Now, in any of our app’s .CFM files, we can create maps and add data in them without having to wait for the overhead of starting Hazelcast. Additionally we’ve now moved our “hz1” variable into the Application scope so it’s available in the entire application, not just the current .CFM file.

With Hazelcast being loaded into the global Application scope, we now have full access to any IMap data, no matter which .CFM file was used to create the data.

Take a look at add_data1.cfm:

<cfscript>
_map = application.hz1.getMap( "my-map" );
_map.put( "1", "John" );
_map.put( "2", "Paul" );
_map.put( "3", "George" );
_map.put( "4", "Ringo" );
</cfscript>
<cfoutput>
<div>Item 1: #_map.get( "1" )#</div>
<div>Item 2: #_map.get( "2" )#</div>
<div>Item 3: #_map.get( "3" )#</div>
<div>Item 4: #_map.get( "4" )#</div>
</cfoutput>

While we’re using the local variable “_map” to access the IMap, Hazelcast is stored globally and map is acting as a pointer back to application.hz1, so any data we save via map.put() in this file is being saved into the “my-map” inside Hazelcast. So we can then access that same data in view_data2.cfm like so:

<cfset _map = application.hz1.getMap( "my-map" ) />
<cfoutput>
<div>Item 1: #_map.get( "1" )#</div>
<div>Item 2: #_map.get( "2" )#</div>
<div>Item 3: #_map.get( "3" )#</div>
<div>Item 4: #_map.get( "4" )#</div>
</cfoutput>

And we’ll get the expected output:

Using Multiple Maps

Now let’s make a second map:

<cfscript>
_movieMap = application.hz1.getMap( "movies" );
_movieMap.put( "comedy", "Caddyshack" );
_movieMap.put( "horror", "Friday The 13th" );
_movieMap.put( "musical", "Grease" );
_movieMap.put( "action", "Leathal Weapon" );
_movieMap.put( "fantasy", "The Princess Bride" );
</cfscript>
<cfoutput>
<div>Movie 1: #_movieMap.get( "comedy" )#</div>
<div>Movie 2: #_movieMap.get( "horror" )#</div>
<div>Movie 3: #_movieMap.get( "musical" )#</div>
<div>Movie 4: #_movieMap.get( "action" )#</div>
<div>Movie 5: #_movieMap.get( "fantasy" )#</div>
</cfoutput>

Here you can see we’re using names instead of integers as our keys. Remember this works like any other key-value data structure. So just like in a CFML struct I can say:

_movieStruct = {
comedy: “Caddyshack”,
horror: “Friday The 13th”,
musical: “Grease”,
action: “Leathal Weapon”,
fantasy: “The Princess Bride”
}

...I can do the same thing with a Map in Hazelcast.

And just like in view_data1.cfm above with the previous example, I can access this Map from any other file in my application:

<cfset _movieMap = application.hz1.getMap( "movies" ) />
<cfoutput>
<div>Movie 1: #_movieMap.get( "comedy" )#</div>
<div>Movie 2: #_movieMap.get( "horror" )#</div>
<div>Movie 3: #_movieMap.get( "musical" )#</div>
<div>Movie 4: #_movieMap.get( "action" )#</div>
<div>Movie 5: #_movieMap.get( "fantasy" )#</div>
</cfoutput>

And of course, I can access both Maps in the same file I want to as well:

<cfset _map = application.hz1.getMap( "my-map" ) />
<cfoutput>
<h2>The Beatles: </h2>
<div>#_map.get( "1" )#</div>
<div>#_map.get( "2" )#</div>
<div>#_map.get( "3" )#</div>
<div>#_map.get( "4" )#</div>
</cfoutput>
<cfset _movieMap = application.hz1.getMap( "movies" ) />
<cfoutput>
<h2>Some Fun Movies: </h2>
<div>#_movieMap.get( "comedy" )#</div>
<div>#_movieMap.get( "horror" )#</div>
<div>#_movieMap.get( "musical" )#</div>
<div>#_movieMap.get( "action" )#</div>
<div>#_movieMap.get( "fantasy" )#</div>
</cfoutput>

Which gives us:

(You can find a working example of the code above in the “2_connect_local2” folder of the Git repo.)

Using SQL With Hazelcast

Hazelcast has a SQL interface that can also be used for manipulating the Maps, and that SQL syntax can be used via CFML just like it can in Java. To access the SQL functionality, we need to configure a couple of things.

First, we need to load the hazelcast-sql JAR file in our Application.cfc, so our this.javaSettings parameter will now be:

this.javaSettings = {
loadPaths: [ expandPath( "/hazelcast/lib/hazelcast-5.0.2.jar" ),
expandPath( "/hazelcast/lib/hazelcast-sql-5.0.2.jar" ) ]
};

Second, we’ll need to enable the Jet engine so Hazelcast can process the SQL statements. This is done as part of the Hazelcast config, so we’ll modify our onApplicationStart() method accordingly:

public any function onApplicationStart()
{
var config = createObject( "java", "com.hazelcast.config.Config" ).init();
var hazelcast = createObject( "java", "com.hazelcast.core.Hazelcast" );
var jetConfig = config.getJetConfig();
jetConfig.setEnabled( true );
application.hz1 = hazelcast.newHazelcastInstance( config );
return true;
}

Now we have access to the com.hazelcast.sql.SqlStatement component, which is what you use to run SQL commands against Hazelcast. Before we can use SqlStatement we have to initialize it. Because the SqlStatement’s constructor cannot be empty, we have to pass it some sort of valid SQL command to begin using the object. So for our first demo, I’ll use that as a place to create an IMap:

objSqlStatement = createObject( "java",
"com.hazelcast.sql.SqlStatement" )
.init( "CREATE OR REPLACE MAPPING myDistributedMap TYPE IMap OPTIONS
('keyFormat'='varchar','valueFormat'='varchar')" );

Not my favorite syntax, but it works! (After you’re comfortable with using Hazelcast in your CFML codebase, this might be a good place to write a “helper” CFC, so the SQL/Hazelcast syntax is less verbose, IMHO.) Once it’s created and initialized, you can reuse that existing SqlStatement object to run other SQL commands, and run them all with the sql.execute() method:

sql = application.hz1.getSql();
sql.execute( objSqlStatement.setSql( "SINK INTO myDistributedMap VALUES('1', 'John')" ) );
sql.execute( objSqlStatement.setSql( "SINK INTO myDistributedMap VALUES('2', 'Paul')" ) );
sql.execute( objSqlStatement.setSql( "SINK INTO myDistributedMap VALUES('3', 'George')" ) );
sql.execute( objSqlStatement.setSql( "SINK INTO myDistributedMap VALUES('4', 'Ringo')" ) );

That will add 4 items into our “myDistributedMap” IMap, the same way it would if we were using the SQL command line interface to talk to Hazelcast directly. (Again, I find this syntax a bit noisy, and would probably write some sort of “helper” library or Custom Tag to reduce the verbosity.)

You can of course also use SELECT statements to retrieve data out of an IMap:

result = sql.execute( objSqlStatement.setSql( "SELECT * FROM myDistributedMap" ) );

And because Hazelcast is built in Java, this “result” object we get back is an Iterator, and has the same Iterator methods you’d expect: next(), hasNext(), etc:

<cfset it = result.iterator() />
<cfoutput>
<h1>The Beatles</h1>
<cfloop condition="#it.hasNext()#">
<cfset item = it.next() />
<div>#item.getObject( 1 )#</div>
</cfloop>
</cfoutput>

That will output:

If you’ve ever used Mura CMS (or the new fork of it, Masa), this Iterator syntax is the same way those CMS engines provide a lot of functionality, so this might familiar to you.

Sorting works the same way you’d expect, with ORDER BY statements:

sql.execute( objSqlStatement.setSql( "SELECT * FROM myDistributedMap ORDER BY this ASC" ) );

Filtering with WHERE statements also works:

sql.execute( objSqlStatement.setSql( "SELECT * FROM myDistributedMap WHERE this LIKE '%Jo%'" ) );

Basically any of the standard stuff you’d do with a SELECT statement against a SQL database, you can also do here against a Hazelcast IMap, including aggregate functions: GROUP BY, HAVING, and so on.

(You can find a working example of the code above in the “3_sql_demo” folder of the Git repo.)

Hazelcast In The Cloud

Instead of running the Hazelcast engine on your own server, it can also be run in the cloud via cloud.hazelcast.com — it’s free to setup an account and get started with a cluster (certain features are only available with a paid account). After you’ve created an account and logged in at cloud.hazelcast.com, you’ll see a screen that looks about like so:

If your cluster is noted as “stopped”, click the “Manage Cluster” menu option and resume the cluster — this will take a few seconds to spin up. Once the cluster is running, we can write some CFML code that connects to Hazelcast from our web app.

Notice that the Cluster Metrics section in Hazelcast looks like this:

“Client Count” is 0. Nothing is talking to our Hazelcast app...yet!

In order for our CFML client to connect to Hazelcast in the cloud, we’ll need our Cluster Name and Discovery Token. In Hazelcast, click the “Connect Your Application” button, go to the “Java” tab, and scroll to the bottom, you’ll see a section that looks about like so:

Save the Cluster Name and Discovery Token somewhere so you can plug them into your code. Connecting with a client application to the cloud is a little different than connecting straight to the Hazelcast system when it runs locally. Instead of using the Config class, we’ll need ClientConfig. Also, we won’t be making an instance of the Hazelcast object, that’s what’s running in the cloud; instead we’ll be using HazelcastClient. HazelcastClient needs the Cluster Name and Discovery Token we saved earlier. Wiring everything together in CFML looks like this:

clientProperty = createObject( "java", "com.hazelcast.client.properties.ClientProperty" );
STATISTICS_ENABLED = clientProperty.STATISTICS_ENABLED;
HAZELCAST_CLOUD_DISCOVERY_TOKEN = clientProperty.HAZELCAST_CLOUD_DISCOVERY_TOKEN;
config = createObject( "java", "com.hazelcast.client.config.ClientConfig" ).init();
config.setProperty( STATISTICS_ENABLED.getName(), "true" );
config.setProperty( HAZELCAST_CLOUD_DISCOVERY_TOKEN.getName(), application.discoveryToken );
config.setClusterName( application.clusterName );
hazelcastClient = CreateObject( "java", "com.hazelcast.client.HazelcastClient" ).newHazelcastClient( config );

Much like our earlier code samples, in a real app we’ll put this code in the onApplicationStart() method, and use “application.hazelcastClient” as the variable so that we can reference our connection to Hazelcast from anywhere in the CFML code. (You can see I’m also storing the Cluster Name and Discovery Token in Application variables, which you don’t have to do, this is just for demo purposes — in a Production app, I’d probably use a secrets manager of some kind, for better security).

Let’s combine that with the getMap() code we looked at earlier:

_map = hazelcastClient.getMap( "musicians" );
_map.put( "drummer", "Ringo" );
_map.put( "bass", "Paul" );
_map.put( "guitar", "George" );
_map.put( "vocals", "John" );

If we put all of this in a file named connect.cfm and run it, then go back to your Hazelcast Dashboard, you should see this:

See that line in the “Client Count” graph jump up to 1? Our CFML app is talking to the Hazelcast Cloud!

Below that, you should now also have something that looks about like the following:

I’ve got a Map Metrics section set to “musicians”, the Entry Count is 4 (because we added 4 things into our Map), and you’ll see a couple of the other graphs now have data in them as well (technically they all do but some of them barely register because we’ve only got a tiny amount of data being stored right now).

(You can find a working example of the code above in the “4_cloud_connect” folder of the Git repo.)

Going Farther

There are many more features available in Hazelcast, this is just the beginning of how to use it. A few features require code be written directly in Java for them to work. There’s probably a way to get CFML compiled down into Java byte code in the format that Hazelcast requires, but that’s a more involved use case and I’m still digging into the details on how to make this work.

Also, some features are kind of expecting the client code to be running in a stateful application, rather than a request/response HTTP style codebase. We can probably still use CFML to do that by just running it in CommandBox as some sort of CLI script or task runner, but again that’s more involved and could be its own series of blog posts just for that info alone.

Enjoy!