Rick

Rick
Rick

Friday, April 18, 2014

Need help with gradlew for a vertx fork and PR - it works, but I want it to be better

I could use some help. I am new to Gradle. And my gradle looks a lot more like ant than gradle.
Background
Hazelcast 3.2 has critical bug fixes and performance enhancements.

Vert.x is currently 6 minor releases behind, and one major release. 

Projects that are using Hazelcast 3.x are unable to deploy on Vert.x due to classpath issues.

https://groups.google.com/forum/#!msg/vertx/3M77Zixjab8/BlDzGIMtuGIJ

So I try to be a good open source community guy. I join eclipse foundation, jump through a few hoops, and write a bug report and then submit a PR. I think this like my 3rd PR so I won't I am a newbie, but I am a newbie. (I get more PRs than I send.)
I write the list. A few people agree... and then.. I get this.
RE: I think only for Vert.x 3.x or as an extra jar maybe ? --Norman Maurer
This makes sense actually. The issue is my gradle skill are not full ninja with gradle.

They are more like half ninja or 1/4 ninja or maybe just cartoon ninja.

But lack of knowledge has never stopped me before so...
I did it.
DONE!
Cue the large aircraft carrier with the banner.

Ok, so it works, but I want it to better so I was hoping that a true gradle ninja would look at the script and help a brother out.
Now back to what I did....
I modified the build script to produce two jar files. One for vertx-hazelcast and one for vertx-hazelcast3.
In addition:
  • modified testsuite to use hazelcast2 or hazelcast3
  • created two sets of cluster xml files
  • modified the build script to create two vertx distributions (one for hazelcast2 and one for hazel cast 3)
  • modified script creates tar.gz and zip file for new distribution with the correct hazelcast jar files copied in
See:

You can see the diffs here so you can get a good idea of before and after.
Hazelcast 3.2 has some critical bug fixes in it.
To use Hazelcast 3.2 with vertx use the distribution file called:
vert.x.hazel3cluster-${version}.zip
Or
vert.x.hazel3cluster-${version}.tar.gz
The files
vert.x-${version}.zip
Or
vert.x-${version}.tar.gz
Contain Hazelcast 2.x support.
In vert.x.hazel3cluster-${version} you will find:
.

├── api-docs
├── bin
│   ├── vertx
│   └── vertx.bat
├── client
│   └── vertxbus-2.1.js
├── conf
│   ├── META-INF
│   │   └── services
│   │       └── org.vertx.java.platform.PlatformManagerFactory
│   ├── cluster.xml
│   ├── default-cluster.xml
│   ├── langs.properties
│   ├── logging.properties
│   └── repos.txt
└── lib
    ├── hazelcast-3.2.jar
    ├── jackson-annotations-2.2.2.jar
    ├── jackson-core-2.2.2.jar
    ├── jackson-databind-2.2.2.jar
    ├── netty-all-4.0.18.Final.jar
    ├── vertx-core-2.1RC4-SNAPSHOT.jar
    ├── vertx-hazelcast3-2.1RC4-SNAPSHOT.jar
    └── vertx-platform-2.1RC4-SNAPSHOT.jar
The key here is that hazelcast-3.2.jar and vertx-hazelcast3-2.1RC4-SNAPSHOT.jar are included.
In vert.x-${version} you will find:
.
├── api-docs
├── bin
│   ├── vertx
│   └── vertx.bat
├── client
│   └── vertxbus-2.1.js
├── conf
│   ├── META-INF
│   │   └── services
│   │       └── org.vertx.java.platform.PlatformManagerFactory
│   ├── cluster.xml
│   ├── default-cluster-hazelcast-3.xml
│   ├── langs.properties
│   ├── logging.properties
│   └── repos.txt
└── lib
    ├── hazelcast-2.6.7.jar
    ├── jackson-annotations-2.2.2.jar
    ├── jackson-core-2.2.2.jar
    ├── jackson-databind-2.2.2.jar
    ├── netty-all-4.0.18.Final.jar
    ├── vertx-core-2.1RC4-SNAPSHOT.jar
    ├── vertx-hazelcast-2.1RC4-SNAPSHOT.jar
    └── vertx-platform-2.1RC4-SNAPSHOT.jar
The key here is that hazelcast-3.2.jar and vertx-hazelcast3-2.1RC4-SNAPSHOT.jar are not included,
hazelcast-2.6.7.jar and vertx-hazelcast-2.1RC4-SNAPSHOT.jar are included.
What follows is some documentation of changes that were made to gradle builds files and vertx to support this effort.

new vertx subproject called vertx-hazelcast3

A new gradle jar project was created called 'vertx-hazelcast3'.
Vertx uses a service location pattern to load the in a pluggable cluster manager.
This fix employs the pluggable nature of vertx cluster manager to plugin Hazelcast 3 instead of Hazelcast 2.

testing with Hazelcast 3

The gradle build can either test hazelcast 3 or hazelcast2.
You specify testing with hazelcast3 by passing gradlew -p as follows:
./gradlew -Ph3 clean test
Or you can pass -Phazelcast3 as follows:
./gradlew -Phazelcast3 clean test
If you do not specify -Ph3 or -Phazelcast3, then hazelcast 2.x will be used for testing.

Chagnes to gradle.properties

gradle.properties has a new property called hazelcastVersion3.
gradle.properties.
...

# dependency versions
hazelcastVersion=2.6.7
hazelcastVersion3=3.2
jacksonCoreVersion=2.2.2
jacksonDatabindVersion=2.2.2

...
This determines which version of hazelcast 3 that gradle will build with.

Changes to gradle.build

In gradle.build we added two new tasks that the assemble task depends on as follows:
task assemble(type: Copy, dependsOn: ['assembleHazelcast2', 'assembleHazelcast3']) {
...
These tasks aid in building the distribution folder that gets turned into a zip file and a tar.gz file.
They ensure that only hazelcast2 or hazelcast3 jar files are included to avoid classpath dependency issues.
Similarly the dist task has been modified to depend on two additional tasks, namely distZipHazelcast3and distTarHazelcast3.
task dist(dependsOn: ['distTar', 'distZip', 'distZipHazelcast3', 'distTarHazelcast3']){
...
These task include the correct distribution folder.
Changes to testsuites
The vertx-testsuites either run hazelcast3 cluster or hazlecast2 cluster by conditionally including the correct dependency based on the h3 and hazelcast3.

Note I tired using a property like compile project(':${myprop}') but gradle gave me the one finger solute.

if (hasProperty("h3") || hasProperty("hazelcast3")) {

    println("YOU ARE TESTING WITH HAZELCAST 3!")

    dependencies {
        compile project(':vertx-core')
        compile project(':vertx-platform')
        compile project(':vertx-hazelcast3')
    }
} else {


    println("normal test")
    dependencies {
        compile project(':vertx-core')
        compile project(':vertx-platform')
        compile project(':vertx-hazelcast')
    }
}

Kafka and Cassandra support, training for AWS EC2 Cassandra 3.0 Training