Posts
We were able to get our integration tests for consul to run in the Travis CI server.
We use consul as a service discovery system for our Microservice lib (QBit microservices). This allows us to get a list of healthy service peers that are up and able to handle streams of calls.
The trick was to get consul running on our integration server. We use Travis.
We use consul as a service discovery system for our Microservice lib (QBit microservices). This allows us to get a list of healthy service peers that are up and able to handle streams of calls.
The trick was to get consul running on our integration server. We use Travis.
First we define a new task and then check to see if the consul executable already exist.
Once we ensure consul exists, we execute it with the right command line args.
We are using gradle to manage our build. It seems to be the most flexible.
Once we ensure consul exists, we execute it with the right command line args.
We are using gradle to manage our build. It seems to be the most flexible.
def execFile = new File(project.rootDir,
'/tmp/consul/bin/consul')
def zipFile = new File(project.rootDir, '/tmp/consul/bin/consul.zip')
Then we need to find the right OS. (We only build on Linux and Mac.)
def linuxDist = "https://dl.bintray.com/mitchellh/consul/0.5.0_linux_amd64.zip"
def macDist = "https://dl.bintray.com/mitchellh/consul/0.5.0_darwin_amd64.zip"
def zipURL = null
Create the parent folder to hold the zip and the bin.
execFile.parentFile.mkdirs()
if (execFile.parentFile.exists()) {
println("${execFile.parentFile} created" );
}
Then we see if we can find the type of OS. We only support 64 bit Linux, but as you can see, we could add more.
if (System.getProperty("os.name").contains("Mac OS X")) {
zipURL = macDist
println("On mac")
} else {
zipURL = linuxDist
println("On linux")
def osArc = System.getProperty("sun.arch.data.model")
def osName = System.getProperty("os.name")
def osVersion = System.getProperty("os.version")
println("os.arc Operating system architecture\t\t $osArc")
println("os.name Operating system name\t\t $osName")
println("os.version Operating system version\t\t $osVersion")
}
There are lots of println(s) because who know where someone might try to run this.
Copy the zip file from the URL:
new URL(zipURL).withInputStream{ i -> zipFile.withOutputStream{ it << i }}
for (int index = 0; index < 10; index++) {
ant.sleep(seconds: 1)
if (zipFile.exists()) {
break;
}
println("Waiting for download $zipURL" )
}
If the zip file exists, then unzip it, and change permissions so it is executable.
if (zipFile.exists()) {
println("${zipFile} ${zipFile.absoluteFile} ${zipFile.exists()} ${zipFile.size()}")
println(execFile.parentFile)
ant.unzip(src: zipFile, dest: execFile.parentFile)
ant.exec(command: "/bin/sh -c \"chmod +x ${execFile}\"")
} else {
println("Unable to create file $zipFile from $zipURL")
}
Lot's of debugging info.
Once we create the zip, unpack it, change permissions, then we can execute consul.
if (!execFile.exists()) {
findItUnpackIt()
}
ant.exec(command: "/bin/sh -c \"${execFile} \
agent -server -bootstrap-expect 1 \
-data-dir /tmp/consul\"",
spawn: true)
Pause for a bit to let consul run before we start our tests:
for (int index = 0; index < 10; index++) {
ant.sleep(seconds: 1)
ant.echo(message: "Waiting for consul $index")
}
The next trick was to wire this into the consul client subproject.
project('cluster:consul-client') {
dependencies {
compile project(":qbit:web:jetty")
}
task runConsul(type: RunConsul) << {
println 'task'
}
test.dependsOn(runConsul)
...
}
Here is the full task to RunConsul.
class RunConsul extends DefaultTask {
def execFile = new File(project.rootDir,
'/tmp/consul/bin/consul')
def zipFile = new File(project.rootDir, '/tmp/consul/bin/consul.zip')
def linuxDist = "https://dl.bintray.com/mitchellh/consul/0.5.0_linux_amd64.zip"
def macDist = "https://dl.bintray.com/mitchellh/consul/0.5.0_darwin_amd64.zip"
def zipURL = null
def findItUnpackIt() {
execFile.parentFile.mkdirs()
if (execFile.parentFile.exists()) {
println("${execFile.parentFile} created" );
}
if (System.getProperty("os.name").contains("Mac OS X")) {
zipURL = macDist
println("On mac")
} else {
zipURL = linuxDist
println("On linux")
def osArc = System.getProperty("sun.arch.data.model")
def osName = System.getProperty("os.name")
def osVersion = System.getProperty("os.version")
println("os.arc Operating system architecture\t\t $osArc")
println("os.name Operating system name\t\t $osName")
println("os.version Operating system version\t\t $osVersion")
}
new URL(zipURL).withInputStream{ i -> zipFile.withOutputStream{ it << i }}
for (int index = 0; index < 10; index++) {
ant.sleep(seconds: 1)
if (zipFile.exists()) {
break;
}
println("Waiting for download $zipURL" )
}
if (zipFile.exists()) {
println("${zipFile} ${zipFile.absoluteFile} ${zipFile.exists()} ${zipFile.size()}")
println(execFile.parentFile)
ant.unzip(src: zipFile, dest: execFile.parentFile)
ant.exec(command: "/bin/sh -c \"chmod +x ${execFile}\"")
} else {
println("Unable to create file $zipFile from $zipURL")
}
}
@TaskAction
void runIt() {
if (!execFile.exists()) {
findItUnpackIt()
}
ant.exec(command: "/bin/sh -c \"${execFile} agent -server -bootstrap-expect 1 -data-dir /tmp/consul\"",
spawn: true)
for (int index = 0; index < 10; index++) {
ant.sleep(seconds: 1)
ant.echo(message: "Waiting for consul $index")
}
}
}
Reactive Programming, Java Microservices, Rick Hightower
More info on QBit
Home[Detailed Tutorial] QBit microservice example
[Detailed Tutorial] Building a HTTP server with WebSocket support using QBit (very easy)
[Detailed Tutorial] Building a simple recommendation engine with QBit (CallBack Blocking)
[Detailed Tutorial] Building a simple recommendation engine with QBit (CallBack nonBlocking)
[Detailed Tutorial] Building a single page; Todo List Application with QBit
[Detailed Tutorial] Using event channels and strongly typed event bus with QBit (The employee example)
[Detailed Tutorial] Using QBit Microservice lib with Spring Boot
[Detailed Tutorial] Using QBit microservice lib's REST support with URI Params
[Detailed Tutorial] Using QBit's Event Bus System (The Employee example)
[Detailed Tutorial] Working with inproc MicroServices within QBit.
[Doc] QBit Java microservice lib auto flushing service queue proxies
[Doc] QBit Java microservice lib introducing EventBus replication and EventBus connectors
[Doc] QBit microservice Java lib Using auto flushing client proxies
[Doc] QBit Microservice WebSocket wire protocol draft 01 JSON ASCII method batching RPC
[Doc] Queue Callbacks for QBit queue based services
[Doc] Using QBit microservice lib's HttpClient GET, POST, et al, JSON, Java 8 Lambda
[Doc] Using QBit microservice lib's WebSocket support
[Quick Start] Building a HTTP server with WebSocket support using QBit (very easy)
[Quick Start] Building a simple recommendation engine with QBit (CallBack Blocking)
[Quick Start] Building a simple recommendation engine with QBit (CallBack nonBlocking)
[Quick Start] Building a simple Rest web microservice server with QBit
[Quick Start] Building a single page; Todo List Application with QBit
[Quick Start] building a single web page application with QBit
[Quick Start] Building a TODO web microservice client with QBit
[Quick Start] Building a TODO web microservice server with QBit
[Quick Start] Building boon for the QBit microservice engine
[Quick Start] Building QBit the microservice lib for Java
[Quick Start] Using event channels and strongly typed event bus with QBit (The employee example)
[Quick Start] Using QBit Microservice lib with Spring Boot
[Quick Start] Using QBit microservice lib's REST support with URI Params
[Quick Start] Using QBit's Event Bus System (The Employee example)
[Quick Start] Working with inproc MicroServices within QBit
[Rough Cut] Delivering up Single Page Applications from QBit Java JSON Microservice lib
[Rough Cut] QBit Java Microservice Lib Working With Workers Sharded and Pooled
[Rough Cut] QBit Microservice Lib Working With CallBacks
[Rough Cut] QBit Microservices using Service Workers and sharded service workers
[Rough Cut] Using QBit microservice lib with Spring Boot
[Rough Cut] Using QBit microservice lib's REST support with URI Params
[Rough Cut] Working with event bus for QBit the microservice engine
[Rough Cut] Working with Event Channels and strongly typed event bus with the QBit Microservice Java, JSON, WebSocket Lib
[Rough Cut] Working with inproc MicroServices
[Rough Cut] Working with private event bus for inproc microservices
[Rough Cut] Working with strongly typed event bus proxies for QBit Java Microservice lib
[Rough Cut] Working with System Manager for QBit Mircoservice lib
[Z Blog Getting Consul to run on Travis CI Server using Gradle so we can test consul integration tests]
[Z Blog] Qbit servlet support also a preview of using jetty and the QBit microservice lib together
[Z Blog] Using Consul from QBit, the Java, JSON, WebSocket and REST microservice library, for health monitoring, clustering and service discovery
[Z Dev Notes] Reactive programming for QBit Java microservices library
[Z Notebook] More benchmarking internal
[Z Notebook] Performance testing for REST
[Z Notebook] Roadmap
Introduction to QBit
Local Service Proxies
OLD Home
QBit Boon New Wave of JSON HTTP and Websocket
QBit Docs
Home
No comments:
Post a Comment