Posts
This step was done after adding zookeeper clustering, but it depends on version of the branch that predates the zookeeper cluster manager. We do this because the zookeeper support is not in the maven public repo yet, and we want the master branch build to work.
The code for this is in this branch.
Adding Docker Support Begin with the end in mind.
We want to be able to launch this service via Docker as follows:
Build and install docker instance
$ gradle clean shadowJar buildDocker
Run docker instance
$ docker run advantageous/vertx:1.0
Test the instance with curl
$ echo $DOCKER_HOST
tcp://192.168.99.100:2376
$ curl http://192.168.99.100:5000/hello-world/?msg="rick"
Hello World! rick
Setup docker on your dev box (assuming OSX, adjust accordingly)
There is an installation guide to using docker on OSX. You will need to install the Docker Toolbox, and run through some of the tutorial (even if you used boot2docker before as a lot has changed). Once you feel comfortable with building and deploying images for Docker come back here.
Add docker support to gradle
We will need to add the gradle docker plugin.
buildscript {
repositories {
mavenCentral()
jcenter() //ADDED THIS
}
dependencies {
classpath 'org.jetbrains.kotlin:kotlin-gradle-plugin:1.0.0-beta-4584'
classpath 'se.transmode.gradle:gradle-docker:1.2' //ADDED THIS
}
}
Use the docker plugin
Apply plugin
apply plugin: 'docker'
Use plugin
group 'advantageous' //changed this
version '1.0' //and this
docker {
baseImage "vertx/vertx3-exec"
maintainer 'Rick Hightower "richardhightowerATgmailDOTcom"'
}
Create build task to build and install docker
task buildDocker(type: Docker) {
tagVersion = System.getenv("BUILD_NUMBER") ?: project.version
push = Boolean.getBoolean("docker.push")
applicationName = project.applicationName
addFile {
from "${project.shadowJar.outputs.files.singleFile}"
into "/opt/hello/"
}
exposePort 8080
entryPoint = ["sh", "-c"]
defaultCommand = ["java -jar /opt/hello/${project.name}-${project.version}-fat.jar"]
}
The above will both create the
Dockerfile and install docker in your local docker instance. It assumes docker is setup correctly on your build box.Contents of generated Dockerfile
cat ./build/docker/Dockerfile
FROM vertx/vertx3-exec
MAINTAINER Rick Hightower "richardhightowerATgmailDOTcom"
ADD add_1.tar /
EXPOSE 8080
ENTRYPOINT ["sh", "-c"]
CMD ["java -jar /opt/hello/vertx-1.0-fat.jar"]
Deploying to Mesos / Marathon
We setup a Mesos master and two Mesos Slaves with Marathon and Zookeeper all up in our private EC2 VPC using this guide. We also setup an DockerHub public organization called
advantageous.
This should work:
Forcing push to DockerHub
gradle -Ddocker.push=true clean shadowJar buildDocker
But instead it takes FOREVER.
Forcing push to DockerHub
gradle clean shadowJar buildDocker
docker push advantageous/vertx-node-eventbus-example
Note I changed the name of the Docker container from
vertx to vertx-node-eventbus-example.Deploying to Mesos / Marathon Create deploy file
cat VertxNodeDocker.json
{
"container": {
"type": "DOCKER",
"docker": {
"image": "advantageous/vertx-node-eventbus-example:1.0"
}
},
"id": "myimage",
"instances": 1,
"cpus": 0.5,
"mem": 512,
"uris": [],
"cmd": "while sleep 10; do date -u +%T; done"
}
Use curl to deploy Docker image you stored in DockerHub to Marathon/Mesos
curl -X POST -H "Content-Type: application/json" http://10.0.0.148:8080/v2/apps -d@VertxNodeDocker.json
For this we setup three Mesos instances (1 master and two slaves) in EC2 in a VPC (isolated cloud resources).
The two slaves (called Agents in Mesos 1.0) point to the one master (for QA, prod you would want at least three master mesos instances)
Slave boxes pointing to master mesos instances.
$ cat /etc/mesos/zk
zk://10.0.0.148:2181/mesos
There are plenty of guides to show how to install Meso and Docker on Linux. Follow those guides and setup your own Mesos master.
Debugging
Logs for Mesos are found in /var/log/mesos and /var/log/zookeeper.
To see a list of deployments in mesos.
List of deployments in Mesos
$ curl http://10.0.0.148:8080/v2/deployments/ | jq .
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 317 0 317 0 0 115k 0 --:--:-- --:--:-- --:--:-- 154k
[
{
"totalSteps": 2,
"currentStep": 2,
"currentActions": [
{
"app": "/myimage",
"action": "ScaleApplication"
}
],
"steps": [
[
{
"app": "/myimage",
"action": "StartApplication"
}
],
[
{
"app": "/myimage",
"action": "ScaleApplication"
}
]
],
"affectedApps": [
"/myimage"
],
"version": "2016-01-23T00:12:39.099Z",
"id": "176ae76a-3f3a-4c6f-a9b6-b539214a4ee9"
}
]
To delete an image, you hit the mesos REST endpoint as follows:
Delete a deployment
$ curl -X DELETE http://10.0.0.148:8080/v2/deployments/176ae76a-3f3a-4c6f-a9b6-b539214a4ee9 | jq .
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 92 0 92 0 0 6028 0 --:--:-- --:--:-- --:--:-- 6133
{
"deploymentId": "dd370d17-3b9e-4682-a107-3b3bd018728c",
"version": "2016-01-25T19:45:16.584Z"
}
Not working
At this point, you deployed the vertx application but it is not able to run because port 8080 is already taken by Marathon. In order to get the vertx application to work, we will need to map its port 8080 with an outside port for the Marathon agent/slave box.
We can do this by following mesos marathon docker guide and crafting a deploy script.
"portMappings": [
{ "containerPort": 8080, "hostPort": 0,
"servicePort": 9000, "protocol": "tcp" }
]
This goes into the
docker area of the deploy VertxNodeDocker.json that we have been using.{
"container": {
"type": "DOCKER",
"docker": {
"image": "advantageous/vertx-node-eventbus-example:1.0",
"network": "BRIDGE",
"portMappings": [
{ "containerPort": 8080, "hostPort": 0,
"servicePort": 9000, "protocol": "tcp" }
]
}
},
"id": "vertx-node-eventbus-example",
"instances": 1,
"cpus": 0.5,
"mem": 512,
"uris": []
}
Undeploy the older version. Then deploy this image.
You can ssh to slave boxes and run
docker ps to see where it ended up.Hooks up fine
ubuntu@ip-10-0-0-188:~$ sudo docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
04e5b69a3bf9 advantageous/vertx-node-eventbus-example:1.0 "sh -c 'java -jar /op" 7 minutes ago Up 7 minutes 0.0.0.0:31627->8080/tcp mesos-00481535-158b-4812-aff3-bf02d4176ef8-S0.84d6bd59-c5cc-4c56-b2d6-e30af1ea11aa
ubuntu@ip-10-0-0-188:~$ curl http://10.0.0.188:31627/hello/
HELLO WORLD FROM KOTLIN
We looked up the ports and then used that information to make a call into the service.
To actually get this information from Marathon, we do this.
Getting service discovery information from Marathon
$ curl http://10.0.0.148:8080/v2/tasks/ | jq .
{
"tasks": [
{
"servicePorts": [
9000
],
"appId": "/vertx-node-eventbus-example",
"id": "vertx-node-eventbus-example.21e8f58e-c3aa-11e5-aace-02422bec481e",
"host": "ip-10-0-0-188.us-west-2.compute.internal",
"ipAddresses": [
{
"protocol": "IPv4",
"ipAddress": "172.17.0.2"
}
],
"ports": [
31627
],
"startedAt": "2016-01-25T21:25:19.459Z",
"stagedAt": "2016-01-25T21:25:18.576Z",
"version": "2016-01-25T20:47:56.140Z",
"slaveId": "00481535-158b-4812-aff3-bf02d4176ef8-S0"
}
]
}
Note that the port in
ports is the correct docker port for our HelloWorld sample. We can curl using this host and port.Working with host
$ curl http://ip-10-0-0-188.us-west-2.compute.internal:31627/hello/
HELLO WORLD FROM KOTLIN
Common commands I use to redeploy the example to mesos
I edit the version of the docker deploy after I make changes in the
build.gradle file.Build microserivce/docker image on my dev box and soon Jenkins
./gradlew clean shadowJar buildDocker
docker push advantageous/vertx-node-eventbus-example
After this, you will see the image on docker hub.
Deploy from any member of Amazon VPC
Edit deploy file and add version that I want to deploy
sudo nano VertxNodeDocker.json
Deploy file to deploy new version of docker image to Mesos
{
"id": "vertx-node-eventbus-example",
"instances": 1,
"cpus": 0.5,
"mem": 512,
"uris": [],
"container": {
"type": "DOCKER",
"docker": {
"image": "advantageous/vertx-node-eventbus-example:1.0.1.3",
"network": "BRIDGE",
"portMappings": [
{ "containerPort": 8080, "hostPort": 0,
"servicePort": 9000, "protocol": "tcp" }
]
}
}
}
I use curl to remove the old copy of the app install for this microservice from mesos.
Remove old docker container from mesos
curl -X DELETE http://10.0.0.148:8080/v2/apps/vertx-node-eventbus-example
Then I redeploy the service to mesos.
Redeploy docker image
curl -X POST -H "Content-Type: application/json" http://10.0.0.148:8080/v2/apps -d@VertxNodeDocker.json | jq .
Then I test the microservice but first I need to know where it was deployed and what port it is using.
Using mesos service discovery
curl http://10.0.0.148:8080/v2/tasks/ | jq .
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 439 0 439 0 0 92811 0 --:--:-- --:--:-- --:--:-- 107k
{
"tasks": [
{
"servicePorts": [
9000
],
"appId": "/vertx-node-eventbus-example",
"id": "vertx-node-eventbus-example.cf1d43a1-c6e5-11e5-bee9-02424e94d413",
"host": "ip-10-0-0-188.us-west-2.compute.internal",
"ipAddresses": [
{
"protocol": "IPv4",
"ipAddress": "172.17.0.2"
}
],
"ports": [
31516
],
"startedAt": "2016-01-30T00:10:10.010Z",
"stagedAt": "2016-01-30T00:10:02.932Z",
"version": "2016-01-30T00:10:02.901Z",
"slaveId": "71ad679f-18ef-4941-a912-69c6cd5adbd4-S1"
}
]
}
With the above host and port, we can now test our deploy.
Testing our deploy
curl http://10.0.0.188:31516/hello/
HELLO WORLD FROM KOTLIN
No comments:
Post a Comment