Rick

Rick
Rick

Friday, May 15, 2015

Docker and Gradle to create Java microservices part 2 Connecting containers with links

Docker and Gradle to create Java microservices part 2

In the last article, we used gradle and docker to create a docker container that had our Java application running in it. (We pick up right where we left off so go back to that if you have not read it.)
This is great, but microservices typically talk to other microservices or other resources like databases, so how can we deploy our service to talk to another microservice. The docker answer to this is docker container links
When we add links to another docker container, docker will add domain name alias to the other container so we don't have to ship IP addresses around. For more background about Docker Container Links check out their documentation.
This can help us with local integration tests and for onboarding new developers. This allows us to setup topologies of docker containers that collborate with each other over the network. 
For this example, we will have one Java application called client and one called server. The project structure will look just like that last project structure. 

Running the client application

docker run --name client --link server:server -t -i example/client:1.0-SNAP
Notice that we pass --link server:server, this will add a DNS like alias for server so we can configure the host address of our server app as server. In practice, you would want a more qualified name. When we start up the server, we will need to give it the docker name server with --name server so that the client can find its address.
Thus under images/etc/client/conf/conf.properties we would have:

Config for client app image images/etc/client/conf/conf.properties

port=9999
host=server

Client gradle build script

Our gradle build script for out client is essentially the same as our myapp example. The major difference is we now depend on QBit, which is a microservice lib that can easily work with HTTP clients and servers.

gradle build script for client

apply plugin: 'java'
apply plugin: 'application'
apply plugin: 'idea'


def installOptDir="/opt/client"
def installConfDir="/etc/client"

mainClassName = 'com.example.ClientMain'
applicationName = 'client'

applicationDefaultJvmArgs = [
        "-Dclient.config.file=/etc/client/conf.properties",
        "-Dlogback.configurationFile=/etc/client/logging.xml"]

repositories {
    mavenCentral()
}

task copyDist(type: Copy) {
    dependsOn "installDist"
    from "$buildDir/install/client"
    into installOptDir
}

task copyConf(type: Copy) {
    from "conf/conf.properties"
    into installConfDir

}


task copyLogConf(type: Copy) {
    from "conf/logging.xml"
    into installConfDir

}

task copyAllConf() {
    dependsOn "copyConf", "copyLogConf"

}

task installClient() {
    dependsOn "copyDist", "copyConf", "copyLogConf"

}

task copyDistToImage(type: Copy) {
    dependsOn "installDist"
    from "$buildDir/install/client"
    into "$projectDir/image/opt/client"
}


dependencies {

    compile group: 'io.advantageous.qbit', name: 'qbit-vertx', version: '0.8.2'
    compile 'ch.qos.logback:logback-core:1.1.3'
    compile 'ch.qos.logback:logback-classic:1.1.3'
    compile 'org.slf4j:slf4j-api:1.7.12'
}

The client main app

The client main class looks very similar to our myapp example. Except now it uses the host and port to connect to an actual server.
package com.example;

/**
 * Created by rick on 5/15/15.
 */

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.Properties;


import io.advantageous.boon.core.Sys;
import io.advantageous.qbit.http.client.HttpClient;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static io.advantageous.qbit.http.client.HttpClientBuilder.httpClientBuilder;

public class ClientMain {

    static final Logger logger = LoggerFactory.getLogger(ClientMain.class);

    public static void main(final String... args) throws IOException {

        final String configLocation = System.getProperty("client.config.file");
        final File confFile = configLocation==null  ?
                new File("./conf/conf.properties") :
                new File(configLocation);



        final Properties properties = new Properties();
        if (confFile.exists()) {
            properties.load(Files.newInputStream(confFile.toPath()));
        } else {
            properties.load(Files.newInputStream(new File("./conf/conf.properties").toPath()));
        }

        final int port = Integer.parseInt(properties.getProperty("port"));
        final String host = properties.getProperty("host");

        logger.info(String.format("The port is set to %d %s\n", port, host));



        final HttpClient httpClient = httpClientBuilder()
                .setHost(host).setPort(port).build();
        httpClient.start();

        for (int index=0; index< 10; index++) {
            System.out.println(httpClient.get("/foo/bar").body());
            Sys.sleep(1_000);
        }

        Sys.sleep(1_000);

        httpClient.stop();
    }

}
It connects to the server with httpClient and does 10 HTTP gets and prints the results out to System.out. The point here is that it is able to find the server and it does not hard code an IP address as both the client and server are Docker container instances which are ephemeral, elastic servers. 

Server application

The server uses the same gradle build file as myapp and the client examples. It uses the same image directory and Dockerfile as those examples as well.
Even the Main class looks similar to the early two examples as the focus in on gradle and Docker not our app per se.

ServerMain.java

package com.example;

import java.io.File;
import java.io.IOException;
import java.nio.file.Files;
import java.util.Properties;
import io.advantageous.qbit.http.server.HttpServer;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

import static io.advantageous.qbit.http.server.HttpServerBuilder.httpServerBuilder;

public class ServerMain {

    static final Logger logger = LoggerFactory.getLogger(ServerMain.class);

    public static void main(final String... args) throws IOException {

        final String configLocation = System.getProperty("server.config.file");
        final File confFile = configLocation==null  ?
                new File("./conf/conf.properties") :
                new File(configLocation);



        final Properties properties = new Properties();
        if (confFile.exists()) {
            properties.load(Files.newInputStream(confFile.toPath()));
        } else {
            properties.load(Files.newInputStream(
                    new File("./conf/conf.properties").toPath()));
        }

        final int port = Integer.parseInt(properties.getProperty("port"));


        HttpServer httpServer = httpServerBuilder()
                .setPort(port).build();


        httpServer.setHttpRequestConsumer(httpRequest -> {
            logger.info("Got request " + httpRequest.address()
                    + " " + httpRequest.getBodyAsString());
            httpRequest.getReceiver()
                    .response(200, "application/json", "\"hello\"");
        });


        httpServer.startServer();




    }

}
We just start a server and send a body of "hello" when called. 

Dockerfile

There was some changes to the Dockerfile to create the images. I ran into some issues with QBit and Open JDKs version number (which has been fixed but not released). I switched the example to use the Oracle 8 JDK.

Dockerfile using Oracle JDK 8

# Pull base image.
FROM ubuntu

# Install Java.
RUN echo oracle-java8-installer shared/accepted-oracle-license-v1-1 select true | debconf-set-selections

RUN  apt-get update
RUN  apt-get install -y software-properties-common python-software-properties
RUN  add-apt-repository ppa:webupd8team/java
RUN  apt-get update

RUN  apt-get install -y oracle-java8-installer
RUN  rm -rf /var/lib/apt/lists/*
RUN  rm -rf /var/cache/oracle-jdk8-installer


# Define working directory.
WORKDIR /data

# Define commonly used JAVA_HOME variable
ENV JAVA_HOME /usr/lib/jvm/java-8-oracle

COPY opt /opt
COPY etc /etc
COPY var /var


ENTRYPOINT /opt/client/bin/run.sh

Running server

To run the server docker container use the following:
docker run --name server -t -i example/server:1.0-SNAP
Note that we pass the --name server

Directory layout for client / server example

We don't repeat much from the first example, but hopefully the directory structure will shed some light on how we ogranized the applications and their docker image directories.
$ pwd
../docker-tut/networked-apps
$ tree
.
├── client
│   ├── build.gradle
│   ├── client.iml
│   ├── conf
│   │   ├── conf.properties
│   │   └── logging.xml
│   ├── gradle
│   │   └── wrapper
│   │       ├── gradle-wrapper.jar
│   │       └── gradle-wrapper.properties
│   ├── gradlew
│   ├── gradlew.bat
│   ├── image
│   │   ├── Dockerfile
│   │   ├── buildImage.sh
│   │   ├── env.sh
│   │   ├── etc
│   │   │   └── client
│   │   │       ├── conf.properties
│   │   │       └── logging.xml
│   │   ├── opt
│   │   │   └── client
│   │   │       ├── bin
│   │   │       │   ├── client
│   │   │       │   ├── client.bat
│   │   │       │   └── run.sh
│   │   │       └── lib
│   │   │           ├── boon-json-0.5.5.jar
│   │   │           ├── boon-reflekt-0.5.5.jar
│   │   │           ├── jackson-annotations-2.2.2.jar
│   │   │           ├── jackson-core-2.2.2.jar
│   │   │           ├── jackson-databind-2.2.2.jar
│   │   │           ├── log4j-1.2.16.jar
│   │   │           ├── logback-classic-1.1.3.jar
│   │   │           ├── logback-core-1.1.3.jar
│   │   │           ├── myapp.jar
│   │   │           ├── netty-all-4.0.20.Final.jar
│   │   │           ├── qbit-boon-0.8.2.jar
│   │   │           ├── qbit-core-0.8.2.jar
│   │   │           ├── qbit-vertx-0.8.2.jar
│   │   │           ├── slf4j-api-1.7.12.jar
│   │   │           ├── vertx-core-2.1.1.jar
│   │   │           └── vertx-platform-2.1.1.jar
│   │   ├── runContainer.sh
│   │   └── var
│   │       └── log
│   │           └── client
│   │               └── readme.md
│   ├── settings.gradle
│   └── src
│       └── main
│           └── java
│               └── com
│                   └── example
│                       └── ClientMain.java
└── server
    ├── build.gradle
    ├── conf
    │   ├── conf.properties
    │   └── logging.xml
    ├── gradle
    │   └── wrapper
    │       ├── gradle-wrapper.jar
    │       └── gradle-wrapper.properties
    ├── gradlew
    ├── gradlew.bat
    ├── image
    │   ├── Dockerfile
    │   ├── buildImage.sh
    │   ├── env.sh
    │   ├── etc
    │   │   └── server
    │   │       ├── conf.properties
    │   │       └── logging.xml
    │   ├── opt
    │   │   └── server
    │   │       ├── bin
    │   │       │   ├── run.sh
    │   │       │   ├── server
    │   │       │   └── server.bat
    │   │       └── lib
    │   │           ├── boon-json-0.5.5.jar
    │   │           ├── boon-reflekt-0.5.5.jar
    │   │           ├── jackson-annotations-2.2.2.jar
    │   │           ├── jackson-core-2.2.2.jar
    │   │           ├── jackson-databind-2.2.2.jar
    │   │           ├── log4j-1.2.16.jar
    │   │           ├── logback-classic-1.1.3.jar
    │   │           ├── logback-core-1.1.3.jar
    │   │           ├── netty-all-4.0.20.Final.jar
    │   │           ├── qbit-boon-0.8.2.jar
    │   │           ├── qbit-core-0.8.2.jar
    │   │           ├── qbit-vertx-0.8.2.jar
    │   │           ├── server.jar
    │   │           ├── slf4j-api-1.7.12.jar
    │   │           ├── vertx-core-2.1.1.jar
    │   │           └── vertx-platform-2.1.1.jar
    │   ├── runContainer.sh
    │   └── var
    │       └── log
    │           └── server
    │               └── readme.md
    ├── server.iml
    ├── settings.gradle
    └── src
        └── main
            └── java
                └── com
                    └── example
                        └── ServerMain.java
As you can see, we followed the first guide as a template very rigourously.
Kafka and Cassandra support, training for AWS EC2 Cassandra 3.0 Training