Rick

Rick
Rick

Wednesday, October 28, 2015

We have been busy over at QBit Java micorservices central

1) Support for non-JSON bodies from REST end-points

Added support for String and byte[] to be passed without JSON parsing.
IssueDocs Added to wiki main.
    @RequestMapping(value = "/body/bytes", method = RequestMethod.POST)
    public boolean bodyPostBytes( byte[] body) {
        String string = new String(body, StandardCharsets.UTF_8);
        return string.equals("foo");
    }

    @RequestMapping(value = "/body/string", method = RequestMethod.POST)
    public boolean bodyPostString(String body) {
        return body.equals("foo");
    }
If the Content-Type of the request is null or is application/json then we will parse the body as JSON. If the Content-Type is set and is not application/json then we will pass the raw String or raw bytes. This allows you to handle non-JSON content from REST. It does not do any auto-conversion. You will get the raw bytes or raw UTF-8 string.
    @Test
    public void testNoJSONParseWithBytes() {

        final HttpTextResponse httpResponse = httpServerSimulator.sendRequest(
                httpRequestBuilder.setUri("/es/body/bytes")
                        .setMethodPost().setContentType("foo")
                        .setBody("foo")
                        .build()
        );
        assertEquals(200, httpResponse.code());
        assertEquals("true", httpResponse.body());
    }


    @Test
    public void testNoJSONParseWithString() {

        final HttpTextResponse httpResponse = httpServerSimulator.sendRequest(
                httpRequestBuilder.setUri("/es/body/string")
                        .setMethodPost().setContentType("foo")
                        .setBody("foo")
                        .build()
        );

        assertEquals(200, httpResponse.code());
        assertEquals("true", httpResponse.body());

    }

2) Added HttpProxy support

You can proxy backend services from a single endpoint. This also allows you to do actions before sending the request, and to not forward the request based on a Predicate.

3) Added ability to return different response codes for success

By default QBit sends a 200 (OK) for a non-void call (a call that has a return or aCallback). If the REST operation has no return or no callback then QBit sends a 202 (Accepted). There may be times when you want to send a 201 (Created) or some other code that is not an Exception. You can do that by setting code on @RequestMapping. By default the code is -1 which means use the default behavior.
Issue Docs Added to wiki main.
  @RequestMapping(value = "/helloj7", code = 221)
    public void helloJSend7(Callback<JSendResponse<List<String>>> callback) {
        callback.returnThis(JSendResponseBuilder.jSendResponseBuilder(Lists.list(
                "hello " + System.currentTimeMillis())).build());
    }

4) Working with non JSON responses

Issue Docs Added to wiki main.
You do not have to return JSON form rest calls. You can return any binary or any text.

4.1) Returning non JSON from REST call

      @RequestMapping(method = RequestMethod.GET)
        public void ping2(Callback<HttpTextResponse> callback) {

            callback.returnThis(HttpResponseBuilder.httpResponseBuilder()
                    .setBody("hello mom").setContentType("mom")
                    .setCode(777)
                    .buildTextResponse());
        }

4.2) Returning binary from REST call

      @RequestMapping(method = RequestMethod.GET)
        public void ping2(Callback<HttpBinaryResponse> callback) {

            callback.returnThis(HttpResponseBuilder.httpResponseBuilder()
                    .setBody("hello mom").setContentType("mom")
                    .setCode(777)
                    .buildBinaryResponse());
        }

5) Create websocket service client that is ServiceDiscovery aware

ServiceDiscovery-aware websocket service client

        final Client client = clientBuilder
                .setServiceDiscovery(serviceDiscovery, "echo")
                .setUri("/echo").setProtocolBatchSize(20).build()
                 .startClient();

        final EchoAsync echoClient = client.createProxy(EchoAsync.class, "echo");
Currently the clientBuilder will load all service endpoints that are registered under the service name, and randomly pick one.
In the future we can RoundRobin calls or shard calls to websocket service and/or provide auto fail over if the connection is closed. We do this for the event bus that uses service discovery but it is not baked into WebSocket based client stubs yet.

For comparison here is a non-ServiceDiscovery version.

        final ClientBuilder clientBuilder = ClientBuilder.clientBuilder();
        final Client client = clientBuilder.setHost("localhost")
                .setPort(8080).setUri("/echo")
                .build().startClient();

        final EchoAsync echoClient = client.createProxy(EchoAsync.class, "echo");
Recall ServiceDiscovery includes Consul based, watching JSON files on disk, and DNS SVR records. It is easy to write your own service discovery as well and plug it into QBit.

6) JSend support for serialization and swagger

We started to add JSend support. The JSend is supported for marshaling JSend objects, and via our Swagger support.
@RequestMapping("/hw")
public class HelloWorldJSend {

    public static class Hello {
        final String hello;

        public Hello(String hello) {
            this.hello = hello;
        }
    }

    @RequestMapping("/hello")
    public String hello() {
        return "hello " + System.currentTimeMillis();
    }



    @RequestMapping("/helloj")
    public JSendResponse<String> helloJSend() {

        return JSendResponseBuilder.jSendResponseBuilder("hello " + System.currentTimeMillis()).build();
    }


    @RequestMapping("/helloj2")
    public JSendResponse<Hello> helloJSend2() {

        return JSendResponseBuilder.jSendResponseBuilder(new Hello("hello " + System.currentTimeMillis())).build();
    }


    @RequestMapping("/helloj3")
    public JSendResponse<List<String>> helloJSend3() {

        return JSendResponseBuilder.jSendResponseBuilder(Lists.list("hello " + System.currentTimeMillis())).build();
    }


    @RequestMapping("/helloj4")
    public JSendResponse<List<Hello>> helloJSend4() {

        return JSendResponseBuilder.jSendResponseBuilder(Lists.list(new Hello("hello " + System.currentTimeMillis()))).build();
    }


    @RequestMapping("/helloj5")
    public void helloJSend5(Callback<JSendResponse<List<Hello>>> callback) {

        callback.returnThis(JSendResponseBuilder.jSendResponseBuilder(Lists.list(new Hello("hello " + System.currentTimeMillis()))).build());
    }

    @RequestMapping("/helloj6")
    public void helloJSend6(Callback<JSendResponse<List<String>>> callback) {
        callback.returnThis(JSendResponseBuilder.jSendResponseBuilder(Lists.list(
                "hello " + System.currentTimeMillis())).build());
    }


    @RequestMapping(value = "/helloj7", code = 221)
    public void helloJSend7(Callback<JSendResponse<List<String>>> callback) {
        callback.returnThis(JSendResponseBuilder.jSendResponseBuilder(Lists.list(
                "hello " + System.currentTimeMillis())).build());
    }
Hitting the above
# String respnose
curl http://localhost:8080/hw/hello | jq .
"hello 1446088919561"

# JSend wrapping a a string
$ curl http://localhost:8080/hw/helloj | jq .
{
  "data": "hello 1446088988074",
  "status": "success"
}

#JSend wrapping a domain Object Hello
$ curl http://localhost:8080/hw/helloj2 | jq .
{
  "data": {
    "hello": "hello 1446089041902"
  },
  "status": "success"
}

#JSend wrapping a list of domain objects
$ curl http://localhost:8080/hw/helloj5 | jq .
{
  "data": [
    {
      "hello": "hello 1446089152089"
    }
  ],
  "status": "success"
}

Use jq to get pretty print JSON from QBit.
In this example we setup the admin interface as well so we can query swagger gen.

Starting up admin

    public static void main(final String... args) {
        final ManagedServiceBuilder managedServiceBuilder =
                ManagedServiceBuilder.managedServiceBuilder().setRootURI("/");

        /* Start the service. */
        managedServiceBuilder.addEndpointService(new HelloWorldJSend())
                .getEndpointServerBuilder()
                .build().startServer();

        /* Start the admin builder which exposes health end-points and meta data. */
        managedServiceBuilder.getAdminBuilder().build().startServer();

        System.out.println("Servers started");

        managedServiceBuilder.getSystemManager().waitForShutdown();


    }

Showing swagger support for JSend

$ curl http://localhost:7777/__admin/meta/ | jq .

{
  "swagger": "2.0",
  "info": {
    "title": "application title goes here",
    "description": "Description not set",
    "contact": {
      "name": "ContactName not set",
      "url": "Contact URL not set",
      "email": "no.contact.email@set.me.please.com"
    },
    "version": "0.1-NOT-SET",
    "license": {
      "name": "licenseName not set",
      "url": "http://www.license.url.com/not/set/"
    }
  },
  "host": "localhost:8080",
  "basePath": "/",
  "schemes": [
    "http",
    "https",
    "wss",
    "ws"
  ],
  "consumes": [
    "application/json"
  ],
  "definitions": {
    "jsend-array-String": {
      "properties": {
        "data": {
          "type": "string"
        },
        "status": {
          "type": "string",
          "description": "Status of return, this can be 'success', 'fail' or 'error'"
        }
      },
      "description": "jsend standard response"
    },
    "Hello": {
      "properties": {
        "hello": {
          "type": "string"
        }
      }
    },
    "jsend-Hello": {
      "properties": {
        "data": {
          "$ref": "#/definitions/Hello"
        },
        "status": {
          "type": "string",
          "description": "Status of return, this can be 'success', 'fail' or 'error'"
        }
      },
      "description": "jsend standard response"
    },
    "jsend-String": {
      "properties": {
        "data": {
          "type": "string"
        },
        "status": {
          "type": "string",
          "description": "Status of return, this can be 'success', 'fail' or 'error'"
        }
      },
      "description": "jsend standard response"
    },
    "jsend-array-Hello": {
      "properties": {
        "data": {
          "type": "array",
          "items": {
            "$ref": "#/definitions/Hello"
          }
        },
        "status": {
          "type": "string",
          "description": "Status of return, this can be 'success', 'fail' or 'error'"
        }
      },
      "description": "jsend standard response"
    }
  },
  "produces": [
    "application/json"
  ],
  "paths": {
    "/hw/helloj7": {
      "get": {
        "operationId": "helloJSend7",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "221": {
            "description": "no return description",
            "schema": {
              "$ref": "#/definitions/jsend-array-String"
            }
          }
        }
      }
    },
    "/hw/helloj6": {
      "get": {
        "operationId": "helloJSend6",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "no return description",
            "schema": {
              "$ref": "#/definitions/jsend-array-String"
            }
          }
        }
      }
    },
    "/hw/helloj5": {
      "get": {
        "operationId": "helloJSend5",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "no return description",
            "schema": {
              "$ref": "#/definitions/jsend-array-Hello"
            }
          }
        }
      }
    },
    "/hw/helloj4": {
      "get": {
        "operationId": "helloJSend4",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "no return description",
            "schema": {
              "$ref": "#/definitions/jsend-array-Hello"
            }
          }
        }
      }
    },
    "/hw/helloj3": {
      "get": {
        "operationId": "helloJSend3",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "no return description",
            "schema": {
              "$ref": "#/definitions/jsend-array-String"
            }
          }
        }
      }
    },
    "/hw/helloj2": {
      "get": {
        "operationId": "helloJSend2",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "no return description",
            "schema": {
              "$ref": "#/definitions/jsend-Hello"
            }
          }
        }
      }
    },
    "/hw/helloj": {
      "get": {
        "operationId": "helloJSend",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "no return description",
            "schema": {
              "$ref": "#/definitions/jsend-String"
            }
          }
        }
      }
    },
    "/hw/hello": {
      "get": {
        "operationId": "hello",
        "summary": "no summary",
        "description": "no description",
        "produces": [
          "application/json"
        ],
        "responses": {
          "200": {
            "description": "no return description",
            "schema": {
              "type": "string"
            }
          }
        }
      }
    }
  }
}
To learn more about swagger see swagger.
More work is needed to support JSend error and failures.

7) Add kv store / cache support part of core

        final KeyValueStoreService<Todo> todoKVStoreInternal = JsonKeyValueStoreServiceBuilder
                .jsonKeyValueStoreServiceBuilder()
                .setLowLevelKeyValueStoreService(keyValueStore)
                .buildKeyValueStore(Todo.class);

        todoKVStore.putWithConfirmation(callback, 
                "testPutWithConfirmationWrapped", new Todo(value));

8) Custom exception error codes

You can use HttpStatusCodeException to send custom HTTP error codes and to wrap exceptions.

Custom http error code exceptions

    @RequestMapping("/echo3")
    public String echoException() {
        throw new HttpStatusCodeException(700, "Ouch!");

    }

    @RequestMapping("/echo4")
    public void echoException2(final Callback<String> callback) {
        callback.onError(HttpStatusCodeException.httpError(900, "Ouch!!"));
    }

    @RequestMapping("/echo5")
    public void echoException3(final Callback<String> callback) {

        try {

            throw new IllegalStateException("Shoot!!");
        }catch (Exception ex) {
            callback.onError(HttpStatusCodeException.httpError(666, ex.getMessage(), ex));

        }
    }

10) Improved speed for HTTP Rest services that have only a few client

QBit was originally written for high-end, high-traffic, high-volume services. It needed to be tuned to work with just a few clients as well as high-end.

11) Bug fixes

There were quite a few potential issue that end up not being issue, but we wrote better test cases to prove (if only to ourselves) that these were not issues.

12) Created JMS (and Kafka support) support so JMS looks like regular QBit queue

Kafka and Cassandra support, training for AWS EC2 Cassandra 3.0 Training