Rise of the machines: Writing high speed services in Java part 1 - Mirco-services... another definition

(Draft.. I mostly wrote this with dictation software and am in the process of editing it.)

This is more of me thinking out loud...

If you are looking for a more polished article on this subject by me:

http://rick-hightower.blogspot.com/2015/01/high-speed-soa.html

http://rick-hightower.blogspot.com/2015/01/high-speed-soa.html


I have been silent as of late. Just busy is all. I have not fallen off of the face of the Earth (against the hopes of some I am sure). I have been among other things working on a new framework called QBit.

QBit is a very fast micro service framework. 
No.. wait.. It is not a framework. It is just a lib.
You can use it inside of a Servlet engine, with Spring, with Guice or standalone.
It is easy to use and non-evasive. 
By non evasive, I mean it does not scan you class-path and guess which libs you might need and then decorate your objects in ways you can't understand. No Magic. Magic sucks when things break.
QBit is about operational predictability. The advantage of async programming without hidden agendas, and techs popping up in your stack traces. 
It is not a container. No byte code manipulation. Just plain Java.
It is FAST, and easy and light weight. It fits the ethos of JSON and Pojos. 
No Magic. No Magic. No Magic.

QBit evolved quite a bit. It is based on ideas of mine (and others on the web influences from Akka, Spring Reactor, Spring Boot, LMAX, Mechanical sympathy blog, etc.) and based on a several closed sourced projects and frameworks that I worked on for large high-traffic sites and mobile app creators. I was privilege to recently work on a some high speed service projects. I learned a few things while trying to use Vertx at scale. I put those things in QBit. I was able to refine some of those learnings with QBit. (Some of the other stuff I wrote is still faster so QBit is a work in progress.) Anyway, I am starting to write more about it to get the word out. Publishing some benchmarks. Sharing some ideas. I don't think I will do a repeat of the JSON benchmarks. It caused some hard feelings. Which may or may not have been my intentions, but never the less, it did not feel right and is not something I feel like repeating. I learned to play nice. I relearn this every now and then. 

QBit microservice lib is fast. Will in win in every use case: no. Is it 100% done: no, but it is useable today.

QBit is 200M inproc messages per second (200M TPS) fast and a million JSON remote method calls a second fast. But more than fast, I think it is a compelling programming model. It is not new per se. It is a different direction than some but it fits the Java 8 Lambda world without letting Java 8 Lambdas throw up all over your code base. It is a POJO oriented (Object Oriented), async friendly, safe, FAST microservice lib.

Been thinking about writing this down for months and perhaps years. It's been a long time coming. I've had the privilege of working on some high speed services. I spent several years investigating how to build high-speed services and how to market products in this new emerging technical market. I ignored most of my research and went a different direction. 

Probably a few years thinking about how to create things in-memory and optimize thread hand offs. How to view system memory as the new disk. The disk as the new tape backup. Time was spent trying to internalize where I felt, and many others long before me, where the world of computers is going. The funny thing is I am not an early adopter. Although you wouldn't know it when I tell people things that are for the most part somewhat revolutionary in some circles. Whether what I say is revolutionary or reckless or obvious really depends on your perspective. It also probably really depends on the last project you worked on and where your area of expertise is. 

I am very opinionated. I try not to be. I fail. 

If you think Node.js is faster than Java, then you don't know Java. If you feel that Spring and Tomcat are the best way to write high speed services then you have fallen off the map in the mid 2000s. (Ok.. that may not make it in the final edit. Play nice and all.) Spring Boot is not bad. Don't get me wrong. I am not anti Spring.  My job is to finish QBit and show it value and maybe not in that order. More random thoughts....

I've been a Java developer for a long time. Before that I programed in C, C++ and assembly. I have no issues with programming in Go, Dart or Python or anything really. I do not define myself as a Java programmer, but merely a programmer who programs mostly in Java. I have no loyalty to the JVM other than it is a great platform to write services.  

I am most comfortable with Java through years of use and abuse. I even tried to use and did use Python and was a pretty big fan of Python. But perhaps too early of an adopter because at the time it was very hard to find jobs which is no longer the case, and I went back to Java hat in hand.

Mostly I am a Java developer. Whether by choice or by market demand or by where my skill is or where I have pigeonholed myself into. (I can see myself programming in Swift, Rust or Go, but Java is good for me too.) That said, I have seen what people use Java for evolve overtime. Way back when.... I used to use Java to program ActiveX components on a webpage which were scripted by JavaScript (or should I say I programmed J++ to do this). Way back when.... I also used Java to write Java Applets which used CORBA to talk to back in services. I spent a lot of time using Java to program servlets and JSPs. I worked at companies where JavaScript was banned. We were not allowed to use JavaScript due to the differences of the way JavaScript ran in different browsers and on different platforms. This led me to adopt such technologies as .Net, Tapestry and JSF. I worked at a few companies that had EJB containers. I read the J2EE blueprints and thought WTF.  I liked CORBA and DCOM. I even wrote a few books on Java and extreme programming (and Python). Later when JavaScript became more and more mature, I can see today technologies like JSF having a more limited use than I imagined, but my goals were to abstract out and not program in JavaScript. I was hopeful for things like Dart and before Dart, GWT (even dabbled in pyjamas). Despite my hatred of JavaScript, I have actually worked with it quite a bit. You will not find it on my resume. I will let you guess why. Client side, I am most interested in ReactJS and ReactJS Native.

Which brings us to, Java is still one of the most popular programming language. At times it is the most popular. At other times it is it's cousin C that it is the most popular. And this begs the question to me what are people using Java for today. 

Java is fast and it is good at handling threads. It has certain features like just in time compilation and an efficient garbage collection which makes it somewhat ideal for a large class of projects. If you have followed The Mechanical Sympathy blogs at all, you can see that people write very high-speed systems with Java. These systems are not your typical IT department sorts of applications. They are used in high-speed trading systems. It is these systems that have captured my imagination.

I actually had many mentors, and gurus who showed me some magic and this led me to go a different direction in my career and makes me very frustrated with scale out versus scale up and out. Without them, I would still be writing GUI front-ends to databases in JSF. I have been involved in high-speed Java computing fairly steady since 2009. (Getting more involved as my curiosity increased and my fear subsided.) They were inspirations but I went a different direction. My style has little to do with some of theirs. They are smarter. 

On the flip side you can see that there are many other frameworks from other programming languages that are popular. Language and framework diversity is the new norm. You can even see JavaScript is being used increasingly not just on the client-side but also now more and more on the server-side. Anyone who has been involved in web development for a long time you might recall that JavaScript was originally slated for both the server side and the client side. One would have to go back a ways in and would have to have experience with the Netscape application server. What is old is new again.  

If one follows the Tech Empower benchmarks like I do. Those benchmarks are like the Superbowl to me. One knows that Java is not alone in high-speed server-side development. One also knows that Java is no slacker in this space. One could even say that Java dominates this space.

This still begs the question what are people using Java for. We know Java is being used in mobile app development although not really called Java as such but that is what it is (and there is some ambiguity and strife there). We know that people write batch jobs with Java. We know that there are a lot of servlets and JSP web applications written in Java, especially in the non-public web, IT department space.  We know that Java gets used for back-in services.

Job demand for Spring Java has leveled. Demand for Java EE has decreased. Demand for Java/JSP has plummeted. But demand for Java overall has been somewhat constant so far but there is starting to be a decrease as we just have more choices. Perhaps Java has become synonymous with Spring, Hibernate, Tomcat, Java EE, etc. so much so that breaking them out into other keywords seems redundant. 

Anyway there seem to be some anomalies in the reporting on some job trend statistics in that as time goes on the graph seems to change retroactively. The bottom line I guess is Java does well on the performance benchmarks and we are moving to a more service oriented world like it or not. Any ability to write high-speed services is a needed skill and capability. Java seems to still be a very good platform for this. If not the best platform for this. The performance is good. The safety is high.

I always been a big fan of writing services. I've never been a big fan of what some people define as SOA. I think services are a foregone conclusion. I think SOA is murky in its definition. (I actually got paid handsomely to help write a SOA marketing strategy and tutorials for a very large IT / consulting company back in the early 2000s [largest?] as an independent. This was back in the days when I actually enjoyed talking to people. There is a long story here, but better left for never or at least a future blog post. Never kiss and tell.)

More recently a lot of people use the term micro services. One issue is that the definition of micro services is somewhat murky. It seems some try to define it as some always do (forehead slap). Yet the ones that are actually doing it, by this I mean the ones who have actually written frameworks that do it, don't seem to define it the same way. I guess I'm going to try over time to come up with my own definition and see through the democracy of the bloggers sphere which definition my definition fits most with in the world of public ideas. This is not to say that I am against SOA or micro services. I am just against the way some people define SOA.


I am glad that coined the phrase microservice. This is the direction we need to go. Words have power and it is time the approach is given a name. A banner. A logo.

I wrote a framework called QBit (with help) and I define it as a micro services platform. Since my definition is not the same as some "experts" who have not written frameworks and even not the same definition of other frameworks exactly, it behoves me to define what I mean by micro services. I guess.

In the end whether you call it micro services, services or SOA there seems to be a trend to build such in Java. My background is firmly in the Spring camp or I guess I should say around the Spring camp and Java EE since I was never fully welcomed in any camp (I am prickly that way which could be another blog post or never. I am fiercely independent to a fault.). I prefer the standards model of JSRs which begot technologies like JCache, JSP, Servlets, EL, etc. (I even worked on a few including code contributions.) However Spring fixed many issues which plagued technologies derived from standards driven by vendors. Not to bring up old history because it's old and I'm bored with it. Spring is very much the de facto standard in the Java world for many. Spring at some point jumped the shark but there is still good stuff there. Lots of good stuff.

More random thoughts, Since I moved to the San Francisco Bay Area in 2009, I have been paid to program in C (embedded medical devices), Java (back end services mostly), Python (back-end) and Dart. I get paid for an ability to figure shit out and not because I am a Java  programmer. Java is a language not a religion. Spring is not a religion. Josh Bloch is not a priest by the way. Quit quoting him like he is the pope. I even dabbled in Go and decided it was at the time not mature enough (at the time, but this was early on). The one thing that is constant is change. I don't know if Java will hold its lead forever.  There are signs of strong competition. I have high hopes for Go. I don't see myself doing much client development so I don't see Swift/Objective C or Android/Java in my future. To be honest, I am surprised Go does not do better than it does. In my future I see C and Go and Java. 

That brings me to my current thoughts on service development in the JVM. We created a framework called QBit (mostly me but I was certainly not alone). QBit allows for efficient queue to queue handoff using principles of mechanical sympathy but not using disruptor. Although I do not rule out using the disruptor in the future. I find it unwieldy for a general purpose services framework. (It is a great tool, but QBit versus disruptor would be like comparing a screwdriver versus a hammer or a Porsche versus a mini-van, it depends if you have a nail or a screw, small genitalia or a family.)

QBit benchmarks will only include QBit. I will let the numbers speak for themselves. It is fast enough where I think if another framework is faster, it will not be by much and QBIt should be faster than most. QBit is more about SOA, Services, REST, JSON, websocket and queue to queue hand off than just raw speed alone although I believe it does really well at raw speed. The tradeoffs if you want to write in-memory high-speed services then you need something like QBit. Also QBit can work with other frameworks so it does not have to be a direct competitor per se. Anyway, that is my plan. Let's see if I can keep my pride in check, and share some ideas and get some useful dialog, feedback and new ideas. 

QBit is very fast. 100M ping pong message thread to thread fast (and even faster) (now more like 200M). 500K remote RPC calls per second fast. It sips CPU and pounds messages through queues. The interface is very much like Spring MVC. QBit uses nearly identical annotations. There is a JSON marshaling layer and a layer that does method call batching. You can work with proxies and stubs. I have used QBit and frameworks like QBit to work with teams to write back end services that power some of the most busy service back ends out there. QBit can take a pounding. QBit’s ancestors have been used as back ends for services whose load caves in an F5. It handles load like no one business. Under load it gets faster. 

QBit is useful today. We plan on adding things that make QBit even better. 

QBit was part of Boon but we broke it out for a few reasons. Boon is big. Bigger than it should be already. Boon has areas that are really well tested and solid and areas which are much more experimental. Boon was written originally as a Python like (batteries included) framework. Boon was my second attempt at this. I got much further with Boon than the first attempt. Boon has many back alleys and side trips.

Also with Boon I was challenging the status quo. I was challenging common Java practices and folk lore. It was the anti-Josh Bloch, lets-just-code, Red Bull induced code rage. It was productive. I created (with some help) a JSON parser that is 4x faster than the mainstream ones for REST style use case. QBit is not Boon. QBit is damn Josh Bloch was right, I wish Boon had more builders, more final properties and higher quality. So QBit is me firmly subscribing to Josh Bloch et al. Boon is still awesome, but when I rewrite it, I will drink deeply of the Josh Bloch kool-aid, not because I followed him blindly but because I rebelled and then decided he was right after all.

QBit is pluggable in that you could use Jackson JSON with QBit. QBit by default uses Vertx for its networking libs but you could use QBit inside of Tomcat or with Netty or for that matter with Jetty. I don't plan on writing Tomcat or Jetty plugins for QBit but I am leaving the door open so it can happen. QBit by default uses Boon and Vertx. QBit has more code coverage than Boon. It has a smaller scope too. You could easily use QBit in a groovy project or in a Spring project. It tries not to be overly opinionated. Boon is inherently opinionated. QBit uses Boon, don't get me wrong. Boon is not something I am walking away from.

QBit like Boon focuses on speed. I believe that already QBit rivals some of the fastest frameworks and should often exceed them. But QBit is really about putting this speed in an easy to use package.

QBit is generally higher quality than Boon. QBit is tied to Java 8. Boon is tied to Java 7. Boon uses maven. QBit uses gradle. QBit relies on Boon and the goal is to break Boon up into smaller libs with similar restricted focus of QBit and the higher quality of QBit.

Did you make it this far? AWESOME! Now back to the subject of microservices.

With that let me define what micro services means to QBit:

• Can run standalone 
• Runs as a Java main method
• Does not need a container
• No magic class loader (could be optional, not the default)
• Runs easily in an IDE for debugging
• Runs from a main method
• Requires no byte code manipulation
• Requires no magic at all
• Requires no class file scanning (you can but it is not required)
• Services are composable via builders and no config files are needed (you can have but not required, spring / guice is NEVER required)
• Does not require retraining your entire team 
• Looks like stuff they are already doing but with some new stuff mixed in
• JSON based remote procedure calls
• Supports REST
• Supports Websocket
• Simple
• Easy to understand
• Supports in-memory service development
• Efficiently utilizes multi-core CPUs
• Micro services are building blocks
• Micro services are self contained and resilient
• Micro services own their data
• Micro services are masters not slaves
• Micro services do not center around databases and caches

QBit does the above today. 

QBit does not compete with Spring in total. You can use QBit in a Spring project. QBit does overlap with Spring MVC REST but is a high-speed subset of Spring MVC functionality.

I have worked on projects where we wrote services that run on tens of servers better than other services that run on 100s and at times 1,000s of servers. That knowledge is put into QBit. QBit works not just in theory but in practice. I may not be the person to push QBit. I sure could use some help.

Micro services do not cover the following:

• Clustering
• Discovery
• Coordination
• Map Reduce

These topics are important but are outside the scope of micro services. Micro services typically do their own sharding. Micro services use high speed messaging to other micro services and REST / websocket to the rest of the world.

QBit does not compete with Kafka. I get this a lot. QBit is a workshop. Kafka is a nail. QBit has a hammer where you could pound the shit out of that Kafka nail. Kafka is not just a nail. Kafka is an awesome nail. You can't build an app with Kafka. 

QBit is not the final word on this topic. There will be others.

Everything is a queue. You have a choice. You can embrace it and control it. You can optimize for it. Or you can hide behind abstractions. QBit opens you up to peeking into what is going on, and allows you to pull some levers without selling your soul.

QBit is a library not a framework. You can mix and match QBit with Spring, Guice, etc.

QBit is FAST!

Java Microservice Lib

QBit has inproc services, REST microservices and WebSocket microservices as well as an in-proc service event bus (which can be per module or per app). It supports workers and in-memory services.

Before we descibe more, here are two sample services.

Todo Service

@RequestMapping("/todo-service")
public class TodoService {

    @RequestMapping("/todo/count")
    public int size() {...

    @RequestMapping("/todo/")
    public List<TodoItem> list() {...

Adder Service using URI params

    @RequestMapping("/adder-service")
    public class AdderService {

        @RequestMapping("/add/{0}/{1}")
        public int add(@PathVariable int a, @PathVariable int b) {...
    }

Status

Lot's of progress. More people are helping out. QBit now works with Vertx (standalone or embedded), Jetty (standalone) or just plain Java Servlets.

License

Apache 2

QBit philosiphy:

At the end of the day QBit is a simple library not a framework. Your app is not a QBit app but a Java app that uses the QBit lib. QBit allows you to work with Java UTIL concurrent, and does not endeavor to hide it from you. Just trying to take the sting out of it.

Does it work

We have used techniques in Boon and QBit with great success in high-end, high-performance, high-scalable apps. We helped clients handle 10x the load with 1/10th the servers of their competitors using techniques in QBit. QBit is us being sick of hand tuning queue access and threads.

Boon and QBit humility policy

Ideas for Boon and QBit often come from all over the web. We make mistakes. Point them out. As a developer of Boon and QBit, we are fellow travelers. If you have an idea or technique you want to share, we listen.

Inspiration

A big inspireation for Boon/QBit was Akka, Go Channels, Active Objects, Apartment Model Threading, Actor, and the Mechnical Sympathy papers.

"I have read the AKKA in Action Book. It was inpsiring, but not the only inspiration for QBit.". "I have written apps where I promised a lot of performance and the techniques from QBit is how I got it."

• Rick Hightower

QBit has ideas that are similar to many frameworks. We are all reading the same papers. QBit got inspiration from the LMAX disruptor papers and this blog post about link transfer queue versus disruptor. We had some theories about queues that blog post insprired us to try them out. Some of these theories are deployed at some of the biggest middleware backends and whose name brands are known around the world. And thus QBit was born. QBit also took an lot of inspiration by the great work done by Tim Fox on Vertx. The first project using something that could actually be called QBit (albiet early QBit) was using Vertx on an web/mobile microserivce for an app that could potentially have 80 million users. It was this experience with Vertx and early QBit that led to QBit development and evolution. QBit is built on the shoulders of giants.

Does QBit compete with...

Spring Disruptor: No. You could use QBit to write plugins for Spring Disruptor I suppose, but QBit does not compete with Spring Disruptor. Spring Boot/Spring MVC: No. We use the same annotations but QBit is geared for high-speed in-memory microservices. It is more like Akka than Spring Boot. QBit has a subset of the features of Spring MVC geared only for microservices, i.e., WebSocket RPC, REST, JSON marshaling, etc. Akka: No. Well Maybe. Akka has similar concepts but they take a different approach. QBit is more focused on Java, and microservices (REST, JSON, WebSocket) than Akka. LMAX Disruptor: No. In fact, we can use disruptor as on of the queues that QBit uses underneath the covers.

(Early benchmarks have been removed. They were here. QBit got a lot faster. Links and reports will be created.)

Code Examples

Basic Queue example (REST style services is further down)

     BasicQueue<Integer> queue =  BasicQueue.create(Integer.class, 1000);

    //Sending threads

     SendQueue<Integer> sendQueue = queue.sendQueue();
     for (int index = 0; index < amount; index++) {
           sendQueue.send(index);
     }
     sendQueue.flushSends();
     ...
     sendQueue.sendAndFlush(code);
     //other methods for sendQueue, writeBatch, writeMany


     //Recieving Threads
     ReceiveQueue<Integer> receiveQueue = queue.receiveQueue();
     Integer item = receiveQueue.take(); 
     //other methods poll(), pollWait(), readBatch(), readBatch(count)

What is QBit again?

QBit is a queuing library for microservices. It is similar to many other projects like Akka, Spring Reactor, etc. QBit is just a library not a platform. QBit has libraries to put a service behind a queue. You can use QBit queues directly or you can create a service. QBit services can be exposed by WebSocket, HTTP, HTTP pipeline, and other types of remoting. A service in QBit is a Java class whose methods are executed behind service queues. QBit implements apartment model threading and is similar to the Actor model or a better description would be Active Objects. QBit does not use a disruptor. It uses regular Java Queues. QBit can do north of 100 million ping pong calls per second which is an amazing speed (seen as high as 200M). QBit also supports calling services via REST, and WebSocket. QBit is microservices in the pure Web sense: JSON, HTTP, WebSocket, etc. QBit uses micro batching to push messages through the pipe (queue, IO, etc.) faster to reduce thread hand-off.

QBit lingo

QBit is a Java microservice lib supporting REST, JSON and WebSocket. It is written in Java but we could one day write a version in Rust or Go or C# (but that would require a large payday).

Service POJO (plain old Java object) behind a queue that can receive method calls via proxy calls or events (May have one thread managing events, method calls, and responses or two one for method calls and events and the other for responses so response handlers do not block service. One is faster unless responses block). Services can use Spring MVC style REST annotations to expose themselves to the outside world via REST and WebSocket.

ServiceBundle Many POJOs behind one response queue and many receive queues. There may be one thread for all responses or not. They also can be one receive queue.

Queue A thread managing a queue. It supports batching. It has events for empty, reachedLimit, startedBatch, idle. You can listen to these events from services that sit behind a queue. You don't have to use Services. You can use Queue's direct. In QBit, you have sender queues and recievers queues. They are seperated to support micro-batching.

ServiceServer ServiceBundle that is exposed to REST and WebSocket communication.

EventBus EventBus is a way to send a lot of messages to services that may be loosely coupled.

ClientProxy ClientProxy is a way to invoke service through async interface, service can be inproc (same process) or remoted over WebSocket.

Non-blocking QBit is a non-blocking lib. You use CallBacks via Java 8 Lambdas. You can also send event messages and get replies. Messaging is built into the system so you can easily coordinate complex tasks. QBit takes an object-oriented approach to service development so services look like normal Java services that you already write, but the services live behind a queue/thread. This is not a new concept. Micorsoft did this with DCOM/COM and called it active objects. Akka does it with actors and called them strongly typed Actors. The important concepts is that you get the speed of reactive and actor style messaging but you develop in a natural OOP approach. QBit is not the first. QBit is not the only.

Speed QBit is VERY fast. There is a of course a lot of room for improvement. But already 200M+ TPS inproc ping pong, 10M-20M+ TPS event bus, 500K TPS RPC calls over WebSocket/JSON, etc. More work needs to be done to improve speed, but now it is fast enough where we are focusing more on usability. The JSON support uses Boon by default which is up to 4x faster than other JSON parsers for the REST/JSON, WebSocket/JSON use case.

CURLable REST services example

Talk is cheap. Let's look at some code. You can get a detailed walk through in the Wiki. We have a lot of documentation already.

We will create a service that is exposed through REST/JSON.

To query the size of the todo list:

curl localhost:8080/services/todo-service/todo/count

To add a new TODO item.

curl -X POST -H "Content-Type: application/json" -d \
'{"name":"xyz","description":"xyz"}' \
http://localhost:8080/services/todo-service/todo 

To get a list of TODO items

curl http://localhost:8080/services/todo-service/todo/

The TODO example will use and track Todo items.

Todo item POJO sans getter

package io.advantageous.qbit.examples;

import java.util.Date;


public class TodoItem {


    private final String description;
    private final String name;
    private final Date due;

The TodoService uses Spring MVC style annotations.

Todo Service

@RequestMapping("/todo-service")
public class TodoService {


    private List<TodoItem> todoItemList = new ArrayList<>();


    @RequestMapping("/todo/count")
    public int size() {

        return todoItemList.size();
    }

    @RequestMapping("/todo/")
    public List<TodoItem> list() {

        return todoItemList;
    }

    @RequestMapping(value = "/todo", method = RequestMethod.POST)
    public void add(TodoItem item) {

        todoItemList.add(item);
    }

}

Side note Why Spring style annotations?

Why did we pick Spring sytle annotations? 1) Spring is not a standard and neither is QBit. 2) We found the Spring annotations to be less verbose. 3) More people use Spring than Java EE. We wrote QBit for people to use. We could easily support JAX-RS style annotaitons, and we probably will. Since QBit focuses on JSON, we do not need all of the complexity of JAX-RS or even all the features of the Spring MVC annotations. Also we can literally use the actual Spring annotations. QBit and Boon use a non-type safe mechanism for annotations which means they are not tied to a particular lib. You can define your own. I hate vendor tie-in even if it is an open source vendor.

Now just start it up.

    public static void main(String... args) {
        ServiceServer server = new ServiceServerBuilder().build();
        server.initServices(new TodoService());
        server.start();
    }

That is it. There is also out of the box WebSocket support with client side proxy generation so you can call into services at the rate of millions of calls per second.

Using URI Params for QBit microservice

    @RequestMapping("/adder-service")
    public class AdderService {


        @RequestMapping("/add/{0}/{1}")
        public int add(@PathVariable int a, @PathVariable int b) {

            return a + b;
        }
    }

WebSocket

You can always invoke QBit services via a WebSocket proxy. The advantage of a WebSocket proxy is it allows you execute 1M RPC+ a second (1 million remote calls every second).

Using a microservice remotely with WebSocket

       /* Start QBit client for WebSocket calls. */
        final Client client = clientBuilder()
                   .setPort(7000).setRequestBatchSize(1).build();


       /* Create a proxy to the service. */
        final AdderServiceClientInterface adderService =
                client.createProxy(AdderServiceClientInterface.class, 
                "adder-service");

        client.start();



       /* Call the service */
        adderService.add(System.out::println, 1, 2);

The output is 3.

3

The above uses a WebSocket proxy interface to call the service async.

    interface AdderServiceClientInterface {

        void add(Callback<Integer> callback, int a, int b);
    }

REST call with URI params

The last client example uses WebSocket. You could also just use REST, and actually use the URI params that we setup. REST is nice but it is going to be slower than WebSocket support.

QBit ships with a nice little HTTP client. We can use it.

You can use it to send async calls and websocket messages with the HTTP client.

Here we will use the http client to invoke our remote method:

Using a microservice remotely with REST QBit microservice client

        HttpClient httpClient = httpClientBuilder()
                .setHost("localhost")
                .setPort(7000).build();

        httpClient.start();
        String results = httpClient
                   .get("/services/adder-service/add/2/2").body();
        System.out.println(results);

The output is 4.

4

Accessing The URI Param example with CURL

You can also access the service from curl.

$ curl http://localhost:7000/services/adder-service/add/2/2

See this full example here: QBit microservice getting started tutorial.

QBit URI params and WebSocket proxy client

Working with WebSocket, HttpClient etc.

QBit has a library for working with and writing async microservices that is lightweight and fun to use.

WebSocket server and client.

Create an HTTP server

        /* Create an HTTP server. */
        HttpServer httpServer = httpServerBuilder()
                .setPort(8080).build();

Setup server WebSocket support

        /* Setup WebSocket Server support. */
        httpServer.setWebSocketOnOpenConsumer(webSocket -> {
            webSocket.setTextMessageConsumer(message -> {
                webSocket.sendText("ECHO " + message);
            });
        });

Start the server

        /* Start the server. */
        httpServer.start();

Setup the WebSocket client

        /** CLIENT. */

        /* Setup an httpClient. */
        HttpClient httpClient = httpClientBuilder()
                .setHost("localhost").setPort(8080).build();
        httpClient.start();

Client WebSocket

        /* Setup the client websocket. */
        WebSocket webSocket = httpClient
                .createWebSocket("/websocket/rocket");

        /* Setup the text consumer. */
        webSocket.setTextMessageConsumer(message -> {
            System.out.println(message);
        });
        webSocket.openAndWait();

        /* Send some messages. */
        webSocket.sendText("Hi mom");
        webSocket.sendText("Hello World!");

Output

ECHO Hi mom
ECHO Hello World!

Now stop the server and client. Pretty easy eh?

High-speed HTTP client and server done microservice style

Starting up an HTTP server

        /* Create an HTTP server. */
        HttpServer httpServer = httpServerBuilder()
                .setPort(8080).build();

        /* Setting up a request Consumer with Java 8 Lambda expression. */
        httpServer.setHttpRequestConsumer(httpRequest -> {

            Map<String, Object> results = new HashMap<>();
            results.put("method", httpRequest.getMethod());
            results.put("uri", httpRequest.getUri());
            results.put("body", httpRequest.getBodyAsString());
            results.put("headers", httpRequest.getHeaders());
            results.put("params", httpRequest.getParams());
            httpRequest.getReceiver()
                .response(200, "application/json", Boon.toJson(results));
        });


        /* Start the server. */
        httpServer.start();

The focus is on ease of use and using Java 8 Lambdas for callbacks so the code is tight and small.

Find out more about QBit's microservice style websocket support here

Using HTTP Client lib

Now, let's try out our HTTP client.

Starting up an HTTP client

        /* Setup an httpClient. */
        HttpClient httpClient = httpClientBuilder()
                  .setHost("localhost").setPort(8080).build();
        httpClient.start();

You just pass the URL, the port and then call start.

Synchronous HTTP calls

Now you can start sending HTTP requests.

No Param HTTP GET

        /* Send no param get. */
        HttpResponse httpResponse = httpClient.get( "/hello/mom" );
        puts( httpResponse );

An HTTP response just contains the results from the server.

No Param HTTP Response

public interface HttpResponse {

    MultiMap<String, String> headers();

    int code();

    String contentType();

    String body();

}

There are helper methods for sync HTTP GET calls.

Helper methods for GET

        /* Send one param get. */
        httpResponse = httpClient.getWith1Param("/hello/singleParam", 
                                        "hi", "mom");
        puts("single param", httpResponse );


        /* Send two param get. */
        httpResponse = httpClient.getWith2Params("/hello/twoParams",
                "hi", "mom", "hello", "dad");
        puts("two params", httpResponse );

...

        /* Send five param get. */
        httpResponse = httpClient.getWith5Params("/hello/5params",
                "hi", "mom",
                "hello", "dad",
                "greetings", "kids",
                "yo", "pets",
                "hola", "neighbors");
        puts("5 params", httpResponse );

The puts method is a helper method it does System.out.println more or less by the way.

The first five params are covered. Beyond five, you have to use the HttpBuilder.

        /* Send six params with get. */

        final HttpRequest httpRequest = httpRequestBuilder()
                .addParam("hi", "mom")
                .addParam("hello", "dad")
                .addParam("greetings", "kids")
                .addParam("yo", "pets")
                .addParam("hola", "pets")
                .addParam("salutations", "all").build();

        httpResponse = httpClient.sendRequestAndWait(httpRequest);
        puts("6 params", httpResponse );

Http Async HTTP Client

There are async calls for GET as well.

Async calls for HTTP GET using Java 8 lambda

        /* Using Async support with lambda. */
        httpClient.getAsync("/hi/async", (code, contentType, body) -> {
            puts("Async text with lambda", body);
        });

        Sys.sleep(100);


        /* Using Async support with lambda. */
        httpClient.getAsyncWith1Param("/hi/async", "hi", "mom", (code, contentType, body) -> {
            puts("Async text with lambda 1 param\n", body);
        });

        Sys.sleep(100);



        /* Using Async support with lambda. */
        httpClient.getAsyncWith2Params("/hi/async",
                "p1", "v1",
                "p2", "v2",
                (code, contentType, body) -> {
                    puts("Async text with lambda 2 params\n", body);
                });

        Sys.sleep(100);


...
        /* Using Async support with lambda. */
        httpClient.getAsyncWith5Params("/hi/async",
                "p1", "v1",
                "p2", "v2",
                "p3", "v3",
                "p4", "v4",
                "p5", "v5",
                (code, contentType, body) -> {
                    puts("Async text with lambda 5 params\n", body);
                });

        Sys.sleep(100);

Find more about the easy to use, fast microservice HTTP client here.

InProc QBit services

QBit allows for services behind queues to be run in-proc as well.

        /* POJO service. */
        final TodoManager todoManagerImpl = new TodoManager();

        /*
        Create the service which manages async calls to todoManagerImpl.
         */
        final Service service = serviceBuilder()
                .setServiceObject(todoManagerImpl)
                .build().start();


        /* Create Asynchronous proxy over Synchronous service. */
        final TodoManagerClientInterface todoManager = 
              service.createProxy(TodoManagerClientInterface.class);

        service.startCallBackHandler();


        System.out.println("This is an async call");
        /* Asynchronous method call. */
        todoManager.add(new Todo("Call Mom", "Give Mom a call"));


        AtomicInteger countTracker = new AtomicInteger(); 
        //Hold count from async call to service... for testing and showing it is an async callback

        System.out.println("This is an async call to count");

        todoManager.count(count -> {
            System.out.println("This lambda expression is the callback " + count);

            countTracker.set(count);
        });


        todoManager.clientProxyFlush(); //Flush all methods. It batches calls.

        Sys.sleep(100);

        System.out.printf("This is the count back from the server %d\n", countTracker.get());

Detailed tutorial on in-proc services is being written.

QBit Event Bus

QBit Event Bus more detailed example

QBit also has a service event bus. This example is a an employee benefits services example.

We have two channels.

public static final String NEW_HIRE_CHANNEL = "com.mycompnay.employee.new";

public static final String PAYROLL_ADJUSTMENT_CHANNEL = "com.mycompnay.employee.payroll";

An employee object looks like this:

public static class Employee {
       final String firstName;
       final int employeeId;

This example has three services: EmployeeHiringService, BenefitsService, and PayrollService.

These services are inproc services. QBit supports WebSocket, HTTP and REST remote services as well, but for now, let's focus on inproc services. If you understand inproc then you will understand remote.

The EmployeeHiringService actually fires off the events to other two services.

public class EmployeeHiringService {


    public void hireEmployee(final Employee employee) {

           int salary = 100;
           System.out.printf("Hired employee %s\n", employee);

           //Does stuff to hire employee

           //Sends events
           final EventManager eventManager = 
                               serviceContext().eventManager();
           eventManager.send(NEW_HIRE_CHANNEL, employee);

           eventManager.sendArray(PAYROLL_ADJUSTMENT_CHANNEL, 
                                     employee, salary);


    }

   }

Notice that we call sendArray so we can send the employee and their salary. The listener for PAYROLL_ADJUSTMENT_CHANNEL will have to handle both an employee and an int that represents the new employees salary. You can also use event bus proxies so you do not have to call into the event bus at all.

The BenefitsService listens for new employees being hired so it can enroll them into the benefits system.

public static class BenefitsService {

       @OnEvent(NEW_HIRE_CHANNEL)
       public void enroll(final Employee employee) {

           System.out.printf("Employee enrolled into benefits system employee %s %d\n",
                   employee.getFirstName(), employee.getEmployeeId());

       }

Daddy needs to get paid.

    public static class PayrollService {

        @OnEvent(PAYROLL_ADJUSTMENT_CHANNEL)
        public void addEmployeeToPayroll(final Employee employee, int salary) {

            System.out.printf("Employee added to payroll  %s %d %d\n",
                    employee.getFirstName(), employee.getEmployeeId(), salary);

        }

    }

The employee is the employee object from the EmployeeHiringService.

so you can get your benefits, and paid!

Find more details here:

QBit Event Bus more detailed example

Private event bus and event bus proxies

TBD

Workers - pools and shards

public class ServiceWorkers {

    public static RoundRobinServiceDispatcher workers() {...

    public static ShardedMethodDispatcher shardedWorkers(final ShardRule shardRule) {...

You can compose sharded workers (for in-memory, thread safe, CPU intensive services), or workers for IO or talking to foreign services or foreign buses.

Here is an example that uses a worker pool with three service workers in it:

Let's say you have a service that does something:

    //Your POJO
    public  class MultiWorker {

        void doSomeWork(...) {
           ...
        }

    }

Now this does some sort of IO and you want to have a bank of these running not just one so you can do IO in parallel. After some performance testing, you found out that three is the magic number.

You want to use your API for accessin this service:

    public  interface MultiWorkerClient {
        void doSomeWork(...);
    }

Now let's create a bank of these and use it.

First create the QBit services which add the thread/queue/microbatch.

        /* Create a service builder. */
        final ServiceBuilder serviceBuilder = serviceBuilder();

        /* Create some qbit services. */
        final Service service1 = serviceBuilder.setServiceObject(new MultiWorker()).build();
        final Service service2 = serviceBuilder.setServiceObject(new MultiWorker()).build();
        final Service service3 = serviceBuilder.setServiceObject(new MultiWorker()).build();

Now add them to a ServiceWorkers object.

        ServiceWorkers dispatcher;
        dispatcher = workers(); //Create a round robin service dispatcher
        dispatcher.addServices(service1, service2, service3);
        dispatcher.start(); // start up the workers

You can add services, POJOs and method consumers, method dispatchers to a service bundle. The service bundle is an integration point into QBit.

Let's add our new Service workers. ServiceWorkers is a ServiceMethodDispatcher.

        /* Add the dispatcher to a service bundle. */
        bundle = serviceBundleBuilder().setAddress("/root").build();
        bundle.addServiceConsumer("/workers", dispatcher);
        bundle.start();

We are probably going to add a helper method to the service bundle so most of this can happen in a single call.

Now you can start using your workers.

        /* Start using the workers. */
        final MultiWorkerClient worker = bundle.createLocalProxy(MultiWorkerClient.class, "/workers");

Now you could use Spring or Guice to configure the builders and the service bundle. But you can just do it like the above which is good for testing and understanding QBit internals.

QBit also supports the concept of sharded services which is good for sharding resoruces like CPU (run a rules engine on each CPU core for a user recommendation enigne).

QBit does not know how to shard your services, you have to give it a hint. You do this through a shard rule.

public interface ShardRule {
    int shard(String methodName, Object[] args, int numWorkers);
}

We worked on an app where the first argument to the services was the username, and then we used that to shard calls to a CPU intensive in-memory rules engine. This technique works. :)

The ServiceWorkers class has a method for creating a sharded worker pool.

    public static ShardedMethodDispatcher shardedWorkers(final ShardRule shardRule) {
        ...
    }

To use you just pass a shard key when you create the service workers.

        dispatcher = shardedWorkers((methodName, methodArgs, numWorkers) -> {
            String userName = methodArgs[0].toString();
            int shardKey =  userName.hashCode() % numWorkers;
            return shardKey;
        });

Then add your services to the ServiceWorkers composition.

        int workerCount = Runtime.getRuntime().availableProcessors();

        for (int index = 0; index < workerCount; index++) {
            final Service service = serviceBuilder
                    .setServiceObject(new ContentRulesEngine()).build();
            dispatcher.addServices(service);

        }

Then add it to the service bundle as before.

        dispatcher.start();

        bundle = serviceBundleBuilder().setAddress("/root").build();

        bundle.addServiceConsumer("/workers", dispatcher);
        bundle.start();

Then just use it:

        final MultiWorkerClient worker = bundle.createLocalProxy(MultiWorkerClient.class, "/workers");

        for (int index = 0; index < 100; index++) {
            String userName = "rickhigh" + index;
            worker.pickSuggestions(userName);
        }

Built in shard rules

public class ServiceWorkers {
...
    public static ShardedMethodDispatcher shardOnFirstArgumentWorkers() {
       ...
    }

...

    public static ShardedMethodDispatcher shardOnFifthArgumentWorkers() {
         ...
    }


    public static ShardedMethodDispatcher shardOnBeanPath(final String beanPath) {
        ...
    }

The shardOnBeanPath allows you to create a complex bean path navigation call and use its property to shard on.

     /* shard on 2nd arg which is an employee
       Use the employees department's id property. */
     dispatcher = shardOnBeanPath("[1].department.id"); 

     /* Same as above. */
     dispatcher = shardOnBeanPath("1/department/id"); 

Read more about Service sharding and service workers here

To learn more about QBit. Try these links:

[Detailed Tutorial] QBit microservice example

[Doc] Queue Callbacks for QBit queue based services

[Quick Start] Building a simple Rest web microservice server with QBit

[Quick Start] Building a TODO web microservice client with QBit

[Quick Start] Building a TODO web microservice server with QBit

[Quick Start] Building boon for the QBit microservice engine

[Quick Start] Building QBit the microservice lib for Java

[Rough Cut] Delivering up Single Page Applications from QBit Java JSON Microservice lib

[Rough Cut] Working with event bus for QBit the microservice engine

[Rough Cut] Working with inproc MicroServices

[Rough Cut] Working with private event bus for inproc microservices

[Rough Cut] Working with strongly typed event bus proxies for QBit Java Microservice lib

[Rough Cut] Working with System Manager for QBit Mircoservice lib

[Z Notebook] More benchmarking internal

[Z Notebook] Performance testing for REST

[Z Notebook] Roadmap

Home

Introduction to QBit

Local Service Proxies

QBit Boon New Wave of JSON HTTP and Websocket

QBit Docs