Java Microservices Architecture

The term "Microservices Architecture" is now a popular term. Unlike many trends this one seems to have some momentum and is more about how people are actually developing services versus vendors commandeering and needlessly complicating something simple which is what SOA has become to a large extent. Services are good. Web Services are good. SOA now has a bad reputations and association with overly complicated systems.

There is no concrete definition of Microservice Architecture but there are certain aspects that are thematic in its application. Just like racism, it is hard to define, but you know when you see it.

The focus on Microservices is a focus on business capability, and a refocus on object oriented programming roots and organizing code around business domains with data and business rules co-located in the same process or set of processes.

To adopt the Microservices style is to recognize that the world is polyglot programming, and the easiest integration is around JSON, HTTP and other web standards like WebSocket. Microservices embrace smart endpoints and dumb pipes.

Microservices is not SOA. In fact, it in many ways it is directly the opposite. For example SOA often embraces WSDL which is a very strongly typed and rigid way to define a service endpoint. WSDL and XML schema takes all of the X out of XML.

A Microservices Architecture embraces JSON and tolerant readers with the idea that the interface may change a bit and it is nicer to allow some flexibility in what properties/fields make up a message.

A smart end point and a dumb pipe would be the opposite end of the spectrum than something like BPEL and ESB provides or other orchestrations mechanisms for SOA. While SOA embraces services being stateless, Microservices embrace the need for state and embrace OOP fundamentals of keeping your business logic and data together.

Read the SOA manifesto, and read the wikipedia page on SOA, try to do the exact opposite and you are a good ways down the road of adopting a Microservice architecture. Shun statelessness and embrace sharding, replication, eventual consistency and service data ownership. Shun WSDL and embrace JSON. REST and HTTP calls rule the day. Streaming calls over WebSocket is done for speed. Be the web.

Microservice doubters

Before you roll your eyes and buck against the goad, realize that words have power. People were doing Ajax before it was called Ajax. Giving something a name, helps guide it evolution and development. People were doing NoSQL before the term was coined, but the ideas were powerful, and in many ways Microservice is a term like Ajax and NoSQL, it is a post adoption name for what people are mostly already doing to a certain extent.  

”Microservices” - is a powerful term. It is one you need to embrace. It is very pragmatic. It is likely that you have been doing some or all of what Microservice Architecture is already. One day Microservice will go the way of Ajax, it will be so common it will be a forgone conclusion. Microservices is about pragmatism not vendors raising barriers to competitors by creating overly complex specifications.

We need to define Microservices a little more formerly so it does not go the way of SOA and every vendor gets to form a committee and dishonor its intentions with a set of nifty shiny baubles. SOA was a huge success and a failure. A failure because the term has actually no meaning or rather so many meanings, and yet Amazon and Google’s version of SOA are a huge success. Look at successful things that we had to call SOA and those are probably more like microservices and plain web services.

From here on out, when I refer to SOA, I am referring to the big enterprise WSDL, SOAP, stateless, course-grained, BPEL, ESB vendor lovefest that is SOA not the useful SOA implementations. Good SOA is called web services and is HTTP calls, REST and WebSocket. SOA is the aforementioned quagmire.

Microservices support mobile clients well

In the Java world, microservices has a special meaning. Besides Android, which uses the language formerly known as Java 1.6, Java on the client is dead or very insignificant. One of Java’s many roles is a platform for building services for mobile devices and web clients. This is another reason to embrace a common, flexible, HTTP standards based communication mechanism, because one of the biggest consumers of services are mobile apps, and web apps. Web Apps are thicker than before. Mobile apps might be written as Web Apps and / or have components that run in both the mobile platform and a pure browser play. This is not new. This is an ongoing trend. The idea of a pure monolithic-style web application written in Java is a dinosaur. It became infeasible somewhere between 2008 and now. Java services can be the MC in the MVC world but Swift, JavaScript, ObjectiveC, Android, own the clients. Clients are getting fatter which means services are getting thinner. This is a major driving force in microservice proliferation.

One thing well principle and fallacy of the monolith

Continuous delivery: The microservice architectural approach is to create smaller services that are focused on a small business domain or crosscutting concern. Microservices adopt the Unix single purpose utility approach to service development. They are small so they can be released more often and are written to be malleable. They are easier to write. They are easier to change. Microservices go hand in hand with continuous integration and continuous delivery. The services are independent enough not to need a gigantic release train to release improvements or new features. In the Java world, this means you will be using other microservice like Jenkins to provide frequent releases.

Do not try explaining the microservice style by comparing it to the monolithic application. No one ever set out to purposely build a huge monolith, and a monolith is not a three tiered web application. Yes monolithic applications happen. But comparing microservices to a monolith is like comparing agile development to not having any process.

The huge monolith is not an architecture style. It is usually a mistake. Collection of technical debt absorbed by a development team when the product team goes off the rails. Also even if someone was building a large web application, it is often the case (ok not that often) that they broke their code up into modules and have internal services that might be easier to break off into service and then one day microservices. If their application is a big ball of spaghetti code, then that has nothing to do with lack of microservices or them trying to develop a monolithic, their process, training, and culture is the issue. And it probably has a lot to do with product pressures and process.

The first issue I have with calling something a monolith which gives it a negative connotation is that what you are really describing is a web application in the context of most discussions of microservices, and not a true monolith. If you have a successful web application and it is getting so large that you have to break it up into many web applications or services, this is a good thing. You have what we call a successful application. There are tools that simplify writing basic web applications like Rails, Grails, Django, some PHP framework and their ilk. These are great tools.

There is nothing wrong with the basic web application. If your application is growing larger in features and functionality than you expected then using these tools to create a basic web application might be the wrong approach or you may have to go off the rails with these tools and start breaking features off into services and perhaps microservices. If you know that many of your clients are going to be mobile clients and you have potentially hundred thousand or 1 million users then these tools are probably the wrong approach. If you're writing a department level application and these tools are exactly the right approach and you may not need to use microservices. Don’t be dogmatic. In the end, the goal is to provide business value and ROI for your company, not impress your friends at the next JUG meeting. Not every tool or architecture fits every application.

Not every application has scalability concerns or the same types of scalability concerns. A microservice may benefit greatly by using some of these monolithic tools and frameworks to provide admin visibility and diagnostics into the microservice. Don't throw the baby out with the bathwater. It's all about engineering trade-offs and knowing what are the driving factors that are going to form your architecture. Sometimes this is an easy thing to do. Other times you have to write that first version, get it going and then come back and revisit the architecture. Microservices are a great approach for many things, and increasingly so with disparate clients, cloud and mobile, but don’t let the tail wag the dog.

Enterprise Applications flashback

While it is true Enterprise Applications are often built with a three tier architecture: backend code, database code and a GUI written in HTML/JavaScript. This has more to do with the world changing than an active choice. The real drivers for Microservices Architecture are cloud/virtualization/OS containerization, EC2, proliferation of mobile devices, cheaper memory, more virtualization, more cores, SSD, trend towards fatter clients, 10 GBE, 100 GBE, etc. The big enterprise web app is becoming obsolete to a certain extent at least for large applications. It is not like we one day came up with a better idea, and then one day we were like hey what if we could make our code more modular. The ground changed under our collective feet. The way we build, deploy, and consume has changed. Hardware evolved. Virtualization evolved. Cloud computing became a real thing. The smart phone /  tablet / mobile revolution happened. Microservice is the response to these external events.

Microservices and NoSQL are two trends that are more focused on how to address software development where deployments are increasingly cloud based and clients are increasingly mobile based. Just like you can’t compare client / server development of the mid-90s to mainframe development from the 70s, you can’t compare enterprise applications from 2001 to microservice development targeting mobile and web clients and other microservices in 2015. The world changes. We adjust. Microservice trend is course correction not a new religion.

History of Enterprise Applications

Remember 1990s, the reason why Enterprise Applications are written with three tiers was was to avoid DLL hell, and the monolith. We just did it with the tools available at the time. Back in the day, we used to build apps that were two tiered. You had to actually go to each users machine and help them install the app. There was a damn good chance they downloaded some shareware that installed a DLL that screwed up the install, and you were in hell. It was not like we were, “Hey James!” .. “What Martin?” “Do you want to build a huge monolith?” “Sure Martin!”.

We tried Applets but Java GUI development back then sucked (1999). I don't think it sucks so bad now, but it lost its window of opportunity for adoption. Then we were left with HTML/JavaScript clients and forcing everyone to use the same browser in the corporation at least for the corporate apps.  I worked at many corporations (2003) that banned JavaScript due to incompatibilities of browsers. You were left with screen painting with HTML. It was a like a colorful green screen of yesteryear, but way slower, but pretty. There are many reasons why this style of “Enterprise Development” does not work in the cloud and for mobile devices. The server-side applications no longer need to handle HTTP requests,  get data from a database and execute all domain logic, and draw pretty pictures in HTML. Much pain, and great expense has been incurred trying get this three tier architecture to scale in the cloud for various devices written in a polyglot of languages. Microservices exists because mobile, cloud, cheaper RAM, cheaper disks, and improved virtualization. It is really just taking the world where it is. It is not revolutionary at all.

Server components, EAR files and WAR files.. may they rest in peace

If you have lived through COM, DCOM, CORBA, EJBs, OSGi, J2EE, SOAP, SOA, DCE, etc. then you know the idea of services and components is not a new thing, but they are a date expired concept for the most part. One issue with enterprise components is they assume the use of hardware servers which are large monoliths and you want to run a lot of things on the same server. That is why we have WAR files and EAR files, and all sorts of nifty components and archives. Well turns out in 2015, that makes no sense. Operating systems and servers are ephemeral, virtualized resources and can be shipped like a component. We have EC2 images AMIs, OpenStack, Vagrant and Docker. The world changed. Move on. Microservices just recognize this trend so you are not developing like you did when the hardware, cloud orchestration, multi-cores, and virtualization was not there. You did not develop code in the 90s with punch cards did you? So don’t use an EAR file or a WAR file in 2015.

Now you can run a JVM in a Docker image which is just a process pretending to be an OS running in an OS that is running in the cloud which is running inside of a virtual machine which is running in Linux server that you don’t own that you share with people who you don’t know. Got a busy season? Well then, spin up 100 more server instances for a few weeks or hours. This is why you run Java microservices as standalone processes and not running inside of a Java EE container.

The Java EE container is no longer needed because servers are not giant refrigerator boxes that you order from Sun and wait three months for (circa 2000). Don’t fight classpath, classloader hell of Java EE. Hell your whole damn OS is now an ephemeral container (Docker). Deliver an image with all the libs you need, don’t deploy to a Java EE server which has to be versioned and configured. You are only running one service in it anyway. Turns out you don’t have five war files running in the same Java EE container since oh about 2007. Let it go.

If you are deploying a WAR file to a Java EE container then you are probably not doing microservice development. If you have more than one WAR file in the container or an EAR file, then you are definitely not doing microservice development. If you are deploying your service as an AMI or docker container and your microservice has a main method, then you might be writing a microservice.

Microservices architectures opt to break software not into components but into reusable, independently release-able services which run as one or more processes. Application and other services communicate with each other. So where we might have used a server side component, we use a microservice running in independent processes. Where we might have had WAR files or EAR files now we have a Docker container or a Amazon AMI that has the entire app preloaded and configure with exactly the libraries it needs (Java and otherwise).

JSON, HTTP, WebSocket … NO WSDL!

Now you just have to document the Microservices HTTP/JSON interface so other developers can call into it. We could say REST, and certainly you can use concepts from REST, but hey HTTP calls are enough to be considered a Microservice.  

Keep this in mind: No XML. No SOAP. No WSDL. No WADL. JSON! Ok you can add some meta data and document how to talk to your service, but the idea is the docs should be documented with curl. If you are only using SOAP or XML then you are not producing a microservice. JSON is a must.  

Documents should sound more like: I give you this request with these headers, params and JSON body and you respond with this JSON. Keep it simple. You can provide things in addition to JSON, but JSON is the minimum requirement. If you are not delivering up JSON and consuming JSON over HTTP or HTTP WebSocket then what you wrote is not probably not a microservice.

Call speed, non-blocking calls

One of the issues with remote calls is speed. This is why you will want to organize services around a domain that will help keep the data for that service with that service and it will not need to interact with other services or a foreign database every time it gets a request for its data.  While remote calls are expensive this can be accommodated for by using async calls, batching, and WebSocket/JSON (Reactive Programming Microservice Java Lib)  . Remember WebSocket allows bi-directional communication. For speed, you should prefer RPC calls that are non-blocking and can be sent in batches (POST or WebSocket).  If you are able to handle requests in streams in an async manner and utilize the hardware efficiently, then you might be doing microservice development.

Another approach for increasing remote call speed is to go all SOA on your API and focus on coarser-grained responses, but this is almost always a mistake. You can write coarser-grained HTTP APIs so more is delivered with each call. This is a problem because it is harder to write and use coarser-grained HTTP APIs as they often conflate many subdomain data in the same call in the name of speed and aggregation. It is easier to batch many smaller calls and create service aggregators. You will need to do both batching and aggregation of domains (coarser grained). Dumb fast pipes and batching calls (turning them into streams) are a good indication that what you wrote is a microservice. Some call this style reactive programming.

Depending on scalability needs services may need be sharded. While a service runs in a single process for scalability that service may really be running in many processes on many virtual machine. Microservices are not stateless. Microservices should own their data. This may mean a private database. This may mean using a data lease model for elasticity. Or a private database shard per sharded service. We will talk more about this later. To learn more about how to scale microservices check out high-speed microservices. To learn more about what a Java microservice looks like read rise of the machine.

If your service is getting all of its data from a database that is shared by one or more web applications, and/or other services and/or other application, then you did not write a microservice. If you service is 100% stateless, then what you wrote is not a microservice.

Microservices do not negate the need for having libraries. If you are making many calls to a microservice, there is an indication that you maybe needed a library instead of a microservice. Adopting microservice architecture does not make you a better systems engineer. You will need some common sense, systems knowledge and/or a very good perf testing regiment. Many will fail and go back to a traditional three tier, web development version of services or some form of enterprise SOA or write a more traditional stateless web service.

Microservice, Process, DevOps

This is a point in the discussion where people conflate a lot of ideas from other pet projects or pet processes and shove that into the microservice realm. Let’s not. Something that has so many definitions tends to get diluted and meaningless.

Concepts like DevOps, continuous delivery, continuous integration, cloud computing, agile processes etc. are compelling in their own right. You don’t need to have an Agile, DevOps, Cloud setup to start getting benefits from microservice development. Certainly they complement each other. Don’t combine them as one concept. Microservices has a lot more to do with cloud, virtualization, OS containerization, etc. then it does agile development vs. RUP.

The idea of cradle-to-grave development of an application or service is not an idea that was invented in the early or late 2000s. Hiring software developers that were also systems engineers and responsible for software development, load testing, testing, and deployment is not a new concept to me at all. This is a recurring concept that I have seen in my 25 year career over and over. It is the more rare concept and one that I find completely dysfunctional where the developers tossed something over the wall to QA and ops to deploy. I feel the strong urge to tell you a lot of antidotes for my career so I have pinched myself very hard, snapped my red suspenders and yanked the hair of my white beard to resist.

Smart endpoints and dumb pipes: Actors, Reactive and Active Object

Enterprise Service Bus (ESB), and vendor driven SOA is a bad idea. It is one you hear a lot of talk about but you never see actual successful deployments of. Complex message routing, choreography, transformation, and applying business rules willy nilly and providing a graphical representation of your overly complicated runtime software process is a horrible idea. I am not saying it is never needed. You can, do and will need to integrate with legacy applications and tools like Camel et al can help, but you should not start designing your system from scratch around ESBs and Camel. It is a necessary evil at times. But evil none the less.

Yes wrapping legacy “monolithic” or yesteryears CORBA, DCOM, TPS, expensive message bus, from high-priced vendor in SOA and ESB or even good old “REST” is sometimes needed, but that does not mean that you get to redefine microservices into it. Microservices is not legacy integration. Microservices is how you develop greenfield services in 2015. It is not a legacy turd polish like SOA and ESB. (Even if sometimes you do need legacy turd polish.)

You know where a good place to store your business logic, code. Business people almost never, never, never change business rules on the fly because if they did that would be like editing code and would require testing or production would go down. This is not to say that there are not real use cases for things like Drools. This is to say Drools and Activiti are the exception not the rule. You know who is good at changing business rules in code and making sure production does not go down, your software development team. All developers prefer smart endpoints and dumb pipes because BPEL, Activiti, WS-Choreography or BPEL or orchestration are demoware bullshit. If you have ever had the joy of maintaining or trying to grok a big mass of spaghetti crapfest, you realize that creating cool, neat-o tools around legacy integration batch jobs does not make them easier to comprehend or maintain.

JSON, HTTP, WebSocket work everywhere. They work in browsers. They work from Python, Ruby, Perl processes. They work from Java. They work from PHP web applications. They work in all mobile clients. They are the least common denominator and they are simple to read and understand and document.

Web Services are great. Vendor driven SOA orchestration and ESBs are a nightmare to debug. Not all Web Services should be microservices. But you should have a very strong and powerful reason for using SOAP/WSDL/Vendor SOA.

Developers the world over prefer to use protocols that the world wide web and Unix are built on. Because it is simple to grok. Operationally predictable. Easy to cache even with third parties like Akamai and tools like Varnish. I prefer Unix cron jobs, web monitoring, cloud, etc. to ESB. Sometimes a batch job is just a batch job. Wrapping it in ESB or making a Map/Reduce cluster might make you feel more important, but is it providing business values. Sometimes it is. More often than not, it is misapplied tech. Not every batch job needs to be a Hadoop Map/Reduce or ESB orchestration, sometimes they can just be batch jobs that are kicked off by cron. (ESBs and Hadoop are needed. I am not anti-ESB or Map/Reduce).

You invoke micro services through HTTP and WebSocket or some form of messaging. You should prefer HTTP and WebSocket and only use messaging (with MOM) if you want durability. WebSocket is lightweight messaging. It is supported by all programming languages, mobile platforms and the web and it exists today. 0MQ, RabbitMQ, JMS, Kafka are all great if you need some level of durability or speed. If you do not, then WebSocket should be enough for 99% of your needs and has some added benefit of being reachable from web clients.

The message bus delivers an opaque message, so you will need to encode the message in a universally understood format, which usually ends up being JSON. There are times when you may need something faster and binary JSON-like formats abound (binary should be the exception, JSON is as fast as most binary formats, and faster than many). If something is available via a message bus call, then it should also be available via an HTTP/JSON call (REST). If you can’t exercise your service API via curl and have it return JSON, then you did not write a microservice.

With tools like Akka, QBit, Vertx, etc. the concept of an event bus or a message bus or streaming messages or streaming calls exists as core concepts to the async invocation model. Messages can be handled at first in-process by a service, as more scale is needed, those messages can be batch/forwarded to other nodes. You can write services internal to your application which are somewhere between a library and a microservice which can one day be more readily broken out into actual microservices. Reactive, Actor and Active Objects embrace the concepts of streams, message queues and event busses. They are a natural fit for a microservice architecture.

Batching of messages, back pressure based batching, are used to send courser grained messages over the wire which can minimize the performance loss of moving services to processes running on other machines. Since Akka, Finagle, QBit and Vertx support in-proc services and out of proc services using the same underlying interface to a dumb pipe, it is easier to move services in-proc or out of proc (microservice) as performance needs dictate.

If you are making a lot of blocking calls, then what you wrote is not a microservice.

Tool-stacks, polyglot programming languages

Standards are great that is why there are so many of them. Attempts to standardize on technology platforms are often thwarted by using vendor products, merging with other companies, buying companies, adopting new mobile platforms, constant churn of new client frameworks and platforms, etc. Microservices Architecture embraces polyglot programming languages and programming languages. This is where microserver HTTP/WebSocket and JSON/tolerant readers come into play as it provides a minimal pipe and format to support change and polyglot of programming languages.

Services own their data

Databases are used for reporting and offline analysis. Service stores (which could use a database), is for a service or a service shard. Operational data and reporting / historical / backup data should be split up. Operation data for a service should only be editable by that service.

Microservices architecture is back to OOP basics. Objects, services when you expose them remotely, own their data and business logic. Each service stores data and models data in its view of the world. Data is specific for its service. This is not a new concept. This is basic OOP.

In additional evidense of going back to OOP roots and domain drviven development, microservices own their data storage. This could mean a database or key/value store on the same docker node as the JVM running the service (which gets replicated and backed up of course). Microservices do not use a shared database for operational data. A micro-service might fallback to a shared database if data is not in the service, but this would be the exception not the rule.

If someone thinks that Microservices are stateless it is ok to roll your eyes at them. A Microserver edits data, and it may replicate it, shard it, and use eventual consistency to back that data up, but it owns the data and it and it alone can edit its data. If that is not what you wrote, then you did not write a microservice. In-memory computing, eventual consistency, replication, fit nicely with microservices. Remember RAM is the new disk, and the hard drive or SSD is the new tape backup. True speed is when you can edit your data without first checking to see if you have the latest copy from the database that sits on another server.

Service Discovery - Design for Failure

In addition to automated deployment, virtualization, and cloud orchestration/automation, microservices use service discovery and service monitoring to recover from failure and to spread load across more nodes. The ability to discover service nodes, and adding them into the mix is called elasticity. This includes monitoring of services. Detecting failures. Removing unhealthy nodes out of the mix. Adding additional services into a running system.

A key component of microservices architecture is reactive programming, which is an async programming mode, and the ability to use back pressure to fail gracefully if the load surpasses the capacity of a node rather than having a cascading failure.

If your service under unexpected load becomes unresponsive then you did not write a microservice. If your service under load, throws error messages and tells clients it is under too much load, and you can spin up new nodes, and the nodes can discover each other (service discovery) and live another day, then you wrote a microservice. Microservices are resilient and elastic.

Summary

Keys ways to identify Microservices:

• Uses Service Discovery - it might be a microservice
• Async model, non-blocking - it might be a microservice
• Uses back pressure to detect when its overloaded to survive another day - it might be a microservice
• Uses monitoring to report health status - it might be a microservice
• Used a bounded domain and owns it data - it might be a microservice
• APIs are reachable via WebSocket, HTTP over JSON - it might be a microservice
• Uses Docker or an Amazon AMI - it might be a microservice
• Your service runs a single process or a set of sharded services - it might be a microservice
• You don’t hard code locations of services, you use service discovery like etcd, consul, or some other cloud friendly, elastic way to spin up services
• You also store other forms of config in cloud friendly places like etcd and consul (or s3 or inside the hidden ec2 instance web server), and you can get updates while the service is running
• You use back pressure. If you service gets overloaded, it does not become unresponsive, it sends currently unavailable messages so upstream clients and services can route around it (they could even get the last known good whatever from s3 or from Akamai.)
• One downstream service that in unresponsive will not kill all upstream services and clients

Keys ways to identify Java Microservices

• You don’t use an EAR or WAR files
• Your services have main methods
• You don’t auto-deploy java components, you auto-deploy AMI images or docker containers
• You use a fatjar that is runnable as a process (or sharded services)
• Or you use gradle distZip or distTar
• Your code is non-blocking. If you have to use a blocking API for integration, you do it from a non-blocking worker pool.

Just remember Microservices are not a new thing, and they are not cool or hip. Microservices are obvious evolutionary architecture to address the revolutionary things that already happened: web, cloud, mobile, server virtualization, OS containerization, multi-core servers, cheaper and cheaper RAM, 64 bit computing, 10GBE, 100GBE, etc. This is not extreme or cool. This is obvious reactionary architecture to this amazing tech landscape that we live in. It is less about the fashion show of technology coolness and more about not using tools and techniques that are antiques in this environment. Give up your punch cards, and WAR files, and join in.

I have more to say but this blog post is way too long already. I leave you with these links. Thank you for reading if you got this far.

In the mean time.. feel free to read about QBit the Java Microservice Lib that focuses on Microservices, WebSocket, JSON and HTTP using Active Objects and high-speed queueing, messaging and event bus for modern cloud and mobile applications back-ends.

References and reading materials:

1. Microservices by Martin Fowler and James Lewis

2. Microservices Architecture by Chris Richardson

3. Micro services – Java, the Unix Way by James Lewis

4. Microservices, or How I Learned To Stop Making Monoliths and Love Conway’s Law by Cliff Moon

5. Micro service architecure by Fred George

6. Microservices are not a free lunch by Benjamin Wootton

7. Antifragility and Microservices by Russ Miles

8. The Unix Philosophy

9. Conway’s Law

10. Amazon Architecture

11. Migrating to microservices by Adrian Cockroft

12. Microservices with Spring Boot

13. Microservices for the Grumpy Neckbeard

14. Microservices definition on Wikipedia

15. Microservices and DevOps by Adrian Cockcroft

16. Building and Deploying Microservices - Bart Blommaerts

17. Microservices on the JVM - Alexander Heusingfeld

18, Microservices Shaun Abrams
19, Java Microservices Architecture -Rick Hightower
20, Microservice Service Discovery with Consul - Rick Hightower
21, Reactive Microservices - Rick Hightower
22, High Speed Microservices -Rick Hightower

More about QBit

What is QBit again?

QBit is a queuing library for microservices. It is similar to many other projects like Akka, Spring Reactor, etc. QBit is just a library not a platform. QBit has libraries to put a service behind a queue. 

QBit queues can be dealt with directly or you can create a service. QBit services can be exposed by WebSocket, HTTP, HTTP pipeline, and other types of remoting. A service in QBit is a Java class whose methods are executed behind service queues. 

QBit implements apartment model threading and is similar to the Actor model or a better description would be Active Objects. QBit does not use a disruptor. It uses regular Java Queues. QBit can do north of 100 million ping pong calls per second which is an amazing speed (seen as high as 200M). QBit also supports calling services via REST, and WebSocket. QBit is microservices in the pure Web sense: JSON, HTTP, WebSocket, etc.

Reactive Programming, Service Discovery, Service types, Client proxy types, Java MicroService Lib update for QBit

Service Discovery

We are integrating now with consul.

This allows services to show health status (up, down, warning).

This is used for service discovery and clustering.

QBit has a clustering event bus (not part of QBit core, but a library).

We had event bus replication. Then we added the clustering by using Consul discovery.

Consul is like ZooKeeper or etcd.

We took what was in the event bus clustering logic and broke that out into a core concept of a ServiceDiscovery.

ServiceDiscovery allows a developer to manage ServicePools.

A ServicePool is a collection of service definitions for a service name, and it fires events when a service is added to the pool, or removed. 

ServiceDefinition is host, port, health, service name and unique service id.

If a service becomes unhealthy using Consul monitoring it gets taken out of the pool.

There is a plan to support a wide variety of ways to do ServiceDiscovery.

Consul is just the first one.

Reactive Programming

We improved the reactive programming support. 

QBit has always been reactive as it is async and uses the Active Object Architecture which is a derivative of the Actor model.

It is now easy to do things like call Service A, call Service B, then use the results from B to call Service C, return the results of A/C, and timeout if A takes longer than 50 ms, timeout if B takes longer than 100 ms, timeout if BC take longer than 250 ms, and timeout if the whole thing takes longer than 250 ms. It does this an async manner. It is wicked fast.

This reactive coordination uses Java 8 lambdas. 

Handling events, REST calls, WebSocket calls are all done through service methods. 

The programming model is POJOs.

QBit is a Java first programming model. It uses common Java idioms to do reactive programming.

It focuses on Java 8. It is one of the few of a crowded field of reactive programming libs/frameworks that focuses on Java 8. It is not a lib written in XYZ that has a few Java examples to mark a check off list. It is written in Java and focuses on Java reactive programming using active objects architecture which is a focus on OOP reactive programming with lambdas and is not a pure functional play. It is a Java 8 play on reactive programming. 

Services can be stateful, which fits the micro service architecture well. Services will typically own or lease the data instead of using a cache.

Service types

Service Bundle, many async services sharing a single result queue and/or sharing a request queue.

Service Server, a wrapper for a Service Bundle that exposes services to the web.

Service Queue, an in-proc, async service.

QBit uses batching queues for performance.

There is also.

CPU Sharded services, each service does a portion of the workload in its own thread to maximize core utilization.

The idea here is you have a large mass of data that you need to do calculations on. You can keep the data in memory (fault it in or just keep in the largest part of the histogram in memory not the long tail). You shard on an argument to the service methods. (This was how I wrote some personalization engine in the recent past).

Worker Pool service, these are for IO where you have to talk to an IO service that is not async (database usually or legacy integration) or even if you just have to do a lot of IO. These services are semi-stateless. They may manage conversational state of many requests but it is transient. 

ServiceQueue wraps a Java object and forces methods calls, responses and events to go through high-speed, batching queues.

ServiceBundle uses a collection of ServiceQueues.

ServiceServer uses a ServiceBundle and exposes it to REST/JSON and WebSocket/JSON.

Events are integrated into the system. You can register for an event using an annotation @EventChannel, or you can implement the event channel interface. Event Bus can be replicated. Event busses can be clustered (optional library). There is not one event bus. You can create as many as you like. Currently the event bus works over WebSocket/JSON. You could receive events from non-Java applications.  

Clients:

QBit supports creating async clients.

You can use the Client object to create a proxy to a remote service.

You can use the Service Queue object to create a local in-proc proxy to a service.

The programming model is the same for local and remote proxies. 

It is possible to make 720M method calls a second (this include sending the response back) from a single remote proxy.

More to come.

In the mean time.. feel free to read about QBit the Java Microservice Lib that focuses on Microservices, WebSocket, JSON and HTTP using Active Objects and high-speed queueing, messaging and event bus for modern cloud and mobile applications back-ends.

QBit lingo

QBit is a Java microservice lib supporting REST, JSON and WebSocket. It is written in Java but I might one day write a version in Rust or Go or C# (but that would require a large payday).

Service POJO (plain old Java object) behind a queue that can receive method calls via proxy calls or events (May have one thread managing events, method calls, and responses or two one for method calls and events and the other for responses so response handlers do not block service. One is faster unless responses block). Services can use Spring MVC style REST annotations to expose themselves to the outside world via REST and WebSocket.

ServiceBundle Many POJOs behind one response queue and many receive queues. There may be one thread for all responses or not. They also can be one receive queue.

Queue A thread managing a queue. It supports batching. It has events for empty, reachedLimit, startedBatch, idle. You can listen to these events from services that sit behind a queue. You don't have to use Services. You can use Queue's direct.

ServiceServer ServiceBundle that is exposed to REST and WebSocket communication

EventBus EventBus is a way to send a lot of messages to services that may be loosely coupled

ClientProxy Way to invoke service through async interface, service can be inproc (same process) or remoted over WebSocket.

Non-blocking QBit is a non-blocking lib. You use CallBacks via Java 8 Lambdas. You can also send event messages and get replies. Messaging is built into the system so you can easily coordinate complex tasks.

Speed There is a lot of room for improvement with Speed. But already QBit is VERY fast. 200M+ TPS inproc ping pong, 10M-20M+ TPS event bus, 500K TPS RPC calls over WebSocket/JSON, etc. More work needs to be done to improve speed, but now it is fast enough where I am working more with usability.

Learn more about QBit:

[Detailed Tutorial] QBit microservice example

[Doc] Queue Callbacks for QBit queue based services

[Quick Start] Building a simple Rest web microservice server with QBit

[Quick Start] Building a TODO web microservice client with QBit

[Quick Start] Building a TODO web microservice server with QBit

[Quick Start] Building boon for the QBit microservice engine

[Quick Start] Building QBit the microservice lib for Java

[Rough Cut] Delivering up Single Page Applications from QBit Java JSON Microservice lib

[Rough Cut] Working with event bus for QBit the microservice engine

[Rough Cut] Working with inproc MicroServices

[Rough Cut] Working with private event bus for inproc microservices

[Rough Cut] Working with strongly typed event bus proxies for QBit Java Microservice lib

[Rough Cut] Working with System Manager for QBit Mircoservice lib

[Z Notebook] More benchmarking internal

[Z Notebook] Performance testing for REST

[Z Notebook] Roadmap

Home

Introduction to QBit

Local Service Proxies

QBit Boon New Wave of JSON HTTP and Websocket

QBit Docs