Rick

Rick
Rick

Sunday, April 20, 2014

New JSON serialization benchmark comparing Boon, GSON, Jackson, and Groovy 2.3

This blog post is actually talking about Andrey's benchmark, not the ones that I did. Sorry for the confusion. So if you came here from InfoQ. I think you really meant to go here which includes an FAQ, and the methodology of the benchmark.

 Here is a brief summary of the benchmark that I created**. 

 (** Actually no... it was forked from Stephane Landelle benchmark, and I kept adding to it. Stephane Landelle is a performance testing guru of Gatling fame, which is similar to JMeter or Grinder. Stephane Landelle was the first user of Boon JSON outside of me. I wrote Boon JSON support but it would not exist as it is today without Stephane writing the benchmarks and taunting me. Also the Boon parser got a LOT better after merge forking it into Groovy. The key is to get a lot of help from smart people. :)
"Jackson is consistently faster than GSON and JSONSmart. Boon JSON parser and the new Groovy 2.3 JSON parser are faster than Jackson. They are faster with InputStream, Reader, reading files, byte[], and char[] and String."



 Again.. my benchmark is here. This blog post was all about Andrey's benchmark, which I did not write. 

 Now... Your regularly scheduled blog post....

Boon comes in 1st place. Groovy comes in a strong second place. Jackson comes in third. GSON comes in last.


See code and full benchmark here:

https://github.com/bura/json-benchmarks


The benchmark was created by Andrey Bloschetsov from Moscow.

JSON serialization benchmarks

Methodology

Participants:
For testing were selected data with different structure:
  • citys - A large array (29470 items) of simple objects. The compact json representation takes about 2.5 MB.
  • repos.json - An array of four objects with complex structure. The compact json representation takes about 342.8 kB.
  • user.json - one object with a complex structure. The compact json representation takes about 4.2 kB.
  • response.json - one object with a simple structure. The compact json representation takes about 425 B.
Serialization was tested in two versions:
  • pojo - Objects were presented as a POJO objects.
  • maplist - Objects were presented as Map-s.
On deserialization were tested only transformation from String into Map-s.

Build and Run

./gradlew clean && ./gradlew shadow -DgroovyVersion="2.2.2" && java -Xmx2048m -jar target/benchmarks.jar ".*Benchmarks.*" -f 1

Summary

  • Boon is in 1st place with 28 points
  • Groovy 2.3 is in 2nd place with 22 points
  • Jackson is in 3rd with 18 points
Boon, Groovy 2.3 and Jackson were usually in the top three.
Points
Toolpoints
Boon28
Groovy 2.322
Jackson18
GSON3
1st place
ToolCountpoints
Boon721 points
Jackson39 points
Groovy 2.326 points
2nd place wins
ToolCountpoints
Groovy 2.3612 points
Boon36 points
Jackson36 points
3rd
ToolCountpoints
Jackson33 points
Boon22 points
Groovy 2.344 points
GSON33 points

Results

Serialization

Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonpojocitysthrpt2027.4120.703ops/s
gsonpojocitysthrpt2015.6020.280ops/s
boonpojocitysthrpt2025.1190.519ops/s
groovypojocitysthrpt201.6840.082ops/s
groovy-2.3pojocitysthrpt2019.5131.120ops/s
Jackson wins but it is almost a tie between Jackson and Boon. Groovy 2.3 close on the heels of Boon and Jackson.
  • Jackson 1st
  • Boon 2nd
  • Groovy 2.3 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonpojoreposthrpt20641.99914.161ops/s
gsonpojoreposthrpt20407.0935.950ops/s
boonpojoreposthrpt2024173.808223.995ops/s
groovypojoreposthrpt2045.3020.301ops/s
groovy-2.3pojoreposthrpt20643.90210.174ops/s
Boon wins by a wide margin. Groovy 2.3 beats Jackson.
  • Boon 1st
  • Groovy 2.3 2nd
  • Jackson 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonpojouserthrpt2052324.635359.331ops/s
gsonpojouserthrpt2022010.943208.352ops/s
boonpojouserthrpt20241065.7871764.331ops/s
groovypojouserthrpt201751.39414.458ops/s
groovy-2.3pojouserthrpt2047781.079370.971ops/s
Boon wins by a wide margin. Groovy 2.3 very close to Jackson, but Jackson wins.
  • Boon 1st
  • Jackson 2nd
  • Groovy 2.3 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonpojorequestthrpt20629985.1358049.270ops/s
gsonpojorequestthrpt20267811.4514064.942ops/s
boonpojorequestthrpt20344047.7953447.635ops/s
groovypojorequestthrpt2028826.587302.826ops/s
groovy-2.3pojorequestthrpt20649596.7006931.673ops/s
Both Jackson and Groovy 2.3 beat Boon by 2x! Groovy 2.3 beats Jackson but almost a tie.
  • Jackson 1st
  • Groovy 2.3 2nd
  • Boon 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonmaplistcitysthrpt2027.0270.686ops/s
gsonmaplistcitysthrpt2015.9970.196ops/s
boonmaplistcitysthrpt2025.0570.371ops/s
groovymaplistcitysthrpt201.6350.073ops/s
groovy-2.3maplistcitysthrpt2019.6820.435ops/s
Jackson and Boon are neck and neck, but Jackson wins. Groovy 2.3 tight on the heels of Jackson and Boon.
  • Jackson 1st
  • Boon 2nd
  • Groovy 2.3 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonmaplistreposthrpt20644.7179.192ops/s
gsonmaplistreposthrpt20403.7604.575ops/s
boonmaplistreposthrpt2024173.738216.084ops/s
groovymaplistreposthrpt2044.3430.253ops/s
groovy-2.3maplistreposthrpt20653.4029.880ops/s
Boon wins by a wide margin. Groovy 2.3 beats Jackson but it is a close race.
  • Boon 1st
  • Groovy 2.3 2nd
  • Jackson 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonmaplistuserthrpt2051317.860590.524ops/s
gsonmaplistuserthrpt2021844.772246.724ops/s
boonmaplistuserthrpt20235728.3182876.144ops/s
groovymaplistuserthrpt201802.38315.716ops/s
groovy-2.3maplistuserthrpt2047497.203526.286ops/s
Boon wins by a wide margin. Groovy 2.3 and Jackson are really close, but Jackson wins.
  • Boon 1st
  • Jackson 2nd
  • Groovy 2.3 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonmaplistrequestthrpt20616497.0138990.165ops/s
gsonmaplistrequestthrpt20268005.6422219.178ops/s
boonmaplistrequestthrpt20353171.0652502.621ops/s
groovymaplistrequestthrpt2028985.824459.033ops/s
groovy-2.3maplistrequestthrpt20630975.8024892.114ops/s
Jackson and Groovy 2.3 beat Boon by 2x.
  • Groovy 2.3 1st
  • Jackson 2nd
  • Boon 3rd

Deserialization

Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonN/Acitysthrpt2021.9060.162ops/s
gsonN/Acitysthrpt2023.3770.501ops/s
boonN/Acitysthrpt2072.5431.054ops/s
groovyN/Acitysthrpt203.1820.034ops/s
groovy-2.3N/Acitysthrpt2052.8060.329ops/s
Boon wins. Groovy 2.3 comes in second.
  • Boon 1st
  • Groovy 2nd
  • GSON 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonN/Areposthrpt20484.3773.571ops/s
gsonN/Areposthrpt20412.5086.415ops/s
boonN/Areposthrpt201647.07015.817ops/s
groovyN/Areposthrpt2031.5540.252ops/s
groovy-2.3N/Areposthrpt201305.87611.445ops/s
Boon beats Jacskon by over 3x, Groovy 2.3 almost ties Boon.
  • Boon 1st
  • Groovy 2.3 2nd
  • Jackson 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonN/Auserthrpt2025109.072225.116ops/s
gsonN/Auserthrpt2026982.245282.070ops/s
boonN/Auserthrpt2072529.259687.818ops/s
groovyN/Auserthrpt202338.93419.511ops/s
groovy-2.3N/Auserthrpt2064431.5771037.016ops/s
Boon beats Jackson by 3x. Groovy 2.3 beats Jackson by 2x.
  • Boon 1st
  • Groovy 2.3 2nd
  • GSON 3rd
Benchmark(dataStyle)(resourceName)ModeSamplesMeanMean errorUnits
jacksonN/Arequestthrpt2090,009.471941.871ops/s
gsonN/Arequestthrpt20268,988.9052165.099ops/s
boonN/Arequestthrpt20672,907.3578514.806ops/s
groovyN/Arequestthrpt2026497.332196.347ops/s
groovy-2.3N/Arequestthrpt20762,926.2135930.640ops/s
Jackson comes in fourth. Groovy 2.3 comes in 1st. Boon comes in second.
  • Groovy 2.3 1st
  • Boon 2nd
  • GSON 3rd

Testing environment

Intel® Core™ i5-2410M CPU @ 2.30GHz × 4, Ubuntu 14.04 (64-Bit), Oracle Java HotSpot 64-bit 1.7.0_55


If you found this interesting you might find these related articles interesting.

Kryo vs. Boon JSON vs. Java Object Serialization
Boon is not just a fast JSON parser, it happens to be one of the fastest ways to do Java Object Serialization period. Boon Java JSON serialization is faster than Java Object Serialization (ObjectOutputStream/ObjectInputStream). It just happens to work with JSON. Kryo, a binary serializer which is the fastest way to serialize Java objects, wins by the way, but for large streams, Boon gets within 85% of Kryo. To get Java JSON serialization within 15% of the fastest Java Binary serializer took quite some effort. Boon JSON serialization is probably in the top five Java serializers for speed, and it is the fastest way to do JSON.

Which is faster serializing to JSON with Jackson or serializing to binary with ObjectOutputStream?
Most people assume that Java object serialization is faster than Jackson JSON serialization because Jackson is using JSON and Java object serialization is binary. Is this true? Find out.

Then a even more serious with many pretty graphs article is here:

Groovy and Boon provide the fastest JSON parser for the JVM
Jackson is consistently faster than GSON and JSONSmart. Boon JSON parser and the new Groovy 2.3 JSON parser are faster than Jackson. They are faster than Jackson with InputStream, Reader, reading files, byte[], and char[] and String. Groovy JSON support and the Boon JSON parser are up to 3x to 5x faster than Jackson at parsing JSON from String and char[], and 2x to 4x faster at parsing byte[].

Looking for a tutorial to get started with Boon?
Fastest Java JSON serializer/parser: Boon JSON in Five minutes!
This five minute guide will get you up to speed using Boon in no time. Parse and serialize Java objects to/fro JSON up to 5x faster! Boon is not a JSON parsing project. It is more than that, but JSON parsing is intrinsic to what Boon is all about. Boon is the fastest way to serialize and parse JSON in Java so far. It is faster at object serialization, enabling JSON expressions, JSON parsing and much more. Boon JSON is FAST! In addition it has a very easy to use, convention-based API. This getting started guide covers making JSON REST calls, JSON object serialization, using annotations, and more.

Need to do ETL with JSON, and/or want to search sort JSON or Java graphs quickly.

Indexed Searching, Querying Java objects, JSON and Maps in-memory and doing ETL like JDK 8 Streams but much faster
Java Boon - Boon Data Repo using Indexed Collections for Java Beans, JSON and Maps like JDK 8 streams but faster. Many languages have sorting, slicing, dicing capabilities for Lists and Maps, and JSON. Boon adds this to Java. Boon Data Repo is an in-memory query engine that can use high speed indexed collections for very fast in-memory queries. This brief tutorial will show you how to use Boon to query JSON, Java objects and Java maps. And also how to do ELT transforms with JSON, Java objects and Java Maps.

Java Boon filtering for Java Beans, JSON and Java Maps like JDK 8 streams but much faster
Java Boon filtering for Java Beans, JSON and Java Maps like JDK 8 streams but much faster. Many languages have support for querying objects and filtering objects easily. Java does in JDK 8, but Boon adds it as well, and it can compliment features of JDK 8 as well as work in Java 7 land. Boon adds filtering to to Java. You can filter JSON, Java, and Maps. You can also use indexed queries which are a lot faster than linear search queries that come with JDK 8.


Boon: Java path expressions for Java instances, Map/List and JSON
Many languages have path expressions. Boon adds this to Java. Boon has powerful path expressions that work with objects, lists and maps. It also works with JSON. Full examples of Java, Maps/Lists and JSON are included in this tutorial.

Just curious about what Boon is...
A Boon to Java Development (introducing Boon!)
Boon comes with helper methods that allow you to easily read files, create lists, sets, maps, concurrent maps, sorted maps, sorted sets, etc. It provides slice notation, searching, easy IO (files, http, etc.). The helper methods are safeList, list, set, sortedSet, safeSet,safeSortedSet, etc. The idea is to make Java feel more like list and maps are built-in types.

Java Collections: How to query them, sort them, easily and efficiently
What if Java collections and Java hierarchies were easily searchable? They are with Boon! What if it were easy to query a complex set of Java objects at runtime? What if there were an API that kept your object indexes (really just TreeMaps, and HashMaps) in sync.? Well then you would have Boon's data repo. This article shows how to use Boon's data repo utilities to query Java objects. Boon comes with a simple criteria API that makes working with Java collections a snap.

Boon Remix: How to Clone Collection in Java - Deep copy of ArrayList and HashSet with Boon
Boon has many utility methods to convert objects into maps, lists, JSON, etc. It also has many collection utility methods. One such utility method is deepCopy, which will deep copy a list or set. Programmer as a way to clone Collection e.g. List, Set, ArrayList, HashSet or any other implementation. Or they write a lot of boiler plate code and spend a lot of time doing things to copy lists and sets. With Boon you can do deep copies without messing around with clone.

Boon: Converting Java from Lists, Maps, and JSON and other boons to Java development
Boon gets some inspiration from Groovy and Python when dealing with maps, lists and JSON. It addition to adding slice notation and such, which I covered before. Boon has first class support for converting between Maps, List, JSON and Java objects. Boon gets some inspiration from Ruby, Groovy, Python and Perl and aims to make Java programming just a bit more fun.

I am still here.  I wrote a microservice framework that uses Boon. It is wicked fast. It is all about JSON, Java idioms, HTTP, and WebSocket to write small, tight, fast, microservices.

Check out QBit.


  • [Detailed Tutorial] QBit microservice example
  • [Doc] Queue Callbacks for QBit queue based services
  • [Quick Start] Building a simple Rest web microservice server with QBit
  • [Quick Start] Building a TODO web microservice client with QBit
  • [Quick Start] Building a TODO web microservice server with QBit
  • [Quick Start] Building boon for the QBit microservice engine
  • [Quick Start] Building QBit the microservice lib for Java
  • [Rough Cut] Delivering up Single Page Applications from QBit Java JSON Microservice lib
  • [Rough Cut] Working with event bus for QBit the microservice engine
  • [Rough Cut] Working with inproc MicroServices
  • [Rough Cut] Working with private event bus for inproc microservices
  • [Rough Cut] Working with strongly typed event bus proxies for QBit Java Microservice lib
  • [Rough Cut] Working with System Manager for QBit Mircoservice lib
  • [Z Notebook] More benchmarking internal
  • [Z Notebook] Performance testing for REST
  • [Z Notebook] Roadmap
  • Home
  • Introduction to QBit
  • Local Service Proxies
  • QBit Boon New Wave of JSON HTTP and Websocket
  • QBit Docs

    • Posted by at

    No comments:

    Post a Comment

    Subscribe to: Post Comments (Atom)
    Kafka and Cassandra support, training for AWS EC2 Cassandra 3.0 Training