Boon 2x faster than Jackson at InputStream and not using index overlay

Boon non-index overlay mode using inputstream4 minutes ago by Richard Hightower

This is with InputStream. According to Tatu comments InputStream is a use case that Boon could not compete in because boon is really just optimized for String. Tatu also said that Boon likely only wins because it uses index overlay.

So here is a test that uses Boon without index overlay and uses InputStream not String.

Benchmark                       Mode Thr     Count  Sec         Mean   Mean error    Units
MainBoonBenchmark.webxml              thrpt  16         6    1   455347.925    46637.751    ops/s
BoonClassicEagerNoLazyParse.webxml    thrpt  16         6    1   401126.575    28331.138    ops/s
JacksonASTBenchmark.webxml            thrpt  16         6    1   233730.506    17868.136    ops/s
MainJacksonObjectBenchmark.webxml     thrpt  16         6    1   227287.992    21363.353    ops/s
BoonReaderSource.webxml               thrpt  16         6    1   216429.247    22538.238    ops/s
BoonAsciiBenchMark.webxml             thrpt  16         6    1   210416.450    10610.062    ops/s
BoonUTF8BenchMark.webxml              thrpt  16         6    1   199869.811     8742.968    ops/s
GSONBenchmark.webxml                  thrpt  16         6    1   168144.639     5311.387    ops/s

MainBoonBenchmark is the index overlay parser it comes in first.
BoonClassicEagerNoLazyParse (the original boon parser) comes in second.

BoonClassicEagerNoLazyParse does not use index overlay so all those comments are just off. You can see for this benchmark index overlay is only about an 11% improvement.

If I include full chop and chop, Jackson will come in fourth.
But then I would have to explain what chop and full chop mean, and then I would have to refer you to the article on InfoQ on index overlay and explain when it makes sense and what use cases you can't use it, and then introduce full chop and chop. :)

www.infoq.com/articles/HIgh-Performance-Parsers...

Jackson used to come after BoonReaderSource but Tata has been busy so now Jackson is only twice as slow BoonClassicEagerNoLazyParse. So there goes that myth about Index overlay and Boon not doing anything and Boon in only able to handle string and not InputStreams.

LET ME REPEAT THAT: Boon using InputStream, and not using index overlay is faster than Jackson.

GSON comes in last place. I remember when I started GSON would often beat Jackson.

Jackson has improved but even the improved Jackson is twice as slow as Boon for parsing with no index overlay.

RE: Input source. Most commonly cited tests start with Java Strings. Strings are rarely used as input source, because they are JVM constructs -- all external input comes as byte streams.

The benchmark linked which has not changed since this was published and for quite some time covers InputStream, string, byte[], etc. It is in the first paragraph. Next time I will use a blink tag. It did not cover not using index overlay because in only rare cases does index overlay not make sense as a viable alternative to full parse and index overlay is actually better for POJO serialization and REST. SO OF COURSE I included it in the benchmark. To think otherwise is just wrong. But there are cases where index overlay does not make sense (thus chop, full chop, and the full parse version), but that is very nuanced discussion, and an argument hard to make when there is some clear FUD. So instead of muddying the water and defending index overlay, let just keep at this, Boon does not need index overlay to beat Jackson at JSON parsing. PERIOD!

. So when user uses, say, JAX-RS style REST handling, where all JSON data gets bound to a POJO, from an InputStream; and reverse direction goes from another POJO into OutputStream, performance experienced is very different from what a benchmark would suggest.

Actually Boon was designed for exactly the REST / Websocket use case. And it was designed for POJO serialization. And you can see in the benchmarks cited that Boon does better than Jackson at POJO serialization in most cases, and in the article by Andrey (Groovy in that case which has some Boon DNA), and in the Gatling Benchmark and its POJO performance was confirmed by Julien Ponge (the author of Golo julien.ponge.org/blog/revisiting-a-json-benchmark/). Boon used to be consistently twice as fast as Jackson at POJO object serialization but Boon added some features which made it slower and Jackson got faster so now Boon usually wins by 30%, but not always. And once you are doing POJO mapping the index overlay vs non-overlay point is completely moot.

Let's go bigger. webxml, which is from the JSON.org examples is small, let's use a big file.. The catalog which is 170K.

Benchmark                                                     Mode Thr     Count  Sec         Mean   Mean error    Units
i.g.j.inputStream.MainBoonBenchmark.citmCatalog              thrpt  16         6    1     1061.592      106.462    ops/s
i.g.j.inputStream.BoonClassicEagerNoLazyParse.citmCatalog    thrpt  16         6    1      979.794       51.968    ops/s
i.g.j.inputStream.BoonReaderSource.citmCatalog               thrpt  16         6    1      681.072       37.804    ops/s
i.g.j.inputStream.JacksonASTBenchmark.citmCatalog            thrpt  16         6    1      554.181       26.974    ops/s
i.g.j.inputStream.GSONBenchmark.citmCatalog                  thrpt  16         6    1      538.389       43.384    ops/s
i.g.j.inputStream.MainJacksonObjectBenchmark.citmCatalog     thrpt  16         6    1      486.275       70.753    ops/s

So again, the claim was that Boon can't handle things fast unless it uses Strings and unless it uses index overlay is just plain wrong.

This is from an inputstream. Jackson is 4th. Boon index overlay beats. Boon full parser, non lazy version beats it. It beats by a wide margin. When Jackson does beat Boon which happens in some uses cases the margin is usually very small. When Boon wins which happens in most of the test cases, it wins by a wide margin with and without index overlay.

Since no one asked me why its Boon is faster or what it does, I wont say other than I have been talking about it on my blog for the last six months, and I'd like to work on an article for InfoQ about it with Jakob Jenkov and/or Stephane Landelle at some point.

Jackson can be faster than Boon. It probably will be. But today. Today. It is not.

Jackson and Boon have different philosophies.

Jackson is more mature. Jackson is more stable. Jackson integrates with more frameworks. Jackson has different and possibly more features. There are use cases where Jackson will be faster. There are many use cases where JSON parser speed will not matter. Jackson probably has less bugs since it has had more eyeballs for more years.

That said... Boon is faster. Today.

InfoQ article on Groovy JSON and Boon.

"Groovy 2.3 will ship with one of the fastest JSON parsers on the JVM, according to Rick Hightower, the ubiquitous consultant and author. In his article on JSON serialization benchmarks, Hightower claims that Groovy's JSON support and the Boon JSON parser are up to 3 to 5 times faster than the current Jackson parser at parsing JSON from String and char arrays, and 2 to 4 times faster at parsing byte arrays."

http://www.infoq.com/news/2014/04/groovy-2.3-json

Computer Club for 4th and 5th graders: Year 2

Computer Club for 4th and 5th graders: 

Preparing our youth for tomorrow should include some basic instruction in computer programming.

In our day and age, knowledge of how a computer works and the logic of programming is increasingly a basic literacy. In many countries, computer programing and computer science is taught in grade school. We live in the Silicon Valley area, and for some reason it has not reached us yet. Not everyone needs to be a programmer, but every child can benefit from learning about programming, but talking about it is not enough so we decided to act.

We spoke about starting a computer club for some time. Then one day the principal said “go ahead”. But how do we organize it? “Don’t worry about it. Invite parents and students and suggest some things and see what they say”. And we did. And then our elementary school had a computer club. It was almost as simple as that.

At first it was just Ms. Young C, and me. We did not know what to expect. The first day of the club for the first year, Ms. Young C and I were sitting in the computer lab waiting for the kids to show up. I told Young that maybe 8 or 12 kids would show up. She agreed. Wow were we wrong!

Over 35 kids showed up that day and I think we peaked around 40 kids that year. This year, our second year, there was just no room (peak attendance was close to 120, and we have at least 80+ regulars). We were filling up the library and lab with computers and Ms Young C and other parent volunteers split up the club into three clubs. I think over 150 kids have been exposed to computer programming since we started, and to be honest what the kids can do far exceeds my wildest expectation. 

The kids are excited about the club and the participation is very high. We have many parent volunteers and this could not exist without them. We also have Ms. Young C who manages the lab and volunteers her time for this effort. She is the cornerstone of the club and the main coordinator. Julie B, the principal of our elementary school, has encouraged and supported our efforts. I wish I had a full list of all of the parents and individuals who were involved because the list would be very long. One of the great things about our school is the staff and parents. Our school is a community and I am happy to be part of it. 

The Computer Club is much better this year than it was last. We learned a lot. The kids all finished a computer programming book. We use Scratch from MIT which was designed to teach children computer programming. They all wrote many programs. They all learned the basics of computer programming. Next year we plan on introducing harder concepts for the 5th graders who have been exposed this year. Next year the fifth graders will all work with Python or JavaScript. Many kids in the club have worked with Scratch, Python and JavaScript. Next year all 5th graders will.

Not every volunteer agrees exactly about the right way to manage the club. There are many opinions, but the important thing is that they are there, and the kids are getting exposure to computer programming and they get to socialize and work with other kids. We try to make computer programming fun and cool, and provide an environment where kids can experiment and learn. Everyone goes at their own pace, and we just want to provide a place were they can get exposure, and kids can socialize with other kids who are interested in computer programming and computer science. Also the kids are learning to work in teams, and the kids often teach other kids how to do things like upload music or images or video or how an “if block” or “for ever block” works. It is awesome to hear kids helping other kids. The best thing for me to hear is one kid ask “Hey how did you do that?” And see one child mentor another child. This is the environment we want to foster and it looks like we are having some success. I’d like to see some of the ways kids teach each other tricks for the XBox or minecraft get changed to kids teaching each other to program. There are many kids who study and program outside of the club. Our main goal is to start the fire, and see where it takes them. 

In our day and age, knowledge of how a computer works and the logic of programming is increasingly a basic literacy. Not everyone needs to be a programmer, but every child can benefit from learning a bit more about how a computer works since our lives are surrounded with computing devices. Be it smart phone, smart TVs, tablets, laptops, desktops, and more, they are surrounded with devices that are more powerful than supercomputers from a few decades ago. It is no longer an oddity but a commonality of our age. Preparing our youth for tomorrow should include some basic instruction in computer programming.

Skills like computer programming can engage students in math and science who would not usually be interested in academics.To write better programs, you need to learn new tricks, and the new tricks are found in algebra, trig, statistics, etc. books.

Programming is a real world skill that can inculcate some kids with an appreciation of logic, math and science who might not get it through traditional means. Learning programming augments and encourages academics by allowing kids to see and build real world application of math and science that they can interact with.

Not every child has a love for academics, I speak from experience when I say my love of education started with computing and electronics. Building circuits and programs, and then seeing the equations results in a working product that I could use and interact with inspired me. It awoke my desire to learn more. I believe that computer programming can get children who might not be academically inclined to develop an interests in Math and science. I know that it was for me. My first experience with computer programming was when I was ten years old at the YMCA in Owen Hills Maryland. I wanted to share my love of computer programming with kids that were my age when I was introduced to it. These kids are a lot sharper than I was at that age and are much more familiar with computers than I was at that age.

How to add Java JSON support without having 200 DTO objects or more annotations than code

If you have 75 annotations in a Java class and 25 more in the REST handler to do JSON than you are doing it wrong. If you have 500 DTO objects to manage every use case or every view then you are using JSON wrong.

Boon was invented after looking at what a SaaS company needed, and seeing what frameworks they were currently using. Boon was the reaction to an idea of let's have 500 DTOs to match each use case or view.

(Find this in the Boon wiki here. Part 1 is here Build your own JSON adventure with Boon.) 

I could not handle writing 500 DTO objects for simple things, and I was sick of there being more annotations in my code then code, and class loaders issues, and , and … so Boon.

Let's say you have a large JSON stream / file / REST POST / Websocket call that looks like this (pretend what you see is large):

Sample JSON of teams

{

    "teamInfo" : {
        "teamRoster": {
            "teamNames": ["duck", "chickens", "penguins"]
        }

    }


} 

Sometimes you want to create a team info (TeamInfo). Sometimes you want just the list or Set. Sometimes you want just the team roster (TeamRoster). Different use cases use different parts of the JSON stream. Perhaps the JSON stream even gets put onto an event bus and many end points grab it and use different parts of the stream.

It is JSON so you should be able to add properties without breaking older event bus listeners.

Sample classes (DTO or domain objects).

    public static class TeamRoster {
        Set<String> teamNames;

        @Override
        public String toString() {
            return "TeamRoster{" +
                    "teamNames=" + teamNames +
                    '}';
        }
    }

    public static class TeamInfo {

        TeamRoster teamRoster;

        @Override
        public String toString() {
            return "TeamInfo{" +
                    "teamRoster=" + teamRoster +
                    '}';
        }
    }

In this example we will read the JSON with Boon from the classpath or file system withjsonResource (full example below).

        /* Using Boon style (easy) 2 parser. */
        jsonObject = Boon.jsonResource(path);

Now if we just want the team info part of the JSON we do this:

        /* Using Boon path. */
        puts ("teamInfo", atIndex(jsonObject, "teamInfo"));

(puts is like System.out.println but better)

teamInfo {teamRoster={teamNames=[duck, chickens, penguins]}} 

atIndex returns the teamInfo map.

You use atIndex to get just the part you care about. We can get the teamInfo, the teamRoster or the teamNames and different use cases (or different listeners on the bus might care about different things) so they can pick and choose what they want and ignore the rest. This is JSON. It is meant to be flexible.

Grabbing just the teamInfo.teamRoster

        puts("Team Roster", atIndex(jsonObject, "teamInfo.teamRoster"));

Grabbing just the teamInfo.teamRoster and teamInfo.teamRoster.teamNames

        puts("Team Roster", atIndex(jsonObject, "teamInfo.teamRoster"));
        puts("Team Names", atIndex(jsonObject, "teamInfo.teamRoster.teamNames"));

output

Team Roster {teamNames=[duck, chickens, penguins]} 
Team Names [duck, chickens, penguins] 

How do we convert team names into a Set?

Easy.

        List<String> teamNames = (List<String>) atIndex(jsonObject, "teamInfo.teamRoster.teamNames");

        puts("Team Names", teamNames);

        Set<String> teamNameSet = set(teamNames);

        puts ("Converted to a set", teamNameSet);

output

Team Names [duck, chickens, penguins] 
Converted to a set [duck, chickens, penguins] 

How do we convert a specific path into the TeamInfo domain object without converting the entire JSON stream?

Getting just the TeamInfo from JSON

        TeamInfo teamInfo = fromMap((Map<String, Object>) atIndex(jsonObject, "teamInfo"), TeamInfo.class);
        puts(teamInfo);

How do we convert a specific path into the TeamRoster domain object without converting the entire JSON stream?

Getting just the TeamRoster from JSON

        TeamRoster teamRoster = fromMap((Map<String, Object>) atIndex(jsonObject, "teamInfo.teamRoster"), TeamRoster.class);

Full example

/*
 */

package com.examples;

import org.boon.Boon;
import org.boon.IO;
import org.boon.json.JsonFactory;
import org.boon.json.JsonParserAndMapper;
import org.boon.json.JsonParserFactory;
import org.boon.json.ObjectMapper;

import java.io.File;
import java.util.List;
import java.util.Map;
import java.util.Set;

import static org.boon.Boon.atIndex;
import static org.boon.Boon.puts;
import static org.boon.Maps.fromMap;
import static org.boon.Sets.set;

/**
 * Created by Richard on 5/6/14.
 */
public class PartialDataTreeExample {


    public static class TeamRoster {
        Set<String> teamNames;

        @Override
        public String toString() {
            return "TeamRoster{" +
                    "teamNames=" + teamNames +
                    '}';
        }
    }

    public static class TeamInfo {

        TeamRoster teamRoster;

        @Override
        public String toString() {
            return "TeamInfo{" +
                    "teamRoster=" + teamRoster +
                    '}';
        }
    }

    public static void main (String... args) {
        File file = new File(".", "src/test/resources/teams.json");
        String path = file.getAbsolutePath().toString();
        puts ("PATH", path);
        puts ("CONTENTS of PATH", IO.read(path));

        /* Jackson style interface. */
        ObjectMapper mapper = JsonFactory.create();
        Object jsonObject = mapper.readValue(file, Object.class);
        puts ("JSON Object", jsonObject);


        /* Using Boon path. */
        puts ("teamInfo", atIndex(jsonObject, "teamInfo"));
        puts("Team Roster", atIndex(jsonObject, "teamInfo.teamRoster"));
        puts("Team Names", atIndex(jsonObject, "teamInfo.teamRoster.teamNames"));


        /* Using Boon style parser (fast). */
        JsonParserAndMapper boonMapper = new JsonParserFactory().create();
        jsonObject = boonMapper.parseFile(path);


        /* Using Boon path. */
        puts ("teamInfo", atIndex(jsonObject, "teamInfo"));
        puts("Team Roster", atIndex(jsonObject, "teamInfo.teamRoster"));
        puts("Team Names", atIndex(jsonObject, "teamInfo.teamRoster.teamNames"));



        /* Using Boon style (easy) 2 parser. */
        jsonObject = Boon.jsonResource(path);


        /* Using Boon path. */
        puts ("teamInfo", atIndex(jsonObject, "teamInfo"));
        puts("Team Roster", atIndex(jsonObject, "teamInfo.teamRoster"));
        puts("Team Names", atIndex(jsonObject, "teamInfo.teamRoster.teamNames"));

        //There is also a Groovy style and a GSON style.

        List<String> teamNames = (List<String>) atIndex(jsonObject, "teamInfo.teamRoster.teamNames");

        puts("Team Names", teamNames);

        Set<String> teamNameSet = set(teamNames);

        puts ("Converted to a set", teamNameSet);


        TeamInfo teamInfo = fromMap((Map<String, Object>) atIndex(jsonObject, "teamInfo"), TeamInfo.class);
        puts(teamInfo);


        TeamRoster teamRoster = fromMap((Map<String, Object>) atIndex(jsonObject, "teamInfo.teamRoster"), TeamRoster.class);
        puts(teamRoster);

    }

}

Looking for a tutorial to get started with Boon?

Fastest Java JSON serializer/parser: Boon JSON in Five minutes!

This five minute guide will get you up to speed using Boon in no time. Parse and serialize Java objects to/fro JSON up to 5x faster! Boon is not a JSON parsing project. It is more than that, but JSON parsing is intrinsic to what Boon is all about. Boon is the fastest way to serialize and parse JSON in Java so far. It is faster at object serialization, enabling JSON expressions, JSON parsing and much more. Boon JSON is FAST! In addition it has a very easy to use, convention-based API. This getting started guide covers making JSON REST calls, JSON object serialization, using annotations, and more.

Need to do ETL with JSON, and/or want to search sort JSON or Java graphs quickly.

Indexed Searching, Querying Java objects, JSON and Maps in-memory and doing ETL like JDK 8 Streams but much faster

Java Boon - Boon Data Repo using Indexed Collections for Java Beans, JSON and Maps like JDK 8 streams but faster. Many languages have sorting, slicing, dicing capabilities for Lists and Maps, and JSON. Boon adds this to Java. Boon Data Repo is an in-memory query engine that can use high speed indexed collections for very fast in-memory queries. This brief tutorial will show you how to use Boon to query JSON, Java objects and Java maps. And also how to do ELT transforms with JSON, Java objects and Java Maps.

Java Boon filtering for Java Beans, JSON and Java Maps like JDK 8 streams but much faster

Java Boon filtering for Java Beans, JSON and Java Maps like JDK 8 streams but much faster. Many languages have support for querying objects and filtering objects easily. Java does in JDK 8, but Boon adds it as well, and it can compliment features of JDK 8 as well as work in Java 7 land. Boon adds filtering to to Java. You can filter JSON, Java, and Maps. You can also use indexed queries which are a lot faster than linear search queries that come with JDK 8.

Boon: Java path expressions for Java instances, Map/List and JSON

Many languages have path expressions. Boon adds this to Java. Boon has powerful path expressions that work with objects, lists and maps. It also works with JSON. Full examples of Java, Maps/Lists and JSON are included in this tutorial.

Just curious about what Boon is...

A Boon to Java Development (introducing Boon!)

Boon comes with helper methods that allow you to easily read files, create lists, sets, maps, concurrent maps, sorted maps, sorted sets, etc. It provides slice notation, searching, easy IO (files, http, etc.). The helper methods are safeList, list, set, sortedSet, safeSet,safeSortedSet, etc. The idea is to make Java feel more like list and maps are built-in types.

Java Collections: How to query them, sort them, easily and efficiently

What if Java collections and Java hierarchies were easily searchable? They are with Boon! What if it were easy to query a complex set of Java objects at runtime? What if there were an API that kept your object indexes (really just TreeMaps, and HashMaps) in sync.? Well then you would have Boon's data repo. This article shows how to use Boon's data repo utilities to query Java objects. Boon comes with a simple criteria API that makes working with Java collections a snap.

Boon Remix: How to Clone Collection in Java - Deep copy of ArrayList and HashSet with Boon

Boon has many utility methods to convert objects into maps, lists, JSON, etc. It also has many collection utility methods. One such utility method is deepCopy, which will deep copy a list or set. Programmer as a way to clone Collection e.g. List, Set, ArrayList, HashSet or any other implementation. Or they write a lot of boiler plate code and spend a lot of time doing things to copy lists and sets. With Boon you can do deep copies without messing around with clone.

Boon: Converting Java from Lists, Maps, and JSON and other boons to Java development

Boon gets some inspiration from Groovy and Python when dealing with maps, lists and JSON. It addition to adding slice notation and such, which I covered before. Boon has first class support for converting between Maps, List, JSON and Java objects. Boon gets some inspiration from Ruby, Groovy, Python and Perl and aims to make Java programming just a bit more fun.

Thoughts

Thoughts? Write me at richard high tower AT g mail dot c-o-m (Rick Hightower).

Further Reading:

If you are new to boon start here:

• Java Boon Byte Buffer Builder
• Java Boon Slice Notation
• Java Boon Slice's work with TreeSets
• Java Boon Description
• More...
• Boon Home
• Boon Source
• Introducing Boon October 2013
• Java Slice Notation
• What if Java collections were easy to search and sort?
• Boon HTTP utils
• Boon Java JSON parser Benchmarks or hell yeah JSON parsing is damn fast!
• Boon JSON parser is really damn fast! Part II
• Boon JSON parser Round III now just not fast as but much faster than other Java JSON parsers
• Boon World's fastest Java JSON parser Round IV from fast to blazing to rocket fuel aka Braggers going to brag
• Boon gets adopted by JSON Path as the default Java JSON parser
• Boon graphics showing just how fast Boon JSON parsing is - about 50% to 200% faster than the graphs shown here now so wicked fast became wickeder - just got sick of making graphics
• 10 minute guide to Boon JSON parsing after I added @JsonIgnore, @JsonProperty, @JsonView, @Exposes, etc.
• Hightower speaks to the master of Java JSON parsing, the king of speed The COW TOWN CODER!
• Boon provides easy Java objects from lists, from maps and from JSON.