Rick

Rick
Rick

Tuesday, March 18, 2014

JSON ETL With Boon and Boon Data Repo, Do JSON ETL, Java ETL, and HashMap ETL form Java Part II

I added a few more example (this is from working code mind you).


Actual code working with path expression.
(This already worked)

list = employeeMapRepo.query(
                selects(
                        selectAs("firstName", "fn"),
                        selectAs("lastName", "ln"),
                        selectAs("contactInfo.phoneNumbers[0]", "ph"),

                        selectAs("salary", "pay", new Function<Integer, Float>() {
                            @Override
                            public Float apply(Integer salary) {
                                float pay = salary.floatValue() / 100;
                                return pay;
                            }
                        })
                )
        );


This turns

[{"name":"Engineering","employees":[{"id":1,"salary":100,"firstName":"Rick","lastName":"Hightower","contactInfo":{"phoneNumbers":["555-555-0000"]}}

Into 

{"fn":"Rick","ln":"Hightower","ph":"555-555-0000","pay":1.0}

Now keep in mind that this works with JSON, Java Instances and Maps. So you pick and it can transform from one to the other and back. What about munging stuff together and calling other functions to do transforms, and …. No problem.


        template = template();
        template.addFunctions(new DecryptionService());
        template.addFunctions(new Salary());



        list = employeeMapRepo.query(
                selects(
                        selectAsTemplate("fullName", "{{firstName}} {{lastName}}",
                                template),
                        selectAs("contactInfo.phoneNumbers[0]", "ph"),
                        selectAsTemplate("pay", "{{pay(salary)}}", template),
                        selectAsTemplate("id", "{{DecryptionService.decrypt(id)}}", template)

                )
        );


Notice that this uses templates to do concatenation, it uses path expressions, and calls methods from the  DecryptionService and the Salary (pay()), which are just pretend services that I made up.

Now we have:

{"fullName":"Rick Hightower", ph:"555-555-0000", pay:1.0, id:49}


Boon has easy ways to turn maps into objects so it is child's play to convert the list of maps into a list of object. Remember of the selection business works with JSON, Java instances and Java HashMaps.

Of course the above is a query so you can add a select clause to filter out objects and such. I am trying not to drown you in detail, but it is easy to manipulate the objects to/fro JSON/Maps/Java instance. Flick of the wrist….

Also if you hate handlebars, there is a JSTL syntax so what the hell... 

        template = jstl();
        template.addFunctions(new DecryptionService());
        template.addFunctions(new Salary());



        list = employeeMapRepo.query(
                selects(
                        selectAsTemplate("fullName", "${lastName},${firstName}",
                                template),
                        selectAsTemplate("ph", "${contactInfo.phoneNumbers[0]}",
                                template),
                        selectAsTemplate("pay", "${pay(salary)}", template),
                        selectAsTemplate("id", "${DecryptionService.decrypt(id)}", template)

                )
        );

Which begets

{"fullName":"Hightower,Rick", "ph":"555-555-0000", pay:1.0, id:49}


I forgot to add the ... show off the where clause....


list = employeeMapRepo.query(
                selects(
                        selectAsTemplate("fullName", "${lastName},${firstName}",
                                template),
                        selectAsTemplate("ph", "${contactInfo.phoneNumbers[0]}",
                                template),
                        selectAsTemplate("pay", "${pay(salary)}", template),
                        selectAsTemplate("id", "${DecryptionService.decrypt(id)}", template)

                ), gt("salary", 50), eq("firstName", "Rick")
        );



[{fullName=Hightower,Rick, ph=555-555-0000, pay=1.0, id=49}] 

Learn more...

See this check in...

https://github.com/RichardHightower/boon/commit/46ba1a19e7f5cb35850ee21e924349166814b102



Kafka and Cassandra support, training for AWS EC2 Cassandra 3.0 Training