"I am constructing an array of bytes in java and I don't know how long the array will be."

"I want some tool like Java's StringBuffer that you can just call .append(byte b) or .append(byte[] buf) and have it buffer all my bytes and return to me a byte array when I'm done. Is there a class that does for bytes what StringBuffer does for Strings? It does not look like the ByteBuffer class is what I'm looking for"

Anyone have a good solution?

Asks Bob

Why yes bob! I do!

I wrote one that is really easy to use and avoids a lot of byte array buffer copying. The API has one method! (ok almost!)

It has one method called add that works like append of StringBuilder.

You can add strings, bytes, byte, long, int, double, float, short, and chars to this ByteBuilder thing.

The API is easy to use and somewhat fail safe. It does not allow you to copy the buffer around and does not promote having two readers of the same buffer. If you want to read from the byte[] array. Boon (framework it is part of) has tons of helper methods for reading from and even searching a byte array. 

This class has a bounds check mode and a I KNOW WHAT I AM DOING MODE with no bounds checking. It is not as dangerous as it sounds. 

The bounds check mode auto-grows the buffer so there is no hassle just like StringBuilder.

https://github.com/RichardHightower/boon/wiki/Auto-Growable-Byte-Buffer-like-a-ByteBuilder

Here is a complete step by step guide on how to use it. Boon and this class are on github.

Java Boon - Auto Growable Byte Buffer like a ByteBuilder

Have you ever wanted an easy to use buffer array that grow automatically and/or you can give it a fix size and just add stuff to it? I have. I wrote one too.

Look.. I can write strings to it (it converts them to UTF-8).

    ByteBuf buf = new ByteBuf();
    buf.add(bytes("0123456789\n"));
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456END\n");

Then later I can read the String out of the buffer:

    String out = new String(buf.readAndReset(), 0, buf.len());
    assertEquals(66, buf.len());
    assertTrue(out.endsWith("END\n"));

ByteBuf is a buffer of bytes (its state is in a byte[]). There is also a similar class called CharBuf (reduced copy buffer version of StringBuilder).

You never have to set the size of the array. ByteBuf will auto-grow as needed in an efficient manner.

If you know exactly how large your data is going to be then you can save some bounds checking by using createExact.

    ByteBuf buf = ByteBuf.createExact(66);
    buf.add(bytes("0123456789\n"));
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456END\n");
    assertEquals(66, buf.len());

If I use create exact, then I am saying... hey.. I know exactly how big it can grow to and it will never go over this number and if it does...you can hit me over the head with a sack of rocks!

The following hits you over the head with a sack of rocks! THROWS AN EXCEPTION!!!!

    ByteBuf buf = ByteBuf.createExact(22);
    buf.add(bytes("0123456789\n"));
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456789\n");
    buf.add("0123456END\n");

ByteBuf works with doubles.

    ByteBuf buf = ByteBuf.createExact(8);

    //add the double
    buf.add(10.0000000000001);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;

    worked |= idxDouble(bytes, 0) == 10.0000000000001 || die("Double worked");

ByteBuf works with float.

    ByteBuf buf = ByteBuf.createExact(8);

    //add the float
    buf.add(10.001f);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;

    worked |= buf.len() == 4 || die("Float worked");


    //read the float
    float flt = idxFloat(bytes, 0);

    worked |= flt == 10.001f || die("Float worked");

Bo knows Football and Baseball. ByteBuf knows floats and ints!

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the int to the array
    buf.add(99);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the int back
    int value = idxInt(bytes, 0);

    worked |= buf.len() == 4 || die("Int worked length = 4");
    worked |= value == 99 || die("Int worked value was 99");

ByteBuf works with char. My teenager drives a car.

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the char to the array
    buf.add('c');

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the char back
    int value = idxChar(bytes, 0);

    worked |= buf.len() == 2 || die("char worked length = 4");
    worked |= value == 'c' || die("char worked value was 'c'");

ByteBuf works with short. I am a bit short.

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the short to the array
    buf.add((short)77);

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the short back
    int value = idxShort(bytes, 0);

    worked |= buf.len() == 2 || die("short worked length = 2");
    worked |= value == 77 || die("short worked value was 77");

ByteBuf even works with bytes. But it does not bite.

    ByteBuf buf = ByteBuf.createExact(8);

    //Add the byte to the array
    buf.add( (byte)33 );

    byte[] bytes = buf.readAndReset();
    boolean worked = true;


    //Read the byte back
    int value = idx(bytes, 0);

    worked |= buf.len() == 1 || die("byte worked length = 1");
    worked |= value == 33 || die("byte worked value was 33");

You can add all sorts of primitives to your byte array with ByteBuf.

    boolean worked = true;
    ByteBuf buf = ByteBuf.create(1);

    //Add the various to the array
    buf.add( (byte)  1 );
    buf.add( (short) 2 );
    buf.add( (char)  3 );
    buf.add(         4 );
    buf.add( (float) 5 );
    buf.add( (long)  6 );
    buf.add( (double)7 );

    worked |= buf.len() == 29 || die("length = 29");


    byte[] bytes = buf.readAndReset();

    byte myByte;
    short myShort;
    char myChar;
    int myInt;
    float myFloat;
    long myLong;
    double myDouble;

Now we just verify that we can read everything back.

    myByte    =   idx       ( bytes, 0 );
    myShort   =   idxShort  ( bytes, 1 );
    myChar    =   idxChar   ( bytes, 3 );
    myInt     =   idxInt    ( bytes, 5 );
    myFloat   =   idxFloat  ( bytes, 9 );
    myLong   =    idxLong   ( bytes, 13 );
    myDouble  =   idxDouble ( bytes, 21 );

    worked |= myByte   == 1 || die("value was 1");
    worked |= myShort  == 2 || die("value was 2");
    worked |= myChar   == 3 || die("value was 3");
    worked |= myInt    == 4 || die("value was 4");
    worked |= myFloat  == 5 || die("value was 5");
    worked |= myLong   == 6 || die("value was 6");
    worked |= myDouble == 7 || die("value was 7");

Once you call

 byte[] bytes = buf.readAndReset() 

then you are saying that you are done with the ByteBuffer!

Once you ask for the bytes, ByteBuf becomes useless as it sets the internal byte array to nothing.

When you call readAndReset, it is giving you its buffer. It says... here is my internal state, you can have it, but I am going to set it to null so nobody else uses it.

It is ok. Just create another if you are sure only one instance at a time is using the buffer (byte []).

You can even use the buffer you were just using as in

ByteBuf buf2 = new ByteBuf.create(bytes); 

This is because no buffer gets copied. ByteBuf writes to the buffer you give it. If you want another copy to be given to ByteBuf then do this:

ByteBuf buf2 = new ByteBuf.create( copy(bytes) ); 

This is boon after all. :)

Come check out boon. You get the above class and idx, and idxInt and idxLong for free!

https://github.com/RichardHightower/boon/