Posts
I found a lot of "advice" on the Internet about File I/O and Java. My recommendation is create your own benchmarks. There is some advice out there that is just plain bad. There is also this misconception that InputStream/OutputStream is slow and that you have to use ByteBuffer to get stuff done. I was able to write faster and read faster with a plain old FileInputStream and FileOutputStream.
Martin's advice was spot on as always, and his article is fairly up to date. The below blog post is based on some ideas on the fastest way to do I/O by Martin Thompson of LMAX / Mechanical Sympathy fame. The blog post presents 2 or 3 ways that are faster using plain Java Streams not NIO.
Note: I only need to test writing speeds of files that are 10MB to 100MB, but I threw in 1GB just for kicks. Enjoy!
Editor Note: There is a lot more detail after the code listing. Also we have tested this out on EC2 4x extra large with high I/O, brand new high end Windows laptop with hybrid SSD/HD, my trusty MacBook pro with SSD, and EC2 Extra Large, and soon some high speed server with a SCSI fibre array bad-a$s setup. The approach I suggest is always the fastest except on the EC2 Extra Large which has some tinker toy ESB system. The work around for that was to use a RAM disk and then rysync the files off of the RAM disk. Base EC2 ESB is super slow. However the 4x extra large with high I/O and maxed out IOPs is wicked fast. (Check out a follow up conversation on this topic at https://github.com/RichardHightower/boon/wiki/Auto-Growable-Byte-Buffer-like-a-ByteBuilder. There is a full blown example using JDK 7 TransferQueue of a system that can sip heap memory and burn a hole in your disk. :) )
I extended some of his ideas and tried out a few more I/O options which were faster on my 2011 17" Mac Pro with an SSD. Obviously, the results will vary with a spindle disk. Next I am going to try it on Amazon EC2. I'll let you know how it goes.
NIO did not win once, and old school FileInputStream does quite well thank you very much. :)
I am sure one could dream up and tweak an NIO example to do better. I went with the lowest tech first: InputStream and as soon as I got that running faster then all of the Martin NIO example except for MemoryMappedNIO for really large files (larger than I planned)... I think I am done for now, but please go forward and come up with some other options, and for heaven's sake send me a link!
Actually NNIO did win, I used Files.readAll from JDK 1.7. It is wicked fast for reads and writes. A good option if the file fits nicely in memory. :)
I only need to test writing speeds of files that are 10MB to 100MB, but I threw in 1GB just for kicks.
The below is based on an article on the fastest way to do I/O by Martin Thompson of LMAX / Mechanical Sympathy fame. I came up with 2 or 3 ways that are faster using plain Java Streams not I/O.
ABOUT TO DO 1,000,000,000 (1GB) file I/O test
1 AVG RandomAccessFile JDK1.0 write=251,129,154 read=1,376,770,590 bytes/sec
2 AVG BufferedStreamFile JDK 1.0 write=202,081,171 read=236,796,226 bytes/sec
3 AVG BufferedChannelFile NIO write=240,476,292 read=798,158,434 bytes/sec
4 AVG MemoryMappedFile NIO write=365,499,370 read=380,522,482 bytes/sec
5 AVG StreamFile1xPage JDK 1.0 write=253,941,521 read=152,793,700 bytes/sec
6 AVG StreamFile2xPage JDK1.0 write=265,185,718 read=1,679,141,166 bytes/sec
7 AVG StreamFile10xPage JDK1.0 write=262,242,133 read=1,918,268,820
(JDK 7 version did not run out of memory error) (not going to write files this big so not too worried about failure, and the solution is obvious).
ABOUT TO DO 100,000,000 (100 MB) test I/O test
1 AVG RandomAccessFile JDK1.0 write=281,381,681 read=1,291,839,752 bytes/sec
2 AVG BufferedStreamFile JDK 1.0 write=166,997,066 read=228,967,826 bytes/sec
3 AVG BufferedChannelFile NIO write=260,330,902 read=861,436,084 bytes/sec
4 AVG MemoryMappedFile NIO write=354,568,658 read=368,841,029 bytes/sec
5 AVG StreamFile1xPage JDK 1.0 write=317,918,605 read=165,774,080 bytes/sec
6 AVG StreamFile2xPage JDK1.0 write=327,131,359 read=1,566,124,100 bytes/sec
7 AVG StreamFile10xPage JDK1.0 write=444,304,431 read=1,794,612,794 bytes/sec
8 AVG Files readAll/writeAll JDK1.7 write=496,834,996 read=551,536,072 bytes/sec
That seems damn close to the metal on writes. Java has gotten a lot better at IO.
NIO came out in JDK 1.4 I think.
Please note that run 5, run 6, run 7 and run 8 are all stuff I dreamt up based on past experience with high speed I/O not in Martin's original examples.
The new stuff (low tech) were all faster than the approaches in the example article especially for smaller files (10 MB, 20 MB, 50 MB, and 100 MB) on an SSD drive. (So I have a particular application in mind... and the benchmarks are geared towards that so this is no knock on Martin who is AWESOME! The low tech solutions even do well with larger files too)
Score card for 1GB
Fastest Read -- Not in original article
7 AVG StreamFile10xPage JDK1.0 write=444,304,431 read=1,794,612,794 bytes/sec
Fastest Write -- Not in original article
8 AVG Files readAll/writeAll JDK1.7 write=496,834,996 read=551,536,072 bytes/sec
2nd Fastest Read -- Not in original article
6 AVG StreamFile2xPage JDK1.0 write=327,131,359 read=1,566,124,100 bytes/sec
2nd Fastest Write -- Not in original article
7 AVG StreamFile10xPage JDK1.0 write=444,304,431 read=1,794,612,794 bytes/sec
3rd Fastest Read -- In original article
1 AVG RandomAccessFile JDK1.0 write=281,381,681 read=1,291,839,752 bytes/sec
3rd Fastest Write -- In original article
4 AVG MemoryMappedFile NIO write=354,568,658 read=368,841,029 bytes/sec
The one of the "new" techniques came in fourth place as well ahead of the two other NIO examples in the original article.
Not bad? :)
Still need to test on EC2, but it should be fine. We have many options, and all are fast enough.
Amazon ESB can be sharded btw..
Ok... 10 MB file speeds
ABOUT TO 10,000,000 (10 MB)
AVG RandomAccessFile JDK1.0 write=390,394,088 read=411,165,500 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=35,428,657 read=36,002,925 bytes/sec
AVG BufferedChannelFile NIO write=203,252,553 read=436,532,590 bytes/sec
AVG MemoryMappedFile NIO write=285,958,112 read=355,709,274 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=295,261,912 read=161,472,782 bytes/sec
AVG StreamFile2xPage JDK1.0 write=354,732,620 read=1,321,428,571 bytes/sec
*2nd Fastest W/R (new)
AVG StreamFile10xPage JDK1.0 write=413,245,757 read=1,515,151,514 bytes/sec
**Fastest Write/Read (new)
AVG Files readAll/writeAll JDK1.7 write=338,987,346 read=529,365,079 bytes/sec
**3rd Fastest Write/Read (new)
20 MB
AVG RandomAccessFile JDK1.0 write=341,171,775 read=654,958,183 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=68,793,838 read=86,611,977 bytes/sec
AVG BufferedChannelFile NIO write=239,788,696 read=733,662,013 bytes/sec
AVG MemoryMappedFile NIO write=341,287,878 read=361,687,507 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=305,942,219 read=153,345,842 bytes/sec
AVG StreamFile2xPage JDK1.0 write=365,374,902 read=1,492,577,597 bytes/sec
** new 3rd F W / 2nd F R
AVG StreamFile10xPage JDK1.0 write=431,328,320 read=1,704,545,454 bytes/sec
**new Fastest R/W
AVG Files readAll/writeAll JDK1.7 write=431,012,341 read=511,854,424 bytes/sec
**new 2nd Fastest W
50 MB
AVG RandomAccessFile JDK1.0 write=293,730,740 read=1,184,992,178 bytes/sec
3 r old article
AVG BufferedStreamFile JDK 1.0 write=172,087,116 read=173,340,836 bytes/sec
AVG BufferedChannelFile NIO write=256,096,023 read=869,295,579 bytes/sec
AVG MemoryMappedFile NIO write=349,081,125 read=370,451,389 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=309,772,708 read=164,809,513 bytes/sec
AVG StreamFile2xPage JDK1.0 write=371,394,764 read=1,561,264,893 bytes/sec
**3 w new 2 r new
AVG StreamFile10xPage JDK1.0 write=438,798,861 read=1,780,753,967 bytes/sec
**2 W new 1 r new
AVG Files readAll/writeAll JDK1.7 write=478,100,664 read=556,127,414 bytes/sec
**1 W new
This code is largely based on Martin's example. I added four new R/W techniques. (Pretty low tech, but fast.)
EDITOR NOTE: More details and benchmarks after this too damn long code listing.......public final class TestSequentialIoPerf {public static final int FILE_SIZE = 1_000_000_000;public static final int BUF_SIZE = 5_000;public static final String FILE_NAME = "test.dat";public static final byte[] BLANK_PAGE = new byte[ FILE_SIZE / 10 ];public static void main ( final String... args ) throws Exception {deleteFile ( FILE_NAME );out.printf ( "ABOUT TO PREALLOCATE %,d\n", FILE_SIZE );preallocateTestFile ( FILE_NAME );out.printf ( "ABOUT TO WRITE %,d\n", FILE_SIZE );for ( final PerfTestCase testCase : testCases ) {long bytesReadPerSecSum = 0;long bytesWrittenPerSecSum = 0;int numRuns = 5;for ( int i = 0; i < numRuns; i++ ) {System.gc ();long writeDurationMs = testCase.test ( PerfTestCase.Type.WRITE,FILE_NAME );System.gc ();long readDurationMs = testCase.test ( PerfTestCase.Type.READ,FILE_NAME );long bytesReadPerSecond = ( FILE_SIZE * 1000L ) / readDurationMs;long bytesWrittenPerSecond = ( FILE_SIZE * 1000L ) / writeDurationMs;bytesWrittenPerSecSum+= bytesWrittenPerSecond;bytesReadPerSecSum+= bytesReadPerSecond;}out.format ( "AVG %s\twrite=%,d\tread=%,d bytes/sec\n",testCase.getName (),(bytesWrittenPerSecSum / numRuns), (bytesReadPerSecSum / numRuns) );}out.printf ( "ABOUT TO DELETE %,d", FILE_SIZE );deleteFile ( FILE_NAME );}private static void preallocateTestFile ( final String fileName )throws Exception {RandomAccessFile file = new RandomAccessFile ( fileName, "rw" );for ( long i = 0; i < FILE_SIZE; i += BLANK_PAGE.length ) {file.write ( BLANK_PAGE, 0, BLANK_PAGE.length );}file.close ();}private static void deleteFile ( final String testFileName ) throws Exception {File file = new File ( testFileName );if ( !file.delete () ) {out.println ( "Failed to delete test file=" + testFileName );out.println ( "Windows does not allow mapped files to be deleted." );}}public abstract static class PerfTestCase {public enum Type {READ, WRITE}private final String name;private int checkSum;public PerfTestCase ( final String name ) {this.name = name;}public String getName () {return name;}public long test ( final Type type, final String fileName ) {long start = System.currentTimeMillis ();try {switch ( type ) {case WRITE: {checkSum = testWrite ( fileName );break;}case READ: {final int checkSum = testRead ( fileName );if ( checkSum != this.checkSum ) {final String msg = getName () +" expected=" + this.checkSum +" got=" + checkSum;throw new IllegalStateException ( msg );}break;}}} catch ( Exception ex ) {ex.printStackTrace ();}return System.currentTimeMillis () - start;}public abstract int testWrite ( final String fileName ) throws Exception;public abstract int testRead ( final String fileName ) throws Exception;}private static PerfTestCase[] testCases ={new PerfTestCase ( "RandomAccessFile JDK1.0 " ) {public int testWrite ( final String fileName ) throws Exception {RandomAccessFile file = new RandomAccessFile ( fileName, "rw" );final byte[] buffer = new byte[ BUF_SIZE ];int pos = 0;int checkSum = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {byte b = ( byte ) i;checkSum += b;buffer[ pos++ ] = b;if ( BUF_SIZE == pos ) {file.write ( buffer, 0, BUF_SIZE );pos = 0;}}file.close ();return checkSum;}public int testRead ( final String fileName ) throws Exception {RandomAccessFile file = new RandomAccessFile ( fileName, "r" );final byte[] buffer = new byte[ BUF_SIZE ];int checkSum = 0;int bytesRead;while ( -1 != ( bytesRead = file.read ( buffer ) ) ) {for ( int i = 0; i < bytesRead; i++ ) {checkSum += buffer[ i ];}}file.close ();return checkSum;}},new PerfTestCase ( "BufferedStreamFile JDK 1.0 " ) {public int testWrite ( final String fileName ) throws Exception {int checkSum = 0;OutputStream out =new BufferedOutputStream ( new FileOutputStream ( fileName ) );for ( long i = 0; i < FILE_SIZE; i++ ) {byte b = ( byte ) i;checkSum += b;out.write ( b );}out.close ();return checkSum;}public int testRead ( final String fileName ) throws Exception {int checkSum = 0;InputStream in =new BufferedInputStream ( new FileInputStream ( fileName ) );int b;while ( -1 != ( b = in.read () ) ) {checkSum += ( byte ) b;}in.close ();return checkSum;}},new PerfTestCase ( "BufferedChannelFile NIO " ) {public int testWrite ( final String fileName ) throws Exception {FileChannel channel =new RandomAccessFile ( fileName, "rw" ).getChannel ();ByteBuffer buffer = ByteBuffer.allocate ( BUF_SIZE );int checkSum = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {byte b = ( byte ) i;checkSum += b;buffer.put ( b );if ( !buffer.hasRemaining () ) {buffer.flip ();channel.write ( buffer );buffer.clear ();}}channel.close ();return checkSum;}public int testRead ( final String fileName ) throws Exception {FileChannel channel =new RandomAccessFile ( fileName, "rw" ).getChannel ();ByteBuffer buffer = ByteBuffer.allocate ( BUF_SIZE );int checkSum = 0;while ( -1 != ( channel.read ( buffer ) ) ) {buffer.flip ();while ( buffer.hasRemaining () ) {checkSum += buffer.get ();}buffer.clear ();}return checkSum;}},new PerfTestCase ( "MemoryMappedFile NIO " ) {public int testWrite ( final String fileName ) throws Exception {FileChannel channel =new RandomAccessFile ( fileName, "rw" ).getChannel ();MappedByteBuffer buffer =channel.map ( READ_WRITE, 0,Math.min ( channel.size (), MAX_VALUE ) );int checkSum = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {if ( !buffer.hasRemaining () ) {buffer =channel.map ( READ_WRITE, i,Math.min ( channel.size () - i, MAX_VALUE ) );}byte b = ( byte ) i;checkSum += b;buffer.put ( b );}channel.close ();return checkSum;}public int testRead ( final String fileName ) throws Exception {FileChannel channel =new RandomAccessFile ( fileName, "rw" ).getChannel ();MappedByteBuffer buffer =channel.map ( READ_ONLY, 0,Math.min ( channel.size (), MAX_VALUE ) );int checkSum = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {if ( !buffer.hasRemaining () ) {buffer =channel.map ( READ_WRITE, i,Math.min ( channel.size () - i, MAX_VALUE ) );}checkSum += buffer.get ();}channel.close ();return checkSum;}},new PerfTestCase ( "StreamFile1xPage JDK 1.0 " ) {public final byte[] buffer = new byte[ BUF_SIZE ];public int testWrite ( final String fileName ) throws Exception {int checkSum = 0;OutputStream out =new FileOutputStream ( fileName ) ;int index = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {byte b = ( byte ) i;checkSum += b;buffer[index] = b;index ++;if (index == buffer.length) {index = 0;out.write ( buffer );}}out.close ();return checkSum;}public int testRead ( final String fileName ) throws Exception {int checkSum = 0;InputStream in =new BufferedInputStream ( new FileInputStream ( fileName ) );int b;while ( -1 != ( b = in.read () ) ) {checkSum += ( byte ) b;}in.close ();return checkSum;}},new PerfTestCase ( "StreamFile2xPage JDK1.0 " ) {public final byte[] buffer = new byte[ BUF_SIZE * 2 ];public int testWrite ( final String fileName ) throws Exception {int checkSum = 0;OutputStream out =new FileOutputStream ( fileName ) ;int index = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {byte b = ( byte ) i;checkSum += b;buffer[index] = b;index ++;if (index == buffer.length) {index = 0;out.write ( buffer );}}out.close ();return checkSum;}public int testRead ( final String fileName ) throws Exception {int checkSum = 0;InputStream in =new FileInputStream ( fileName );int count = buffer.length;while (count == buffer.length) {count = in.read ( buffer );for (int index = 0; index < count; index++) {checkSum += buffer[index];}}in.close ();return checkSum;}},new PerfTestCase ( "StreamFile10xPage JDK1.0 " ) {public final byte[] buffer = new byte[ BUF_SIZE * 10 ];public int testWrite ( final String fileName ) throws Exception {int checkSum = 0;OutputStream out =new FileOutputStream ( fileName ) ;int index = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {byte b = ( byte ) i;checkSum += b;buffer[index] = b;index ++;if (index == buffer.length) {index = 0;out.write ( buffer );}}out.close ();return checkSum;}public int testRead ( final String fileName ) throws Exception {int checkSum = 0;InputStream in =new FileInputStream ( fileName );int count = buffer.length;while (count == buffer.length) {count = in.read ( buffer );for (int index = 0; index < count; index++) {checkSum += buffer[index];}}in.close ();return checkSum;}},new PerfTestCase ( "Files readAll/writeAll JDK1.7" ) {public final byte[] buffer = new byte[ FILE_SIZE ];public int testWrite ( final String fileName ) throws Exception {final Path filePath = Paths.get ( fileName );int checkSum = 0;for ( long i = 0; i < FILE_SIZE; i++ ) {byte b = ( byte ) i;checkSum += b;buffer[ (int) i ] = b;}Files.write ( filePath, buffer);return checkSum;}public int testRead ( final String fileName ) throws Exception {final Path filePath = Paths.get ( fileName );final byte[] inBuffer = Files.readAllBytes ( filePath );int checkSum = 0;for (int index = 0; index < inBuffer.length; index++) {checkSum += inBuffer[index];}return checkSum;}},};}
These are temporary files so saving them long term in not an option anyway as they are going to be in EC2 instances that can be spun up and spun down. Since the entire instance will go away when you shut it down, long term storage is not a concern. Another process grabs batches of files and moves them before spin down. This is more about fast staging then long term storage. I did notice that NIO does much better with EBS on Amazon EC2 then FileInputStream.
Here is a 4GB RAM disk:
The above was in the RAM disk.
To be fair.. this set not in the RAM disk:
Here is a 4GB RAM disk:
$ sudo mount -t tmpfs -o size=4000M tmpfs /home/ubuntu/output/
If you can't win, cheat!
ABOUT TO WRITE 50,000,000 50MB
AVG RandomAccessFile JDK1.0 write=229,506,176 read=727,481,473 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=106,388,478 read=125,019,173 bytes/sec
AVG BufferedChannelFile NIO write=171,052,271 read=518,987,970 bytes/sec
AVG MemoryMappedFile NIO write=179,660,462 read=191,627,993 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=184,957,337 read=95,193,891 bytes/sec
AVG StreamFile2xPage JDK1.0 write=202,546,436 read=833,991,672 bytes/sec
AVG StreamFile10xPage JDK1.0 write=216,967,619 read=961,799,217 bytes/sec
AVG Files readAll/writeAll JDK1.7 write=303,308,166 read=360,781,716 bytes/sec
JDK 7 Files.writeAllBytes is the winner for writes and StreamFile10xPage is the winner for reads.
The above was in the RAM disk.
To be fair.. this set not in the RAM disk:
AVG RandomAccessFile JDK1.0 write=214,072,272 read=754,175,101 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=20,332,126 read=123,639,159 bytes/sec
AVG BufferedChannelFile NIO write=161,421,597 read=519,102,520 bytes/sec
AVG MemoryMappedFile NIO write=144,011,074 read=190,159,323 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=23,963,791 read=93,811,826 bytes/sec
AVG StreamFile2xPage JDK1.0 write=23,087,176 read=695,507,734 bytes/sec
AVG StreamFile10xPage JDK1.0 write=22,525,374 read=888,222,158 bytes/sec
AVG Files readAll/writeAll JDK1.7 write=25,463,255 read=350,633,094 bytes/sec
RandomAccessFile would have been my first choice / knee jerk reaction from previous experience with File I/O (Martin already wrote an example on that one or I would have). So on straight up ESB on EC2 Extra Large with optimized ESB, RandomAccessFile is the clear winner for writes, and it beats NIO. :) Going old school!
FileInputStream is a dog for writes when working with Amazon EC2 ESB. I wonder what it would be like on a dedicated spindle real HARD DISK disk. That is frightening. The StreamFile10xPage is again the winner for reads.
I am sure I am going to use a RAM disk due the ephemeral nature of the EC2 instance to begin with so let's continue.
TEST 20,000,000 20 MB file
AVG RandomAccessFile JDK1.0 write=225,447,041 read=698,412,697 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=82,297,128 read=65,690,374 bytes/sec
AVG BufferedChannelFile NIO write=153,322,931 read=414,110,711 bytes/sec
AVG MemoryMappedFile NIO write=174,978,066 read=189,009,287 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=183,973,757 read=94,909,978 bytes/sec
AVG StreamFile2xPage JDK1.0 write=200,760,042 read=795,704,874 bytes/sec
AVG StreamFile10xPage JDK1.0 write=215,810,247 read=887,770,562 bytes/sec
AVG Files readAll/writeAll JDK1.7 write=270,505,459 read=324,118,191 bytes/sec
Good rate. About 3x more than I need for this app.
WRITE 10,000,000 10 MB
AVG RandomAccessFile JDK1.0 write=224,305,364 read=695,304,695 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=41,715,867 read=35,600,588 bytes/sec
AVG BufferedChannelFile NIO write=132,271,726 read=287,344,963 bytes/sec
AVG MemoryMappedFile NIO write=151,561,651 read=187,612,513 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=184,809,555 read=94,045,693 bytes/sec
AVG StreamFile2xPage JDK1.0 write=202,091,503 read=781,818,181 bytes/sec
AVG StreamFile10xPage JDK1.0 write=215,837,868 read=905,263,157 bytes/sec
AVG Files readAll/writeAll JDK1.7 write=225,653,424 read=365,560,922 bytes/sec
Good rate, I think I will go with 20 MB files for this app StreamFile10xPage JDK1.0 for reads and writes on to a RAM disk. I will use large buffers (byte array) in queues. Memory is not a constraint and the data is replicated to two servers so not worried about a server crash either. There is no one size fits all. NIO did better than FileStream on ESB but not as good a RandomAccessFile.
Now on a real disk for a brand new Windows laptop using a real disk:
WRITE 20,000,000 20 MB (high end windows laptop)
AVG RandomAccessFile JDK1.0 write=326,188,047 read=871,784,511 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=71,130,937 read=89,240,615 bytes/sec
AVG BufferedChannelFile NIO write=227,392,598 read=586,814,218 bytes/sec
AVG MemoryMappedFile NIO write=285,603,623 read=379,854,734 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=267,267,638 read=157,346,001 bytes/sec
AVG StreamFile2xPage JDK1.0 write=340,977,639 read=1,094,771,241 bytes/sec
AVG StreamFile10xPage JDK1.0 write=403,344,568 read=1,375,361,480 bytes/sec
AVG Files readAll/writeAll JDK1.7 write=328,312,292 read=534,099,437 bytes/sec
The winner... StreamFile10xPage!
WRITE 50,000,000 50 MB (high end windows laptop)
AVG RandomAccessFile JDK1.0 write=386,160,106 read=1,002,747,404 bytes/sec
AVG BufferedStreamFile JDK 1.0 write=133,116,693 read=161,451,086 bytes/sec
AVG BufferedChannelFile NIO write=226,187,089 read=639,847,714 bytes/sec
AVG MemoryMappedFile NIO write=302,733,644 read=414,859,551 bytes/sec
AVG StreamFile1xPage JDK 1.0 write=252,394,715 read=158,530,421 bytes/sec
AVG StreamFile2xPage JDK1.0 write=295,390,959 read=1,171,746,001 bytes/sec
AVG StreamFile10xPage JDK1.0 write=371,276,283 read=1,140,140,636 bytes/sec
AVG Files readAll/writeAll JDK1.7 write=413,546,066 read=488,204,366 bytes/sec
The winner... StreamFile10xPage!
I also ran this on an EC2 4x extra large with maxed out IOPs and high-speed I/O. I don't have the exact numbers but it was close to two X the speed of my mab book pro SSD.
No comments:
Post a Comment