Java measurements

This table shows 4 measurements - CPU Time, Elapsed Time, Memory and Code.

Each row shows those measurements for a particular Java program with a particular command-line input value N.

This table shows the current Java programs.

Program Source Code  N  CPU secs Elapsed secs Memory KB Code B
binary-trees  #2  120.200.20?603
binary-trees  #2  160.920.92158,400603
binary-trees  #2  2017.3017.33512,128603
binary-trees  #3  120.180.18?584
binary-trees  #3  160.870.87157,148584
binary-trees  #3  2016.8416.87515,100584
binary-trees-redux  #2  120.180.18?607
binary-trees-redux  #2  160.960.9670,976607
binary-trees-redux  #2  20105.81105.89166,840607
chameneos-redux  #2  60,0000.330.336761429
chameneos-redux  #2  600,0001.221.2242,0401429
chameneos-redux  #2  6,000,0009.989.9892,5881429
chameneos-redux  #3  60,0001.411.4127,4561267
chameneos-redux  #3  600,00013.9413.9545,4641267
chameneos-redux  #3  6,000,000113.69113.7258,9121267
chameneos-redux  #4  60,0001.551.5631,0681607
chameneos-redux  #4  600,00011.9011.9129,4041607
chameneos-redux  #4  6,000,000115.37115.4048,4401607
chameneos-redux   60,0000.330.354201770
chameneos-redux   600,0000.620.6227,1601770
chameneos-redux   6,000,0003.363.3731,7921770
fannkuch-redux  #2  100.670.6821,544514
fannkuch-redux  #2  116.256.2522,400514
fannkuch-redux  #2  1273.1473.1623,852514
fannkuch-redux   100.590.5925,2961282
fannkuch-redux   115.305.3024,3561282
fannkuch-redux   1269.8769.8841,3361282
fasta  #4  250,0000.220.237081507
fasta  #4  2,500,0000.750.7622,9361507
fasta  #4  25,000,0005.365.3726,1601507
fasta  #2  250,0000.280.288201240
fasta  #2  2,500,0001.111.1123,2641240
fasta  #2  25,000,0008.348.3523,9961240
fasta  #5  250,0000.270.276762457
fasta  #5  2,500,0000.770.7728,5402457
fasta  #5  25,000,0005.355.3632,9162457
fasta-redux  #3  250,0000.180.18?1443
fasta-redux  #3  2,500,0000.450.4523,9281443
fasta-redux  #3  25,000,0002.602.6125,7241443
k-nucleotide  #4  250,0001.091.09178,4961873
k-nucleotide  #4  2,500,0006.236.24206,7641873
k-nucleotide  #4  25,000,00048.7448.78180,6601873
k-nucleotide  #2  250,0001.131.1347,8961602
k-nucleotide  #2  2,500,0005.895.90163,6281602
k-nucleotide  #2  25,000,00046.8146.85462,6001602
k-nucleotide  #5  250,0001.371.38181,4242211
k-nucleotide  #5  2,500,0008.048.24207,5162211
k-nucleotide  #5  25,000,00060.5860.63181,4242211
k-nucleotide  #3  250,0001.161.1649,2361630
k-nucleotide  #3  2,500,0005.725.72162,6961630
k-nucleotide  #3  25,000,00047.0247.05466,8201630
k-nucleotide  #7  250,0001.111.1164,5721844
k-nucleotide  #7  2,500,0004.384.39320,3161844
k-nucleotide  #7  25,000,00034.7834.841,119,0841844
mandelbrot  #3  1,0000.340.34704903
mandelbrot  #3  4,0002.892.9026,464903
mandelbrot  #3  16,00042.9642.9788,556903
mandelbrot   1,0000.330.33704665
mandelbrot   4,0002.862.8723,276665
mandelbrot   16,00042.7242.7337,236665
mandelbrot  #6  1,0000.270.27424802
mandelbrot  #6  4,0001.901.9124,068802
mandelbrot  #6  16,00027.5827.5986,204802
mandelbrot  #2  1,0000.260.26680796
mandelbrot  #2  4,0001.681.6827,284796
mandelbrot  #2  16,00023.8623.8788,200796
meteor-contest  #2  2,0980.470.4729,4365177
meteor-contest   2,0980.890.8927,2564368
n-body  #2  500,0000.380.386801424
n-body  #2  5,000,0002.422.4223,4321424
n-body  #2  50,000,00022.6722.6723,8121424
pidigits  #3  2,0001.391.4046,3201826
pidigits  #3  6,0003.503.5054,6241826
pidigits  #3  10,0006.096.1062,8841826
pidigits   2,0001.511.53170,936800
pidigits   6,0006.997.00379,236800
pidigits   10,00018.3418.37379,564800
pidigits  #2  2,0000.280.30424938
pidigits  #2  6,0001.261.2727,680938
pidigits  #2  10,0003.153.1528,972938
pidigits  #4  2,0001.451.4743,5201808
pidigits  #4  6,0003.403.4049,9761808
pidigits  #4  10,0006.186.1858,4321808
regex-dna  #8  50,0000.680.7027,9641377
regex-dna  #8  500,0002.722.8136,9161377
regex-dna  #8  5,000,00020.9420.95567,2761377
regex-dna  #6  50,0000.700.7154,9361410
regex-dna  #6  500,0002.932.9498,5961410
regex-dna  #6  5,000,000Failed  1410
regex-dna  #2  50,0000.810.8234,8121534
regex-dna  #2  500,0003.803.81113,5841534
regex-dna  #2  5,000,00033.3433.36763,2721534
regex-dna  #4  50,0000.840.8541,308921
regex-dna  #4  500,0003.223.2395,276921
regex-dna  #4  5,000,00023.4623.47842,236921
regex-dna  #7  50,0000.630.6427,7281284
regex-dna  #7  500,0002.762.7637,4801284
regex-dna  #7  5,000,00023.3023.31566,6281284
regex-dna  #5  50,0000.840.8537,008929
regex-dna  #5  500,0003.253.25113,456929
regex-dna  #5  5,000,00023.1823.19766,072929
reverse-complement  #7  250,0000.360.4427,9601640
reverse-complement  #7  2,500,0000.521.1950,5721640
reverse-complement  #7  25,000,0002.018.64275,7761640
reverse-complement  #6  250,0000.210.21704745
reverse-complement  #6  2,500,0000.370.37680745
reverse-complement  #6  25,000,0001.631.63524,612745
reverse-complement  #4  250,0000.230.23424592
reverse-complement  #4  2,500,0000.460.4635,760592
reverse-complement  #4  25,000,0002.332.34485,936592
reverse-complement  #3  250,0000.230.244241661
reverse-complement  #3  2,500,0000.450.4661,1161661
reverse-complement  #3  25,000,0002.422.42307,5841661
spectral-norm   5000.350.4421,880514
spectral-norm   3,0005.245.2425,344514
spectral-norm   5,50017.0917.1025,552514
spectral-norm  #2  5000.360.36680950
spectral-norm  #2  3,0005.335.3424,320950
spectral-norm  #2  5,50017.4117.4125,568950
thread-ring  #7  500,0002.532.5431,932473
thread-ring  #7  5,000,00023.0123.0349,820473
thread-ring  #7  50,000,000220.79220.9447,644473
thread-ring  #3  500,0002.832.84365,104530
thread-ring  #3  5,000,00023.8623.88366,076530
thread-ring  #3  50,000,000228.30228.45365,860530

 Java : ubiquitous jit server virtual machine 

java version "1.8.0_25"
Java(TM) SE Runtime Environment (build 1.8.0_25-b17)
Java HotSpot(TM) 64-Bit Server VM (build 25.25-b02, mixed mode)

Home Page: Java SE at a Glance

Download: Java SE Downloads

Let's see how much, or how little, the time taken to invoke the JVM might contribute to the usual Java program times shown in the benchmarks game. Here are some additional (Intel® Q6600® quad-core) elapsed time measurements, taken after the Java programs started and before they exited.

In the first case (Cold), we simply started and measured the program 66 times; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<1; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

In the second case (Warmed), we started the program once and repeated measurements again and again and again 66 times without restarting the JVM; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<66; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

Compare these additional measurements against the usual Java program measurements shown in the benchmarks game --

"1.7.0_06" Java HotSpot(TM) 64-Bit Server VM
System.nanoTime()  1) Cold   2) Warmed   
  mean σ mean σ   usual
meteor contest   0.0118s 0.0007 0.0016s 0.0002 0.22s
fasta-redux   2.45s 0.00 2.32s 0.00 2.51s
spectral-norm   4.44s 0.02 4.20s 0.16 4.51s
pidigits   4.69s 0.09 4.44s 0.05 4.61s
fasta   5.07s 0.46 4.84s 0.02 5.13s
chameneos-redux   5.84s 0.46 5.70s 0.48 5.65s
mandelbrot   7.93s 0.23 7.99s 0.01 7.02s
k-nucleotide   8.09s 0.28  --   --  8.05s
regex-dna   8.65s 0.27  --   --  8.61s
binary-trees   10.54s 0.28 7.66s 0.16 9.08s
fannkuch-redux   16.89s 1.32 17.26s 0.10 17.38s
nbody   22.43s 0.00 22.41s 0.00 22.50s
binary-trees-redux   34.15s 0.39 33.93s 0.31 33.38s

The largest and most obvious effects of bytecode loading and dynamic optimization can be seen with the meteor-contest program which only runs for a fraction of a second.

A more sophisticated way to make JVM timings following warmup iterations is to use JMH, see "Java Microbenchmark Harness (the lesser of two evils)" pdf.

"1.8.0" Java HotSpot(TM) 64-Bit Server VM
  Mode Samples Mean Mean error
spectralnorm.testMethod   ss 65   4.181 0.008
nbody.testMethod   ss 65   22.462 0.001

But please don't just assume those warmed-up timings will be a whole lot faster for the JVM programs shown here, at the workloads shown here.

Revised BSD license

  Home   Conclusions   License   Play