Java measurements

This table shows 4 measurements - CPU Time, Elapsed Time, Memory and Code.

Each row shows those measurements for a particular Java program with a particular command-line input value N.

This table shows the current Java programs.

Program Source Code  N  CPU secs Elapsed secs Memory KB Code B
binary-trees  #6  120.190.20?583
binary-trees  #6  160.620.6397,408583
binary-trees  #6  207.387.39414,024583
binary-trees  #5  120.370.371,748926
binary-trees  #5  160.860.89102,300926
binary-trees  #5  209.979.99530,224926
binary-trees  #2  120.230.231,988603
binary-trees  #2  160.970.97128,928603
binary-trees  #2  2017.1617.18519,692603
binary-trees  #3  120.210.221,572584
binary-trees  #3  160.910.92162,080584
binary-trees  #3  2017.2617.28518,964584
binary-trees-redux  #2  120.210.221,984607
binary-trees-redux  #2  161.001.0075,784607
binary-trees-redux  #2  20105.33105.41169,572607
chameneos-redux  #2  60,0000.350.361,7641429
chameneos-redux  #2  600,0001.341.3547,5801429
chameneos-redux  #2  6,000,00011.2111.2198,7081429
chameneos-redux  #3  60,0001.511.5437,2761267
chameneos-redux  #3  600,00011.7611.7751,2161267
chameneos-redux  #3  6,000,000114.34114.3763,8801267
chameneos-redux  #4  60,0001.571.5935,9721607
chameneos-redux  #4  600,00011.9711.9836,0041607
chameneos-redux  #4  6,000,000112.64112.6751,9241607
chameneos-redux   60,0000.360.381,9561770
chameneos-redux   600,0000.820.8335,5161770
chameneos-redux   6,000,0003.553.5641,1241770
fannkuch-redux  #2  100.630.6330,144514
fannkuch-redux  #2  115.685.6929,676514
fannkuch-redux  #2  1270.5270.5439,896514
fannkuch-redux   100.620.6231,5761282
fannkuch-redux   115.285.2931,4881282
fannkuch-redux   1269.2569.2747,7841282
fasta  #4  250,0000.240.242,0121507
fasta  #4  2,500,0000.740.7529,4201507
fasta  #4  25,000,0005.145.1531,3761507
fasta  #2  250,0000.310.312,0241240
fasta  #2  2,500,0001.101.1031,5281240
fasta  #2  25,000,0008.018.0132,8681240
fasta  #5  250,0000.310.311,9882457
fasta  #5  2,500,0000.830.8435,2242457
fasta  #5  25,000,0005.425.4339,0242457
fasta-redux  #3  250,0000.200.221,7041443
fasta-redux  #3  2,500,0000.470.4828,9921443
fasta-redux  #3  25,000,0002.632.6430,3641443
k-nucleotide  #4  250,0001.161.16181,7041873
k-nucleotide  #4  2,500,0006.266.27211,6081873
k-nucleotide  #4  25,000,00048.9248.96185,5041873
k-nucleotide  #2  250,0001.201.2053,2241602
k-nucleotide  #2  2,500,0005.415.42107,9001602
k-nucleotide  #2  25,000,00046.7946.83468,3081602
k-nucleotide  #5  250,0001.541.56185,6442211
k-nucleotide  #5  2,500,0007.687.70216,7202211
k-nucleotide  #5  25,000,00057.9758.03218,6362211
k-nucleotide  #3  250,0001.241.2850,9761630
k-nucleotide  #3  2,500,0005.425.59108,0681630
k-nucleotide  #3  25,000,00047.4047.44467,0241630
k-nucleotide  #7  250,0001.131.1468,0361844
k-nucleotide  #7  2,500,0004.524.54350,0761844
k-nucleotide  #7  25,000,00034.7934.831,124,1041844
mandelbrot  #3  1,0000.370.381,600903
mandelbrot  #3  4,0003.053.0631,208903
mandelbrot  #3  16,00043.6743.6989,152903
mandelbrot   1,0000.360.361,788665
mandelbrot   4,0002.922.9229,600665
mandelbrot   16,00043.0243.0343,560665
mandelbrot  #6  1,0000.300.302,004802
mandelbrot  #6  4,0001.971.9835,536802
mandelbrot  #6  16,00027.6427.6689,792802
mandelbrot  #2  1,0000.310.321,664796
mandelbrot  #2  4,0001.721.7230,316796
mandelbrot  #2  16,00023.9223.9391,684796
meteor-contest  #2  2,0980.490.4933,2765177
meteor-contest   2,0980.920.9331,1804368
n-body  #2  500,0000.400.411,7801424
n-body  #2  5,000,0002.442.4430,6401424
n-body  #2  50,000,00022.6722.6830,0561424
n-body   500,000Failed  1424
pidigits  #3  2,0001.481.5252,0321826
pidigits  #3  6,0003.653.6563,5881826
pidigits  #3  10,0006.506.5072,4521826
pidigits   2,0001.351.36169,480800
pidigits   6,0006.816.83376,904800
pidigits   10,00018.1618.20381,788800
pidigits  #2  2,0000.300.332,020938
pidigits  #2  6,0001.261.2731,548938
pidigits  #2  10,0003.203.2033,052938
pidigits  #4  2,0001.521.5352,3401808
pidigits  #4  6,0003.713.7261,7641808
pidigits  #4  10,0006.596.6068,9001808
regex-dna  #8  50,0000.630.6533,7601377
regex-dna  #8  500,0002.582.5942,8921377
regex-dna  #8  5,000,00021.5521.56574,7041377
regex-dna  #6  50,0000.940.9559,1601410
regex-dna  #6  500,0002.952.95104,2761410
regex-dna  #6  5,000,000Failed  1410
regex-dna  #2  50,0000.930.9440,4201534
regex-dna  #2  500,0004.044.06121,3921534
regex-dna  #2  5,000,00033.2733.29715,0321534
regex-dna  #4  50,0000.850.8845,360921
regex-dna  #4  500,0003.133.1477,188921
regex-dna  #4  5,000,00023.6723.69716,328921
regex-dna  #7  50,0000.680.6834,1561284
regex-dna  #7  500,0002.572.5841,8761284
regex-dna  #7  5,000,00021.3921.40575,3721284
regex-dna  #5  50,0001.001.0141,728929
regex-dna  #5  500,0003.013.02115,952929
regex-dna  #5  5,000,00023.3423.35750,952929
reverse-complement  #7  250,0000.400.5033,3521640
reverse-complement  #7  2,500,0000.711.2458,8401640
reverse-complement  #7  25,000,0002.127.55283,7601640
reverse-complement  #6  250,0000.230.242,008745
reverse-complement  #6  2,500,0000.410.411,732745
reverse-complement  #6  25,000,0001.701.70474,400745
reverse-complement  #4  250,0000.260.292,072592
reverse-complement  #4  2,500,0000.550.8354,112592
reverse-complement  #4  25,000,0002.372.38369,992592
reverse-complement  #3  250,0000.280.298241661
reverse-complement  #3  2,500,0000.480.4876,4281661
reverse-complement  #3  25,000,0002.342.34313,9521661
spectral-norm   5000.390.4628,840514
spectral-norm   3,0005.215.2229,632514
spectral-norm   5,50016.9616.9632,708514
spectral-norm  #2  5000.380.382,040950
spectral-norm  #2  3,0005.055.0632,040950
spectral-norm  #2  5,50016.2216.2232,332950
thread-ring  #7  500,0002.692.7037,484473
thread-ring  #7  5,000,00024.3824.4039,784473
thread-ring  #7  50,000,000236.46236.5952,452473
thread-ring  #3  500,0003.073.09371,196530
thread-ring  #3  5,000,00025.3125.33370,624530
thread-ring  #3  50,000,000248.19248.34371,820530

 Java : ubiquitous jit server virtual machine 

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.45-b02, mixed mode)

Home Page: Java SE at a Glance

Download: Java SE Downloads

Let's see how much, or how little, the time taken to invoke the JVM might contribute to the usual Java program times shown in the benchmarks game. Here are some additional (Intel® Q6600® quad-core) elapsed time measurements, taken after the Java programs started and before they exited.

In the first case (Cold), we simply started and measured the program 66 times; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<1; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

In the second case (Warmed), we started the program once and repeated measurements again and again and again 66 times without restarting the JVM; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<66; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

Compare these additional measurements against the usual Java program measurements shown in the benchmarks game --

"1.7.0_06" Java HotSpot(TM) 64-Bit Server VM
System.nanoTime()  1) Cold   2) Warmed   
  mean σ mean σ   usual
meteor contest   0.0118s 0.0007 0.0016s 0.0002 0.22s
fasta-redux   2.45s 0.00 2.32s 0.00 2.51s
spectral-norm   4.44s 0.02 4.20s 0.16 4.51s
pidigits   4.69s 0.09 4.44s 0.05 4.61s
fasta   5.07s 0.46 4.84s 0.02 5.13s
chameneos-redux   5.84s 0.46 5.70s 0.48 5.65s
mandelbrot   7.93s 0.23 7.99s 0.01 7.02s
k-nucleotide   8.09s 0.28  --   --  8.05s
regex-dna   8.65s 0.27  --   --  8.61s
binary-trees   10.54s 0.28 7.66s 0.16 9.08s
fannkuch-redux   16.89s 1.32 17.26s 0.10 17.38s
nbody   22.43s 0.00 22.41s 0.00 22.50s
binary-trees-redux   34.15s 0.39 33.93s 0.31 33.38s

The largest and most obvious effects of bytecode loading and dynamic optimization can be seen with the meteor-contest program which only runs for a fraction of a second.

A more sophisticated way to make JVM timings following warmup iterations is to use JMH, see "Java Microbenchmark Harness (the lesser of two evils)" pdf.

"1.8.0" Java HotSpot(TM) 64-Bit Server VM
  Mode Samples Mean Mean error
spectralnorm.testMethod   ss 65   4.181 0.008
nbody.testMethod   ss 65   22.462 0.001

But please don't just assume those warmed-up timings will be a whole lot faster for the JVM programs shown here, at the workloads shown here.

Revised BSD license

  Home   Conclusions   License   Play