Java measurements

This table shows 4 measurements - CPU Time, Elapsed Time, Memory and Code.

Each row shows those measurements for a particular Java program with a particular command-line input value N.

This table shows the current Java programs.

Program Source Code  N  CPU secs Elapsed secs Memory KB Code B
binary-trees  #2  120.210.21432603
binary-trees  #2  160.950.9584,352603
binary-trees  #2  2022.9622.99448,112603
binary-trees  #3  120.210.21432584
binary-trees  #3  160.930.9397,816584
binary-trees  #3  2023.1023.14450,320584
binary-trees-redux  #2  120.190.19?607
binary-trees-redux  #2  160.991.0063,672607
binary-trees-redux  #2  20106.29106.37156,016607
chameneos-redux  #2  60,0000.320.328041429
chameneos-redux  #2  600,0001.241.2427,7401429
chameneos-redux  #2  6,000,00010.1510.1654,2761429
chameneos-redux  #3  60,0001.431.4323,4121267
chameneos-redux  #3  600,00011.6711.6829,3641267
chameneos-redux  #3  6,000,000131.02131.0725,9041267
chameneos-redux  #4  60,0001.421.4221,6001607
chameneos-redux  #4  600,00011.8811.8920,9361607
chameneos-redux  #4  6,000,000113.21113.2522,5881607
chameneos-redux   60,0000.320.324321770
chameneos-redux   600,0000.670.6823,2081770
chameneos-redux   6,000,0003.353.3524,7561770
fannkuch-redux  #2  100.680.6820,004514
fannkuch-redux  #2  115.655.6519,776514
fannkuch-redux  #2  1273.1073.1319,920514
fannkuch-redux   100.580.5819,7081282
fannkuch-redux   115.225.2219,8841282
fannkuch-redux   1268.8968.9119,8921282
fasta  #4  250,0000.230.237521507
fasta  #4  2,500,0000.750.7619,5721507
fasta  #4  25,000,0005.375.3720,8881507
fasta  #2  250,0000.280.288041240
fasta  #2  2,500,0001.111.1121,7521240
fasta  #2  25,000,0008.378.3720,3321240
fasta  #5  250,0000.310.314202457
fasta  #5  2,500,0000.820.8229,3842457
fasta  #5  25,000,0005.445.4432,6642457
fasta-redux  #3  250,0000.180.18?1443
fasta-redux  #3  2,500,0000.450.4519,3641443
fasta-redux  #3  25,000,0002.602.6119,9001443
k-nucleotide  #4  250,0001.391.40173,0681873
k-nucleotide  #4  2,500,0006.986.99183,7881873
k-nucleotide  #4  25,000,00060.7460.79174,1041873
k-nucleotide  #2  250,0001.271.2947,8081602
k-nucleotide  #2  2,500,0005.855.85108,2481602
k-nucleotide  #2  25,000,00048.5748.61505,8401602
k-nucleotide  #5  250,0001.891.89188,6322211
k-nucleotide  #5  2,500,0008.308.31202,1322211
k-nucleotide  #5  25,000,00069.2969.36194,9602211
k-nucleotide  #3  250,0001.241.2451,6721630
k-nucleotide  #3  2,500,0006.006.01108,0921630
k-nucleotide  #3  25,000,00048.7948.83504,9841630
mandelbrot  #3  1,0000.350.35684903
mandelbrot  #3  4,0002.892.8921,696903
mandelbrot  #3  16,00042.4942.5172,692903
mandelbrot   1,0000.330.33684665
mandelbrot   4,0002.872.8819,568665
mandelbrot   16,00042.7242.7419,564665
mandelbrot  #6  1,0000.270.271,060802
mandelbrot  #6  4,0001.911.9221,760802
mandelbrot  #6  16,00027.5227.5473,004802
mandelbrot  #2  1,0000.250.25804796
mandelbrot  #2  4,0001.691.6921,712796
mandelbrot  #2  16,00023.8123.8272,688796
meteor-contest  #2  2,0980.480.4824,3925177
meteor-contest   2,0980.890.9022,2604368
n-body  #2  500,0000.390.394281424
n-body  #2  5,000,0002.422.4219,3041424
n-body  #2  50,000,00022.6622.6719,5681424
pidigits  #3  2,0001.441.4646,9361826
pidigits  #3  6,0003.613.6146,3721826
pidigits  #3  10,0006.326.3254,8041826
pidigits   2,0001.481.49114,904800
pidigits   6,0007.017.03150,984800
pidigits   10,00019.1719.21176,492800
pidigits  #2  2,0000.280.33708938
pidigits  #2  6,0001.271.2721,840938
pidigits  #2  10,0003.143.1423,224938
pidigits  #4  2,0001.461.4839,1041808
pidigits  #4  6,0003.553.5541,3121808
pidigits  #4  10,0006.436.4452,0481808
regex-dna  #8  50,0000.580.6023,8001377
regex-dna  #8  500,0002.712.8038,5841377
regex-dna  #8  5,000,00022.9122.92566,7841377
regex-dna  #6  50,0000.820.8345,8001410
regex-dna  #6  500,0002.972.9779,4441410
regex-dna  #6  5,000,00022.1822.19518,0681410
regex-dna  #2  50,0000.850.8627,6801534
regex-dna  #2  500,0004.174.23115,1281534
regex-dna  #2  5,000,00040.8441.44576,0721534
regex-dna  #4  50,0000.810.8235,532921
regex-dna  #4  500,0003.053.0666,764921
regex-dna  #4  5,000,00023.8023.82709,576921
regex-dna  #7  50,0000.590.6022,7801284
regex-dna  #7  500,0002.732.7332,8961284
regex-dna  #7  5,000,00021.4021.42563,6361284
regex-dna  #5  50,0000.840.8529,616929
regex-dna  #5  500,0003.253.2599,116929
regex-dna  #5  5,000,00023.5523.57707,988929
reverse-complement  #7  250,0000.370.4822,4721640
reverse-complement  #7  2,500,0000.541.2047,2081640
reverse-complement  #7  25,000,0002.169.23270,5361640
reverse-complement  #6  250,0000.210.22804745
reverse-complement  #6  2,500,0000.380.39800745
reverse-complement  #6  25,000,0001.861.87517,532745
reverse-complement  #4  250,0000.230.23428592
reverse-complement  #4  2,500,0000.470.4830,380592
reverse-complement  #4  25,000,0002.562.57481,176592
reverse-complement  #3  250,0000.230.296881661
reverse-complement  #3  2,500,0000.520.7850,1961661
reverse-complement  #3  25,000,0002.702.71300,4241661
spectral-norm   5000.360.36712514
spectral-norm   3,0005.255.2521,152514
spectral-norm   5,50017.0917.1021,152514
spectral-norm  #2  5000.360.4319,832950
spectral-norm  #2  3,0005.405.4020,636950
spectral-norm  #2  5,50017.5117.5121,424950
thread-ring  #7  500,0002.622.6327,692473
thread-ring  #7  5,000,00023.8223.8427,968473
thread-ring  #7  50,000,000232.00232.1527,964473
thread-ring  #3  500,0002.862.8730,812530
thread-ring  #3  5,000,00025.0725.0944,584530
thread-ring  #3  50,000,000245.24245.41234,384530

 Java : ubiquitous jit server virtual machine 

java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) 64-Bit Server VM (build 25.0-b70, mixed mode)

Home Page: Java SE at a Glance

Download: Java SE Downloads

Let's see how much, or how little, the time taken to invoke the JVM might contribute to the usual Java program times shown in the benchmarks game. Here are some additional (Intel® Q6600® quad-core) elapsed time measurements, taken after the Java programs started and before they exited.

In the first case (Cold), we simply started and measured the program 66 times; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<1; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

In the second case (Warmed), we started the program once and repeated measurements again and again and again 66 times without restarting the JVM; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<66; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

Compare these additional measurements against the usual Java program measurements shown in the benchmarks game --

"1.7.0_06" Java HotSpot(TM) 64-Bit Server VM
System.nanoTime()  1) Cold   2) Warmed   
  mean σ mean σ   usual
meteor contest   0.0118s 0.0007 0.0016s 0.0002 0.22s
fasta-redux   2.45s 0.00 2.32s 0.00 2.51s
spectral-norm   4.44s 0.02 4.20s 0.16 4.51s
pidigits   4.69s 0.09 4.44s 0.05 4.61s
fasta   5.07s 0.46 4.84s 0.02 5.13s
chameneos-redux   5.84s 0.46 5.70s 0.48 5.65s
mandelbrot   7.93s 0.23 7.99s 0.01 7.02s
k-nucleotide   8.09s 0.28  --   --  8.05s
regex-dna   8.65s 0.27  --   --  8.61s
binary-trees   10.54s 0.28 7.66s 0.16 9.08s
fannkuch-redux   16.89s 1.32 17.26s 0.10 17.38s
nbody   22.43s 0.00 22.41s 0.00 22.50s
binary-trees-redux   34.15s 0.39 33.93s 0.31 33.38s

The largest and most obvious effects of bytecode loading and dynamic optimization can be seen with the meteor-contest program which only runs for a fraction of a second.

A more sophisticated way to make JVM timings following warmup iterations is to use JMH, see "Java Microbenchmark Harness (the lesser of two evils)" pdf.

"1.8.0" Java HotSpot(TM) 64-Bit Server VM
  Mode Samples Mean Mean error
spectralnorm.testMethod   ss 65   4.181 0.008
nbody.testMethod   ss 65   22.462 0.001

But please don't just assume those warmed-up timings will be a whole lot faster for the JVM programs shown here, at the workloads shown here.

Revised BSD license

  Home   Conclusions   License   Play