Java measurements

This table shows 4 measurements - CPU Time, Elapsed Time, Memory and Code.

Each row shows those measurements for a particular Java program with a particular command-line input value N.

This table shows the current Java programs.

Program Source Code  N  CPU secs Elapsed secs Memory KB Code B
binary-trees  #2  120.230.733,936603
binary-trees  #2  160.980.85153,580603
binary-trees  #2  2018.0213.13471,008603
binary-trees  #3  120.200.18?584
binary-trees  #3  160.960.82153,560584
binary-trees  #3  2017.7512.78471,136584
binary-trees-redux  #2  120.220.19?607
binary-trees-redux  #2  161.000.7955,124607
binary-trees-redux  #2  20253.4073.87134,328607
chameneos-redux  #2  60,0000.740.322481429
chameneos-redux  #2  600,0004.371.5131,5041429
chameneos-redux  #2  6,000,00038.7813.3132,7441429
chameneos-redux  #3  60,0000.810.343481267
chameneos-redux  #3  600,0003.501.1033,7321267
chameneos-redux  #3  6,000,00026.787.9634,1001267
chameneos-redux   60,0000.410.242521770
chameneos-redux   600,0001.470.8718,8761770
chameneos-redux   6,000,0005.764.9625,3641770
chameneos-redux  #4  60,0000.750.312521607
chameneos-redux  #4  600,0003.020.9918,1801607
chameneos-redux  #4  6,000,00019.825.6417,7081607
fannkuch-redux   100.810.312521282
fannkuch-redux   114.941.3515,8921282
fannkuch-redux   1268.0417.2515,6961282
fannkuch-redux  #2  100.770.7215,852514
fannkuch-redux  #2  116.786.7416,076514
fannkuch-redux  #2  1276.5076.4616,164514
fasta  #5  250,0000.380.794,0682457
fasta  #5  2,500,0000.840.372362457
fasta  #5  25,000,0005.341.7229,4842457
fasta  #2  250,0000.300.242521240
fasta  #2  2,500,0001.141.0416,8681240
fasta  #2  25,000,0007.927.8216,7121240
fasta  #4  250,0000.250.212481507
fasta  #4  2,500,0000.780.7115,6441507
fasta  #4  25,000,0005.135.0516,4441507
fasta-redux  #3  250,0000.210.18?1443
fasta-redux  #3  2,500,0000.430.392481443
fasta-redux  #3  25,000,0002.031.9815,8961443
k-nucleotide  #5  250,0002.301.05165,4842211
k-nucleotide  #5  2,500,00011.534.29157,4842211
k-nucleotide  #5  25,000,00081.2336.06178,6442211
k-nucleotide  #2  250,0001.420.5127,8281602
k-nucleotide  #2  2,500,0006.772.02138,2681602
k-nucleotide  #2  25,000,00049.7014.28909,1081602
k-nucleotide  #3  250,0001.550.5524,4641630
k-nucleotide  #3  2,500,0005.901.7980,2761630
k-nucleotide  #3  25,000,00045.3313.06923,0681630
k-nucleotide  #4  250,0001.881.73168,9281873
k-nucleotide  #4  2,500,0007.226.78151,1801873
k-nucleotide  #4  25,000,00055.3854.37176,9761873
mandelbrot   1,0000.350.32248665
mandelbrot   4,0002.902.8615,620665
mandelbrot   16,00043.3943.3615,652665
mandelbrot  #2  1,0000.350.19?796
mandelbrot  #2  4,0001.930.6017,300796
mandelbrot  #2  16,00027.176.9573,488796
mandelbrot  #6  1,0000.360.21248802
mandelbrot  #6  4,0002.160.6617,244802
mandelbrot  #6  16,00030.447.7669,188802
mandelbrot  #3  1,0000.460.23248903
mandelbrot  #3  4,0003.480.9817,476903
mandelbrot  #3  16,00042.2510.7269,932903
meteor-contest   2,0980.980.8717,8364368
meteor-contest  #2  2,0980.520.292485177
n-body  #2  500,0000.450.4215,6601424
n-body  #2  5,000,0002.622.6015,6081424
n-body  #2  50,000,00024.5024.4815,6561424
pidigits   2,0001.891.21159,896800
pidigits   6,00010.279.37368,216800
pidigits   10,00027.0726.11369,760800
pidigits  #2  2,0000.330.32248938
pidigits  #2  6,0001.601.5016,340938
pidigits  #2  10,0004.154.0418,352938
pidigits  #3  2,0001.930.7528,8161826
pidigits  #3  6,0005.782.2536,2321826
pidigits  #3  10,00012.884.7942,4361826
pidigits  #4  2,0001.990.8126,3121808
pidigits  #4  6,0006.382.5532,4881808
pidigits  #4  10,00013.144.7942,2121808
regex-dna  #5  50,0001.080.4225,684929
regex-dna  #5  500,0003.541.2586,984929
regex-dna  #5  5,000,00026.449.20713,236929
regex-dna  #8  50,0000.691.1222,4201377
regex-dna  #8  500,0002.762.5130,5761377
regex-dna  #8  5,000,00020.9520.62562,7481377
regex-dna  #7  50,0000.670.4518,4721284
regex-dna  #7  500,0002.742.3727,8001284
regex-dna  #7  5,000,00021.2520.93558,5001284
regex-dna  #6  50,0001.040.3944,5561410
regex-dna  #6  500,000Failed  1410
regex-dna  #2  50,0001.080.4224,4641534
regex-dna  #2  500,0005.041.64103,1161534
regex-dna  #2  5,000,00039.1612.47710,7481534
regex-dna  #4  50,0000.840.5428,072921
regex-dna  #4  500,0003.472.9063,544921
regex-dna  #4  5,000,00025.8224.24725,364921
reverse-complement  #4  250,0000.290.21248592
reverse-complement  #4  2,500,0000.600.5127,232592
reverse-complement  #4  25,000,0003.143.04478,080592
reverse-complement  #3  250,0000.320.242481661
reverse-complement  #3  2,500,0000.600.373481661
reverse-complement  #3  25,000,0003.161.63296,1361661
reverse-complement  #7  250,0000.540.382481640
reverse-complement  #7  2,500,0000.851.0341,4161640
reverse-complement  #7  25,000,0002.707.31264,9121640
reverse-complement  #6  250,0000.280.20?745
reverse-complement  #6  2,500,0000.440.37248745
reverse-complement  #6  25,000,0002.011.74513,220745
spectral-norm  #2  5000.490.26248950
spectral-norm  #2  3,0005.351.4816,432950
spectral-norm  #2  5,50016.564.2916,460950
spectral-norm   5000.380.32248514
spectral-norm   3,0005.085.0316,432514
spectral-norm   5,50016.5016.4416,968514
thread-ring  #7  500,0005.494.4322,048473
thread-ring  #7  5,000,00051.2341.9121,780473
thread-ring  #7  50,000,000499.14373.9621,876473
thread-ring  #3  500,0005.694.4128,296530
thread-ring  #3  5,000,00052.0241.3727,952530
thread-ring  #3  50,000,000503.46390.17286,340530

 Java : ubiquitous jit server virtual machine 

java version "1.8.0"
Java(TM) SE Runtime Environment (build 1.8.0-b132)
Java HotSpot(TM) Server VM (build 25.0-b70, mixed mode)

Home Page: Java SE at a Glance

Download: Java SE Downloads

Let's see how much, or how little, the time taken to invoke the JVM might contribute to the usual Java program times shown in the benchmarks game. Here are some additional (Intel® Q6600® quad-core) elapsed time measurements, taken after the Java programs started and before they exited.

In the first case (Cold), we simply started and measured the program 66 times; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<1; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

In the second case (Warmed), we started the program once and repeated measurements again and again and again 66 times without restarting the JVM; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<66; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

Compare these additional measurements against the usual Java program measurements shown in the benchmarks game --

"1.7.0_06" Java HotSpot(TM) 64-Bit Server VM
System.nanoTime()  1) Cold   2) Warmed   
  mean σ mean σ   usual
meteor contest   0.0118s 0.0007 0.0016s 0.0002 0.22s
fasta-redux   2.45s 0.00 2.32s 0.00 2.51s
spectral-norm   4.44s 0.02 4.20s 0.16 4.51s
pidigits   4.69s 0.09 4.44s 0.05 4.61s
fasta   5.07s 0.46 4.84s 0.02 5.13s
chameneos-redux   5.84s 0.46 5.70s 0.48 5.65s
mandelbrot   7.93s 0.23 7.99s 0.01 7.02s
k-nucleotide   8.09s 0.28  --   --  8.05s
regex-dna   8.65s 0.27  --   --  8.61s
binary-trees   10.54s 0.28 7.66s 0.16 9.08s
fannkuch-redux   16.89s 1.32 17.26s 0.10 17.38s
nbody   22.43s 0.00 22.41s 0.00 22.50s
binary-trees-redux   34.15s 0.39 33.93s 0.31 33.38s

The largest and most obvious effects of bytecode loading and dynamic optimization can be seen with the meteor-contest program which only runs for a fraction of a second.

A more sophisticated way to make JVM timings following warmup iterations is to use JMH, see "Java Microbenchmark Harness (the lesser of two evils)" pdf.

"1.8.0" Java HotSpot(TM) 64-Bit Server VM
  Mode Samples Mean Mean error
spectralnorm.testMethod   ss 65   4.181 0.008
nbody.testMethod   ss 65   22.462 0.001

But please don't just assume those warmed-up timings will be a whole lot faster for the JVM programs shown here, at the workloads shown here.

Revised BSD license

  Home   Conclusions   License   Play