Java measurements

This table shows 4 measurements - CPU Time, Elapsed Time, Memory and Code.

Each row shows those measurements for a particular Java program with a particular command-line input value N.

This table shows the current Java programs.

Program Source Code  N  CPU secs Elapsed secs Memory KB Code B
binary-trees  #2  120.240.20?603
binary-trees  #2  161.050.85162,736603
binary-trees  #2  2019.0313.56505,456603
binary-trees  #5  120.480.8420,128926
binary-trees  #5  161.480.5926,520926
binary-trees  #5  2019.645.88513,180926
binary-trees  #6  120.230.644,204583
binary-trees  #6  160.640.5145,888583
binary-trees  #6  208.265.93386,484583
binary-trees  #3  120.240.20?584
binary-trees  #3  160.960.8098,204584
binary-trees  #3  2017.9113.03488,716584
binary-trees-redux  #2  120.250.21368607
binary-trees-redux  #2  161.070.8464,656607
binary-trees-redux  #2  20252.4274.26142,128607
chameneos-redux  #2  60,0000.570.314761429
chameneos-redux  #2  600,0003.841.4742,4801429
chameneos-redux  #2  6,000,00024.229.8959,8921429
chameneos-redux  #3  60,0000.830.364801267
chameneos-redux  #3  600,0003.631.1543,9681267
chameneos-redux  #3  6,000,00026.517.9250,1761267
chameneos-redux   60,0000.380.264801770
chameneos-redux   600,0001.160.6727,7881770
chameneos-redux   6,000,0004.443.8238,2961770
chameneos-redux  #4  60,0000.670.304641607
chameneos-redux  #4  600,0002.980.9829,4881607
chameneos-redux  #4  6,000,00018.165.3030,7321607
fannkuch-redux   100.790.334561282
fannkuch-redux   115.391.4925,8321282
fannkuch-redux   1267.5617.1325,8041282
fannkuch-redux  #2  100.700.6523,744514
fannkuch-redux  #2  115.725.6824,000514
fannkuch-redux  #2  1273.9673.9223,796514
fasta  #5  250,0000.420.244802457
fasta  #5  2,500,0000.990.4228,1762457
fasta  #5  25,000,0005.901.8231,6002457
fasta  #2  250,0000.350.264801240
fasta  #2  2,500,0001.121.0224,4361240
fasta  #2  25,000,0007.627.5225,2361240
fasta  #4  250,0000.280.244761507
fasta  #4  2,500,0000.770.6824,0281507
fasta  #4  25,000,0004.784.6824,9481507
fasta-redux  #3  250,0000.220.20?1443
fasta-redux  #3  2,500,0000.450.404721443
fasta-redux  #3  25,000,0001.991.9324,5881443
k-nucleotide  #7  250,0001.660.5929,1881844
k-nucleotide  #7  2,500,0005.181.65310,8081844
k-nucleotide  #7  25,000,00035.2710.611,199,1841844
k-nucleotide  #5  250,0002.421.03188,9322211
k-nucleotide  #5  2,500,0009.103.80163,8442211
k-nucleotide  #5  25,000,00077.0832.87209,1482211
k-nucleotide  #2  250,0001.800.6440,0881602
k-nucleotide  #2  2,500,0005.501.7189,8401602
k-nucleotide  #2  25,000,00044.6212.83518,0361602
k-nucleotide  #3  250,0001.960.6751,0401630
k-nucleotide  #3  2,500,0005.291.8290,4361630
k-nucleotide  #3  25,000,00044.7912.87517,6321630
k-nucleotide  #4  250,0001.991.18171,2281873
k-nucleotide  #4  2,500,0007.666.98160,9001873
k-nucleotide  #4  25,000,00059.5258.48196,4041873
mandelbrot   1,0000.380.35472665
mandelbrot   4,0002.962.9223,208665
mandelbrot   16,00043.7643.7323,428665
mandelbrot  #2  1,0000.420.23480796
mandelbrot  #2  4,0002.000.6325,196796
mandelbrot  #2  16,00027.797.1287,832796
mandelbrot  #6  1,0000.380.22456802
mandelbrot  #6  4,0002.300.7125,280802
mandelbrot  #6  16,00029.037.4487,332802
mandelbrot  #3  1,0000.520.26472903
mandelbrot  #3  4,0003.100.9125,364903
mandelbrot  #3  16,00042.4910.8086,168903
meteor-contest   2,0980.980.8626,8804368
meteor-contest  #2  2,0980.530.314805177
n-body  #2  500,0000.450.8723,7481424
n-body  #2  5,000,0002.622.5924,0481424
n-body  #2  50,000,00024.3424.3124,1561424
n-body   500,000Failed  1424
pidigits   2,0001.991.20165,428800
pidigits   6,0009.578.84375,128800
pidigits   10,00026.3125.37376,616800
pidigits  #2  2,0000.360.33456938
pidigits  #2  6,0001.721.6225,656938
pidigits  #2  10,0004.174.0726,324938
pidigits  #3  2,0001.740.7437,7121826
pidigits  #3  6,0005.562.4344,7081826
pidigits  #3  10,0007.954.2454,5121826
pidigits  #4  2,0001.870.6836,5281808
pidigits  #4  6,0005.452.3345,6481808
pidigits  #4  10,0008.334.3653,4441808
regex-dna  #5  50,0001.040.4333,172929
regex-dna  #5  500,0003.581.29105,896929
regex-dna  #5  5,000,00025.798.80729,244929
regex-dna  #8  50,0000.730.4827,9281377
regex-dna  #8  500,0002.682.2936,8241377
regex-dna  #8  5,000,00020.0819.68567,3281377
regex-dna  #7  50,0000.740.4828,3881284
regex-dna  #7  500,0002.582.2536,5881284
regex-dna  #7  5,000,00020.4620.05567,1401284
regex-dna  #6  50,0001.290.4955,1681410
regex-dna  #6  500,000Failed  1410
regex-dna  #2  50,0001.280.4831,9601534
regex-dna  #2  500,0005.551.80117,7681534
regex-dna  #2  5,000,00040.0012.59725,0641534
regex-dna  #4  50,0000.960.5637,160921
regex-dna  #4  500,0003.422.7360,760921
regex-dna  #4  5,000,00024.3722.66735,240921
reverse-complement  #4  250,0000.330.24476592
reverse-complement  #4  2,500,0000.670.5235,200592
reverse-complement  #4  25,000,0002.732.59486,716592
reverse-complement  #3  250,0000.360.254681661
reverse-complement  #3  2,500,0000.570.374561661
reverse-complement  #3  25,000,0002.911.44308,0601661
reverse-complement  #7  250,0000.540.414601640
reverse-complement  #7  2,500,0001.131.2851,8681640
reverse-complement  #7  25,000,0004.109.44275,9041640
reverse-complement  #6  250,0000.280.22480745
reverse-complement  #6  2,500,0000.460.38472745
reverse-complement  #6  25,000,0001.951.67524,704745
spectral-norm  #2  5000.490.30456950
spectral-norm  #2  3,0005.351.5125,956950
spectral-norm  #2  5,50016.534.3026,476950
spectral-norm   5000.420.35476514
spectral-norm   3,0005.275.1826,232514
spectral-norm   5,50016.9916.9126,444514
thread-ring  #7  500,0006.345.2332,088473
thread-ring  #7  5,000,00057.5042.9032,124473
thread-ring  #7  50,000,000570.60417.2431,880473
thread-ring  #3  500,0006.665.19370,136530
thread-ring  #3  5,000,00057.4343.29369,764530
thread-ring  #3  50,000,000539.38395.29371,116530

 Java : ubiquitous jit server virtual machine 

java version "1.8.0_45"
Java(TM) SE Runtime Environment (build 1.8.0_45-b14)
Java HotSpot(TM) Server VM (build 25.45-b02, mixed mode)

Home Page: Java SE at a Glance

Download: Java SE Downloads

Let's see how much, or how little, the time taken to invoke the JVM might contribute to the usual Java program times shown in the benchmarks game. Here are some additional (Intel® Q6600® quad-core) elapsed time measurements, taken after the Java programs started and before they exited.

In the first case (Cold), we simply started and measured the program 66 times; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<1; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

In the second case (Warmed), we started the program once and repeated measurements again and again and again 66 times without restarting the JVM; and then discarded the first measurement leaving 65 data points.

   public static void main(String[] args){
      for (int i=0; i<66; ++i){ 
         System.gc(); 
         long t1 = System.nanoTime();
         nbody.program_main(args);
         long t2 = System.nanoTime();
         System.err.println( String.format( "%.6f", (t2 - t1) * 1e-9 ) );         
      }
   }

Compare these additional measurements against the usual Java program measurements shown in the benchmarks game --

"1.7.0_06" Java HotSpot(TM) 64-Bit Server VM
System.nanoTime()  1) Cold   2) Warmed   
  mean σ mean σ   usual
meteor contest   0.0118s 0.0007 0.0016s 0.0002 0.22s
fasta-redux   2.45s 0.00 2.32s 0.00 2.51s
spectral-norm   4.44s 0.02 4.20s 0.16 4.51s
pidigits   4.69s 0.09 4.44s 0.05 4.61s
fasta   5.07s 0.46 4.84s 0.02 5.13s
chameneos-redux   5.84s 0.46 5.70s 0.48 5.65s
mandelbrot   7.93s 0.23 7.99s 0.01 7.02s
k-nucleotide   8.09s 0.28  --   --  8.05s
regex-dna   8.65s 0.27  --   --  8.61s
binary-trees   10.54s 0.28 7.66s 0.16 9.08s
fannkuch-redux   16.89s 1.32 17.26s 0.10 17.38s
nbody   22.43s 0.00 22.41s 0.00 22.50s
binary-trees-redux   34.15s 0.39 33.93s 0.31 33.38s

The largest and most obvious effects of bytecode loading and dynamic optimization can be seen with the meteor-contest program which only runs for a fraction of a second.

A more sophisticated way to make JVM timings following warmup iterations is to use JMH, see "Java Microbenchmark Harness (the lesser of two evils)" pdf.

"1.8.0" Java HotSpot(TM) 64-Bit Server VM
  Mode Samples Mean Mean error
spectralnorm.testMethod   ss 65   4.181 0.008
nbody.testMethod   ss 65   22.462 0.001

But please don't just assume those warmed-up timings will be a whole lot faster for the JVM programs shown here, at the workloads shown here.

Revised BSD license

  Home   Conclusions   License   Play