OnlineResults
Go to http://microbenchmarks.appspot.com to get an API key. Paste this in your ~/.caliperrc configuration file. With this in place, Caliper will upload the results for each benchmark you run.
Click the → and ← icons in the column headers to move columns. Rearranging columns will also cause the rows to be rearranged so that values are grouped left to right. Moving the rightmost column to the right pivots the table.
Click linear runtime to toggle between linear and logarithmic display. Logarithmic display is most useful when measurements span a wide range.
Uncheck the checkboxes in the Variables table to hide values from display.
Click a run's name in the Runs table to rename it. Renames will be saved to the server.
When the output of a trials is subject to random variation, some barchart bars will be terminated with lighter colors and T lines.
The endcaps on the barcharts are Box Plots, and are composed of up to 5 values:
- The left-side
Tindicates the minimum measured value. - The left-hand edge of the lighter-colored sections is the first quartile. 25% of the measurements are below this value; 75% are above it.
- The middle edge that divides the two lighter-colored sections is the median. 50% of the measurements are below this value; 50% are above it.
- The right-hand edge of the lighter-colored sections is the third quartile. 75% of the measurements are below this value; 25% are above it.
- The right-side
Tindicates the maximum measured value.
Many bars will not have all of these features. In that case, the omitted feature is not interesting: it is equal to the adjacent value. If all measurements are exactly equal, then the minimum, first-quartile, median, third-quartile and maximum will all be together at the bar's end.
Variation may be due to the way the benchmark was coded. For example, use of a java.util.Random within the benchmark can yield unrepeatable results. Variation may also come from uncontrollable factors in the underlying operating system and hardware. For example, benchmarks that use large amounts memory will be impacted by behind-the-scenes caches and paging.