Criterion.Analysis

criterion-1.1.1.0: Robust, reliable performance measurement and analysis

Copyright | (c) 2009-2014 Bryan O'Sullivan
License | BSD-style
Maintainer | bos@serpentine.com
Stability | experimental
Portability | GHC
Safe Haskell | None
Language | Haskell98

Criterion.Analysis

Description

Analysis code for benchmarks.

Synopsis

- data Outliers = Outliers {
  - samplesSeen :: !Int64
  - lowSevere :: !Int64
  - lowMild :: !Int64
  - highMild :: !Int64
  - highSevere :: !Int64
  }
- data OutlierEffect
  - = Unaffected
  - | Slight
  - | Moderate
  - | Severe
- data OutlierVariance = OutlierVariance {
  - ovEffect :: OutlierEffect
  - ovDesc :: String
  - ovFraction :: Double
  }
- data SampleAnalysis = SampleAnalysis {
  - anRegress :: [Regression]
  - anOverhead :: Double
  - anMean :: Estimate
  - anStdDev :: Estimate
  - anOutlierVar :: OutlierVariance
  }
- analyseSample :: Int -> String -> Vector Measured -> ExceptT String Criterion Report
- scale :: Double -> SampleAnalysis -> SampleAnalysis
- analyseMean :: Sample -> Int -> Criterion Double
- countOutliers :: Outliers -> Int64
- classifyOutliers :: Sample -> Outliers
- noteOutliers :: Outliers -> Criterion ()
- outlierVariance :: Estimate -> Estimate -> Double -> OutlierVariance
- resolveAccessors :: [String] -> Either String [(String, Measured -> Maybe Double)]
- validateAccessors :: [String] -> String -> Either String [(String, Measured -> Maybe Double)]
- regress :: GenIO -> [String] -> String -> Vector Measured -> ExceptT String Criterion Regression

# Documentation

data Outliers

Outliers from sample data, calculated using the boxplot technique.

Constructors

Outliers |
Fields samplesSeen :: !Int64 lowSevere :: !Int64 More than 3 times the interquartile range (IQR) below the first quartile. lowMild :: !Int64 Between 1.5 and 3 times the IQR below the first quartile. highMild :: !Int64 Between 1.5 and 3 times the IQR above the third quartile. highSevere :: !Int64 More than 3 times the IQR above the third quartile.

Instances

Eq Outliers |
Data Outliers |
Read Outliers |
Show Outliers |
Generic Outliers |
NFData Outliers |
ToJSON Outliers |
FromJSON Outliers |
Monoid Outliers |
Binary Outliers |
type Rep Outliers |

data OutlierEffect

A description of the extent to which outliers in the sample data affect the sample mean and standard deviation.

Constructors

Unaffected | Less than 1% effect.
Slight | Between 1% and 10%.
Moderate | Between 10% and 50%.
Severe | Above 50% (i.e. measurements are useless).

Instances

Eq OutlierEffect |
Data OutlierEffect |
Ord OutlierEffect |
Read OutlierEffect |
Show OutlierEffect |
Generic OutlierEffect |
NFData OutlierEffect |
ToJSON OutlierEffect |
FromJSON OutlierEffect |
Binary OutlierEffect |
type Rep OutlierEffect |

data OutlierVariance

Analysis of the extent to which outliers in a sample affect its standard deviation (and to some extent, its mean).

Constructors

OutlierVariance |
Fields ovEffect :: OutlierEffect Qualitative description of effect. ovDesc :: String Brief textual description of effect. ovFraction :: Double Quantitative description of effect (a fraction between 0 and 1).

Instances

Eq OutlierVariance |
Data OutlierVariance |
Read OutlierVariance |
Show OutlierVariance |
Generic OutlierVariance |
NFData OutlierVariance |
ToJSON OutlierVariance |
FromJSON OutlierVariance |
Binary OutlierVariance |
type Rep OutlierVariance |

data SampleAnalysis

Result of a bootstrap analysis of a non-parametric sample.

Constructors

SampleAnalysis |
Fields anRegress :: [Regression] Estimates calculated via linear regression. anOverhead :: Double Estimated measurement overhead, in seconds. Estimation is performed via linear regression. anMean :: Estimate Estimated mean. anStdDev :: Estimate Estimated standard deviation. anOutlierVar :: OutlierVariance Description of the effects of outliers on the estimated variance.

Instances

Eq SampleAnalysis |
Data SampleAnalysis |
Read SampleAnalysis |
Show SampleAnalysis |
Generic SampleAnalysis |
NFData SampleAnalysis |
ToJSON SampleAnalysis |
FromJSON SampleAnalysis |
Binary SampleAnalysis |
type Rep SampleAnalysis |

analyseSample

Arguments

:: Int | Experiment number.
-> String | Experiment name.
-> Vector Measured | Sample data.
-> ExceptT String Criterion Report |

Perform an analysis of a measurement.

scale

Arguments

:: Double | Value to multiply by.
-> SampleAnalysis |
-> SampleAnalysis |

Multiply the `Estimate`s in an analysis by the given value, using `scale`.

analyseMean

Arguments

:: Sample |
-> Int | Number of iterations used to compute the sample.
-> Criterion Double |

Display the mean of a `Sample`, and characterise the outliers present in the sample.

countOutliers :: Outliers -> Int64

Count the total number of outliers in a sample.

classifyOutliers :: Sample -> Outliers

Classify outliers in a data set, using the boxplot technique.

noteOutliers :: Outliers -> Criterion ()

Display a report of the `Outliers` present in a `Sample`.

outlierVariance

Arguments

:: Estimate | Bootstrap estimate of sample mean.
-> Estimate | Bootstrap estimate of sample standard deviation.
-> Double | Number of original iterations.
-> OutlierVariance |

Compute the extent to which outliers in the sample data affect the sample mean and standard deviation.

resolveAccessors :: [String] -> Either String [(String, Measured -> Maybe Double)]

Given a list of accessor names (see `measureKeys`), return either a mapping from accessor name to function or an error message if any names are wrong.

validateAccessors

Arguments

:: [String] | Predictor names.
-> String | Responder name.
-> Either String [(String, Measured -> Maybe Double)] |

Given predictor and responder names, do some basic validation, then hand back the relevant accessors.

regress

Arguments

:: GenIO |
-> [String] | Predictor names.
-> String | Responder name.
-> Vector Measured |
-> ExceptT String Criterion Regression |

Regress the given predictors against the responder.

Errors may be returned under various circumstances, such as invalid names or lack of needed data.

See `olsRegress` for details of the regression performed.