Hypothesis Test Calculator based on the asymptotic formulae for the profile likelihood ratio.
It performs hypothesis tests using the asymptotic formula for the profile likelihood, and uses the Asimov data set to compute expected significances or limits.
See G. Cowan, K. Cranmer, E. Gross and O. Vitells: Asymptotic formulae for likelihood- based tests of new physics. Eur. Phys. J., C71:1–19, 2011. It provides methods to perform hypothesis tests using the likelihood function, and computes the \(p\)-values for the null and the alternate hypothesis using the asymptotic formulae for the profile likelihood ratio described in the given paper.
The calculator provides methods to produce the Asimov dataset, i.e. a dataset generated where the observed values are equal to the expected ones. The Asimov data set is then used to compute the observed asymptotic \(p\)-value for the alternate hypothesis and the asymptotic expected \(p\)-values.
The asymptotic formulae are valid only for one POI (parameter of interest). So the calculator works only for one-dimensional (one POI) models. If more than one POI exists, only the first one is used.
The calculator can generate Asimov datasets from two kinds of PDFs:
Definition at line 27 of file AsymptoticCalculator.h.
Public Member Functions | |
AsymptoticCalculator (RooAbsData &data, const ModelConfig &altModel, const ModelConfig &nullModel, bool nominalAsimov=false) | |
constructor for asymptotic calculator from Data set and ModelConfig | |
const RooArgSet & | GetBestFitParams () const |
return best fit value for all parameters | |
const RooArgSet & | GetBestFitPoi () const |
return snapshot of the best fit parameter | |
HypoTestResult * | GetHypoTest () const override |
re-implement HypoTest computation using the asymptotic | |
const RooRealVar * | GetMuHat () const |
return best fit parameter (firs of poi) | |
bool | Initialize () const |
initialize the calculator by performing a global fit and make the Asimov data set | |
TClass * | IsA () const override |
bool | IsOneSidedDiscovery () const |
bool | IsTwoSided () const |
void | SetAlternateModel (const ModelConfig &altModel) override |
Set the model for the alternate hypothesis. | |
void | SetData (RooAbsData &data) override |
Set the DataSet. | |
void | SetNullModel (const ModelConfig &nullModel) override |
re-implementation of setters since they are needed to re-initialize the calculator | |
void | SetOneSided (bool on) |
set test statistic for one sided (upper limits) | |
void | SetOneSidedDiscovery (bool on) |
set the test statistics for one-sided discovery | |
void | SetQTilde (bool on) |
set using of qtilde, by default is controlled if RoORealVar is limited or not | |
void | SetTwoSided () |
set the test statistics for two sided (in case of upper limits for discovery does not make really sense) | |
void | Streamer (TBuffer &) override |
void | StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b) |
Public Member Functions inherited from RooStats::HypoTestCalculatorGeneric | |
HypoTestCalculatorGeneric (const RooAbsData &data, const ModelConfig &altModel, const ModelConfig &nullModel, TestStatSampler *sampler=nullptr) | |
Constructor. | |
~HypoTestCalculatorGeneric () override | |
const ModelConfig * | GetAlternateModel (void) const |
const RooAbsData * | GetData (void) const |
virtual const RooArgSet * | GetFitInfo () const |
HypoTestResult * | GetHypoTest () const override |
inherited methods from HypoTestCalculator interface | |
const ModelConfig * | GetNullModel (void) const |
TestStatSampler * | GetTestStatSampler (void) const |
Returns instance of TestStatSampler. | |
TClass * | IsA () const override |
void | SetAlternateModel (const ModelConfig &altModel) override |
Set the model for the alternate hypothesis (S+B) | |
void | SetData (RooAbsData &data) override |
Set the DataSet. | |
void | SetNullModel (const ModelConfig &nullModel) override |
set the model for the null hypothesis (only B) | |
void | Streamer (TBuffer &) override |
void | StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b) |
void | UseSameAltToys () |
Set this for re-using always the same toys for alternate hypothesis in case of calls at different null parameter points This is useful to get more stable bands when running the HypoTest inversion. | |
Public Member Functions inherited from RooStats::HypoTestCalculator | |
virtual | ~HypoTestCalculator () |
virtual void | SetCommonModel (const ModelConfig &model) |
Set a common model for both the null and alternate, add to the workspace if not already there. | |
void | StreamerNVirtual (TBuffer &ClassDef_StreamerNVirtual_b) |
Static Public Member Functions | |
static TClass * | Class () |
static const char * | Class_Name () |
static constexpr Version_t | Class_Version () |
static const char * | DeclFileName () |
static RooAbsData * | GenerateAsimovData (const RooAbsPdf &pdf, const RooArgSet &observables) |
generate the asimov data for the observables (not the global ones) need to deal with the case of a sim pdf | |
static double | GetExpectedPValues (double pnull, double palt, double nsigma, bool usecls, bool oneSided=true) |
function given the null and the alt p value - return the expected one given the N - sigma value | |
static RooAbsData * | MakeAsimovData (const ModelConfig &model, const RooArgSet &allParamValues, RooArgSet &globObs) |
Make a nominal Asimov data set from a model. | |
static RooAbsData * | MakeAsimovData (RooAbsData &data, const ModelConfig &model, const RooArgSet &poiValues, RooArgSet &globObs, const RooArgSet *genPoiValues=nullptr) |
Make Asimov data. | |
static void | SetPrintLevel (int level) |
set print level (static function) | |
Static Public Member Functions inherited from RooStats::HypoTestCalculatorGeneric | |
static TClass * | Class () |
static const char * | Class_Name () |
static constexpr Version_t | Class_Version () |
static const char * | DeclFileName () |
Static Public Member Functions inherited from RooStats::HypoTestCalculator | |
static TClass * | Class () |
static const char * | Class_Name () |
static constexpr Version_t | Class_Version () |
static const char * | DeclFileName () |
Static Protected Member Functions | |
static double | EvaluateNLL (RooStats::ModelConfig const &modelConfig, RooAbsData &data, const RooArgSet *poiSet=nullptr) |
static void | FillBins (const RooAbsPdf &pdf, const RooArgList &obs, RooAbsData &data, int &index, double &binVolume, int &ibin) |
fill bins by looping recursively on observables | |
static RooAbsData * | GenerateAsimovDataSinglePdf (const RooAbsPdf &pdf, const RooArgSet &obs, const RooRealVar &weightVar, RooCategory *channelCat=nullptr) |
Compute the asimov data set for an observable of a pdf. | |
static RooAbsData * | GenerateCountingAsimovData (RooAbsPdf &pdf, const RooArgSet &obs, const RooRealVar &weightVar, RooCategory *channelCat=nullptr) |
Generate counting Asimov data for the case when the pdf cannot be extended. | |
static bool | SetObsToExpected (RooAbsPdf &pdf, const RooArgSet &obs) |
set observed value to the expected one works for Gaussian, Poisson or LogNormal assumes mean parameter value is the argument not constant and not depending on observables (if more than two arguments are not constant will use first one but print a warning !) need to iterate on the components of the Poisson to get n and nu (nu can be a RooAbsReal) (code from G. | |
static bool | SetObsToExpected (RooProdPdf &prod, const RooArgSet &obs) |
Inpspect a product pdf to find all the Poisson or Gaussian parts to set the observed values to expected ones. | |
Private Attributes | |
RooAbsData * | fAsimovData |
asimov data set | |
RooArgSet | fAsimovGlobObs |
snapshot of Asimov global observables | |
RooArgSet | fBestFitParams |
snapshot of all best fitted Parameter values | |
RooArgSet | fBestFitPoi |
snapshot of best fitted POI values | |
bool | fIsInitialized |
! flag to check if calculator is initialized | |
double | fNLLAsimov |
double | fNLLObs |
bool | fNominalAsimov |
make Asimov at nominal parameter values | |
bool | fOneSided |
for one sided PL test statistic (upper limits) | |
bool | fOneSidedDiscovery |
for one sided PL test statistic (for discovery) | |
int | fUseQTilde |
flag to indicate if using qtilde or not (-1 (default based on RooRealVar)), 0 false, 1 (true) | |
Static Private Attributes | |
static int | fgPrintLevel = 1 |
control print level (0 minimal, 1 normal, 2 debug) | |
Additional Inherited Members | |
Protected Member Functions inherited from RooStats::HypoTestCalculatorGeneric | |
virtual int | CheckHook (void) const |
virtual void | PostHook () const |
virtual int | PreAltHook (RooArgSet *, double) const |
virtual void | PreHook () const |
virtual int | PreNullHook (RooArgSet *, double) const |
Protected Attributes inherited from RooStats::HypoTestCalculatorGeneric | |
const ModelConfig * | fAltModel |
unsigned int | fAltToysSeed |
const RooAbsData * | fData |
TestStatSampler * | fDefaultSampler |
TestStatistic * | fDefaultTestStat |
const ModelConfig * | fNullModel |
TestStatSampler * | fTestStatSampler |
#include <RooStats/AsymptoticCalculator.h>
AsymptoticCalculator::AsymptoticCalculator | ( | RooAbsData & | data, |
const ModelConfig & | altModel, | ||
const ModelConfig & | nullModel, | ||
bool | nominalAsimov = false |
||
) |
constructor for asymptotic calculator from Data set and ModelConfig
Definition at line 92 of file AsymptoticCalculator.cxx.
|
static |
|
static |
|
inlinestaticconstexpr |
Definition at line 122 of file AsymptoticCalculator.h.
|
inlinestatic |
Definition at line 122 of file AsymptoticCalculator.h.
|
staticprotected |
Definition at line 290 of file AsymptoticCalculator.cxx.
|
staticprotected |
fill bins by looping recursively on observables
Definition at line 847 of file AsymptoticCalculator.cxx.
|
static |
generate the asimov data for the observables (not the global ones) need to deal with the case of a sim pdf
Definition at line 1128 of file AsymptoticCalculator.cxx.
|
staticprotected |
Compute the asimov data set for an observable of a pdf.
It generates binned data following the binning of the observables.
Definition at line 1049 of file AsymptoticCalculator.cxx.
|
staticprotected |
Generate counting Asimov data for the case when the pdf cannot be extended.
This function assumes that the pdf is a RooPoisson or can be decomposed in a product of RooPoisson, or is a RooGaussian. Otherwise, we cannot know how to make the Asimov data sets.
Definition at line 1010 of file AsymptoticCalculator.cxx.
|
inline |
return best fit value for all parameters
Definition at line 94 of file AsymptoticCalculator.h.
|
inline |
return snapshot of the best fit parameter
Definition at line 90 of file AsymptoticCalculator.h.
|
static |
function given the null and the alt p value - return the expected one given the N - sigma value
Definition at line 798 of file AsymptoticCalculator.cxx.
|
overridevirtual |
re-implement HypoTest computation using the asymptotic
It performs an hypothesis tests using the likelihood function and computes the p values for the null and the alternate using the asymptotic formulae for the profile likelihood ratio.
See G. Cowan, K. Cranmer, E. Gross and O. Vitells. Asymptotic formulae for likelihood- based tests of new physics. Eur. Phys. J., C71:1–19, 2011. The formulae are valid only for one POI. If more than one POI exists consider as POI only the first one
Implements RooStats::HypoTestCalculator.
Definition at line 463 of file AsymptoticCalculator.cxx.
|
inline |
return best fit parameter (firs of poi)
Definition at line 92 of file AsymptoticCalculator.h.
bool AsymptoticCalculator::Initialize | ( | ) | const |
initialize the calculator by performing a global fit and make the Asimov data set
Initialize the calculator The initialization will perform a global fit of the model to the data and build an Asimov data set.
It will then also fit the model to the Asimov data set to find the likelihood value of the Asimov data set nominalAsimov is an option for using Asimov data set obtained using nominal nuisance parameter values By default the nuisance parameters are fitted to the data NOTE: If a fit has been done before, one for speeding up could set all the initial parameters to the fit value and in addition set the null snapshot to the best fit
Definition at line 130 of file AsymptoticCalculator.cxx.
|
inlineoverridevirtual |
Reimplemented from RooStats::HypoTestCalculator.
Definition at line 122 of file AsymptoticCalculator.h.
|
inline |
Definition at line 83 of file AsymptoticCalculator.h.
|
inline |
Definition at line 82 of file AsymptoticCalculator.h.
|
static |
Make a nominal Asimov data set from a model.
model | ModelConfig that contains the model pdf and the model parameters | |
allParamValues | The parameters of the model will be set to the values given in this set | |
[out] | asimovGlobObs | Global observables set to values satisfying the constraints |
The parameter values (including the nuisance parameter) can result from a fit to data or be at the nominal values.
Definition at line 1325 of file AsymptoticCalculator.cxx.
|
static |
Make Asimov data.
Make the Asimov data from the ModelConfig and list of poi.
realData | Real data | |
model | Model config defining the pdf and the parameters | |
paramValues | The snapshot of POI and parameters used for finding the best nuisance parameter values (conditioned at these values) | |
[out] | asimovGlobObs | Global observables set to values satisfying the constraints |
genPoiValues | Optional. A different set of POI values used for generating. By default the same POI are used for generating and for finding the nuisance parameters given an observed data set, a model and a snapshot of the poi. |
Definition at line 1212 of file AsymptoticCalculator.cxx.
|
inlineoverridevirtual |
Set the model for the alternate hypothesis.
Implements RooStats::HypoTestCalculator.
Definition at line 72 of file AsymptoticCalculator.h.
|
inlineoverridevirtual |
Set the DataSet.
Implements RooStats::HypoTestCalculator.
Definition at line 76 of file AsymptoticCalculator.h.
|
inlineoverridevirtual |
re-implementation of setters since they are needed to re-initialize the calculator
Implements RooStats::HypoTestCalculator.
Definition at line 68 of file AsymptoticCalculator.h.
|
staticprotected |
set observed value to the expected one works for Gaussian, Poisson or LogNormal assumes mean parameter value is the argument not constant and not depending on observables (if more than two arguments are not constant will use first one but print a warning !) need to iterate on the components of the Poisson to get n and nu (nu can be a RooAbsReal) (code from G.
Petrucciani and extended by L.M.)
Definition at line 957 of file AsymptoticCalculator.cxx.
|
staticprotected |
Inpspect a product pdf to find all the Poisson or Gaussian parts to set the observed values to expected ones.
Definition at line 919 of file AsymptoticCalculator.cxx.
|
inline |
set test statistic for one sided (upper limits)
Definition at line 58 of file AsymptoticCalculator.h.
|
inline |
set the test statistics for one-sided discovery
Definition at line 65 of file AsymptoticCalculator.h.
|
static |
set print level (static function)
Definition at line 85 of file AsymptoticCalculator.cxx.
|
inline |
set using of qtilde, by default is controlled if RoORealVar is limited or not
Definition at line 87 of file AsymptoticCalculator.h.
|
inline |
set the test statistics for two sided (in case of upper limits for discovery does not make really sense)
Definition at line 62 of file AsymptoticCalculator.h.
|
overridevirtual |
Reimplemented from RooStats::HypoTestCalculator.
|
inline |
Definition at line 122 of file AsymptoticCalculator.h.
|
mutableprivate |
asimov data set
Definition at line 135 of file AsymptoticCalculator.h.
|
mutableprivate |
snapshot of Asimov global observables
Definition at line 136 of file AsymptoticCalculator.h.
|
mutableprivate |
snapshot of all best fitted Parameter values
Definition at line 138 of file AsymptoticCalculator.h.
|
mutableprivate |
snapshot of best fitted POI values
Definition at line 137 of file AsymptoticCalculator.h.
|
staticprivate |
control print level (0 minimal, 1 normal, 2 debug)
Definition at line 131 of file AsymptoticCalculator.h.
|
mutableprivate |
! flag to check if calculator is initialized
Definition at line 129 of file AsymptoticCalculator.h.
|
mutableprivate |
Definition at line 133 of file AsymptoticCalculator.h.
|
mutableprivate |
Definition at line 132 of file AsymptoticCalculator.h.
|
private |
make Asimov at nominal parameter values
Definition at line 128 of file AsymptoticCalculator.h.
|
private |
for one sided PL test statistic (upper limits)
Definition at line 126 of file AsymptoticCalculator.h.
|
mutableprivate |
for one sided PL test statistic (for discovery)
Definition at line 127 of file AsymptoticCalculator.h.
|
mutableprivate |
flag to indicate if using qtilde or not (-1 (default based on RooRealVar)), 0 false, 1 (true)
Definition at line 130 of file AsymptoticCalculator.h.