Hypothesis Test Calculator based on the asymptotic formulae for the profile likelihood ratio.

It performs hypothesis tests using the asymptotic formula for the profile likelihood, and uses the Asimov data set to compute expected significances or limits.

See G. Cowan, K. Cranmer, E. Gross and O. Vitells: Asymptotic formulae for likelihood- based tests of new physics. Eur. Phys. J., C71:1–19, 2011. It provides methods to perform hypothesis tests using the likelihood function, and computes the \(p\)-values for the null and the alternate hypothesis using the asymptotic formulae for the profile likelihood ratio described in the given paper.

The calculator provides methods to produce the Asimov dataset, i.e. a dataset generated where the observed values are equal to the expected ones. The Asimov data set is then used to compute the observed asymptotic \(p\)-value for the alternate hypothesis and the asymptotic expected \(p\)-values.

The asymptotic formulae are valid only for one POI (parameter of interest). So the calculator works only for one-dimensional (one POI) models. If more than one POI exists, only the first one is used.

The calculator can generate Asimov datasets from two kinds of PDFs:

"Counting" distributions: RooPoisson, RooGaussian, or products of RooPoissons.
Extended, i.e. number of events can be read off from extended likelihood term.

Definition at line 27 of file AsymptoticCalculator.h.

Public Member Functions
	AsymptoticCalculator (RooAbsData &data, const ModelConfig &altModel, const ModelConfig &nullModel, bool nominalAsimov=false)
	constructor for asymptotic calculator from Data set and ModelConfig More...

	~AsymptoticCalculator ()

const RooArgSet &	GetBestFitParams () const
	return best fit value for all parameters More...

const RooArgSet &	GetBestFitPoi () const
	return snapshot of the best fit parameter More...

virtual HypoTestResult *	GetHypoTest () const
	re-implement HypoTest computation using the asymptotic More...

const RooRealVar *	GetMuHat () const
	return best fit parameter (firs of poi) More...

bool	Initialize () const
	initialize the calculator by performing a global fit and make the Asimov data set More...

bool	IsOneSidedDiscovery () const

bool	IsTwoSided () const

virtual void	SetAlternateModel (const ModelConfig &altModel)
	Set the model for the alternate hypothesis (S+B) More...

virtual void	SetData (RooAbsData &data)
	Set the DataSet. More...

virtual void	SetNullModel (const ModelConfig &nullModel)
	re-implementation of setters since they are needed to re-initialize the calculator More...

void	SetOneSided (bool on)
	set test statistic for one sided (upper limits) More...

void	SetOneSidedDiscovery (bool on)
	set the test statistics for one-sided discovery More...

void	SetQTilde (bool on)
	set using of qtilde, by default is controlled if RoORealVar is limited or not More...

void	SetTwoSided ()
	set the test statistics for two sided (in case of upper limits for discovery does not make really sense) More...

Public Member Functions inherited from RooStats::HypoTestCalculatorGeneric
	HypoTestCalculatorGeneric (const RooAbsData &data, const ModelConfig &altModel, const ModelConfig &nullModel, TestStatSampler *sampler=0)
	Constructor. More...

	~HypoTestCalculatorGeneric ()

const ModelConfig *	GetAlternateModel (void) const

const RooAbsData *	GetData (void) const

virtual const RooArgSet *	GetFitInfo () const

virtual HypoTestResult *	GetHypoTest () const
	inherited methods from HypoTestCalculator interface More...

const ModelConfig *	GetNullModel (void) const

TestStatSampler *	GetTestStatSampler (void) const
	Returns instance of TestStatSampler. More...

virtual void	SetAlternateModel (const ModelConfig &altModel)
	Set the model for the alternate hypothesis (S+B) More...

virtual void	SetData (RooAbsData &data)
	Set the DataSet. More...

virtual void	SetNullModel (const ModelConfig &nullModel)

void	UseSameAltToys ()
	Set this for re-using always the same toys for alternate hypothesis in case of calls at different null parameter points This is useful to get more stable bands when running the HypoTest inversion. More...

Public Member Functions inherited from RooStats::HypoTestCalculator
virtual	~HypoTestCalculator ()

virtual HypoTestResult *	GetHypoTest () const =0

virtual void	SetAlternateModel (const ModelConfig &model)=0

virtual void	SetCommonModel (const ModelConfig &model)

virtual void	SetData (RooAbsData &data)=0

virtual void	SetNullModel (const ModelConfig &model)=0

Static Public Member Functions
static RooAbsData *	GenerateAsimovData (const RooAbsPdf &pdf, const RooArgSet &observables)
	generate the asimov data for the observables (not the global ones) need to deal with the case of a sim pdf More...

static double	GetExpectedPValues (double pnull, double palt, double nsigma, bool usecls, bool oneSided=true)
	function given the null and the alt p value - return the expected one given the N - sigma value More...

static RooAbsData *	MakeAsimovData (const ModelConfig &model, const RooArgSet &allParamValues, RooArgSet &globObs)
	Make a nominal Asimov data set from a model. More...

static RooAbsData *	MakeAsimovData (RooAbsData &data, const ModelConfig &model, const RooArgSet &poiValues, RooArgSet &globObs, const RooArgSet *genPoiValues=0)
	Make Asimov data. More...

static void	SetPrintLevel (int level)
	set print level (static function) More...

Static Protected Member Functions
static double	EvaluateNLL (RooAbsPdf &pdf, RooAbsData &data, const RooArgSet condObs, const RooArgSet globObs, const RooArgSet *poiSet=0)

static void	FillBins (const RooAbsPdf &pdf, const RooArgList &obs, RooAbsData &data, int &index, double &binVolume, int &ibin)
	fill bins by looping recursively on observables More...

static RooAbsData *	GenerateAsimovDataSinglePdf (const RooAbsPdf &pdf, const RooArgSet &obs, const RooRealVar &weightVar, RooCategory *channelCat=0)
	Compute the asimov data set for an observable of a pdf. More...

static RooAbsData *	GenerateCountingAsimovData (RooAbsPdf &pdf, const RooArgSet &obs, const RooRealVar &weightVar, RooCategory *channelCat=0)
	Generate counting Asimov data for the case when the pdf cannot be extended. More...

static bool	SetObsToExpected (RooAbsPdf &pdf, const RooArgSet &obs)
	set observed value to the expected one works for Gaussian, Poisson or LogNormal assumes mean parameter value is the argument not constant and not depending on observables (if more than two arguments are not constant will use first one but print a warning !) need to iterate on the components of the Poisson to get n and nu (nu can be a RooAbsReal) (code from G. More...

static bool	SetObsToExpected (RooProdPdf &prod, const RooArgSet &obs)
	Inpspect a product pdf to find all the Poisson or Gaussian parts to set the observed values to expected ones. More...

Private Attributes
RooAbsData *	fAsimovData

RooArgSet	fAsimovGlobObs

RooArgSet	fBestFitParams

RooArgSet	fBestFitPoi

bool	fIsInitialized

double	fNLLAsimov

double	fNLLObs

bool	fNominalAsimov

bool	fOneSided

bool	fOneSidedDiscovery

int	fUseQTilde
	flag to check if calculator is initialized More...

Static Private Attributes
static int	fgPrintLevel = 1

Additional Inherited Members
Protected Member Functions inherited from RooStats::HypoTestCalculatorGeneric
virtual int	CheckHook (void) const

virtual void	PostHook () const

virtual int	PreAltHook (RooArgSet *, double) const

virtual void	PreHook () const

virtual int	PreNullHook (RooArgSet *, double) const

Protected Attributes inherited from RooStats::HypoTestCalculatorGeneric
const ModelConfig *	fAltModel

unsigned int	fAltToysSeed

const RooAbsData *	fData

TestStatSampler *	fDefaultSampler

TestStatistic *	fDefaultTestStat

const ModelConfig *	fNullModel

TestStatSampler *	fTestStatSampler

#include <RooStats/AsymptoticCalculator.h>

Inheritance diagram for RooStats::AsymptoticCalculator:

[legend]

Constructor & Destructor Documentation

◆ AsymptoticCalculator()

AsymptoticCalculator::AsymptoticCalculator	(	RooAbsData &	data,
		const ModelConfig &	altModel,
		const ModelConfig &	nullModel,
		bool	nominalAsimov = `false`
	)

constructor for asymptotic calculator from Data set and ModelConfig

Definition at line 94 of file AsymptoticCalculator.cxx.

◆ ~AsymptoticCalculator()

RooStats::AsymptoticCalculator::~AsymptoticCalculator ( )

inline

Definition at line 40 of file AsymptoticCalculator.h.

Member Function Documentation

◆ EvaluateNLL()

Double_t AsymptoticCalculator::EvaluateNLL	(	RooAbsPdf &	pdf,
		RooAbsData &	data,
		const RooArgSet *	condObs,
		const RooArgSet *	globObs,
		const RooArgSet *	poiSet = `0`
	)

staticprotected

Definition at line 291 of file AsymptoticCalculator.cxx.

◆ FillBins()

void AsymptoticCalculator::FillBins	(	const RooAbsPdf &	pdf,
		const RooArgList &	obs,
		RooAbsData &	data,
		int &	index,
		double &	binVolume,
		int &	ibin
	)

staticprotected

fill bins by looping recursively on observables

Definition at line 857 of file AsymptoticCalculator.cxx.

◆ GenerateAsimovData()

RooAbsData * AsymptoticCalculator::GenerateAsimovData	(	const RooAbsPdf &	pdf,
		const RooArgSet &	observables
	)

static

generate the asimov data for the observables (not the global ones) need to deal with the case of a sim pdf

Definition at line 1132 of file AsymptoticCalculator.cxx.

◆ GenerateAsimovDataSinglePdf()

RooAbsData * AsymptoticCalculator::GenerateAsimovDataSinglePdf	(	const RooAbsPdf &	pdf,
		const RooArgSet &	allobs,
		const RooRealVar &	weightVar,
		RooCategory *	channelCat = `0`
	)

staticprotected

Compute the asimov data set for an observable of a pdf.

It generates binned data following the binning of the observables.

Definition at line 1054 of file AsymptoticCalculator.cxx.

◆ GenerateCountingAsimovData()

RooAbsData * AsymptoticCalculator::GenerateCountingAsimovData	(	RooAbsPdf &	pdf,
		const RooArgSet &	observables,
		const RooRealVar &	weightVar,
		RooCategory *	channelCat = `0`
	)

staticprotected

Generate counting Asimov data for the case when the pdf cannot be extended.

This function assumes that the pdf is a RooPoisson or can be decomposed in a product of RooPoisson, or is a RooGaussian. Otherwise, we cannot know how to make the Asimov data sets.

Definition at line 1016 of file AsymptoticCalculator.cxx.

◆ GetBestFitParams()

const RooArgSet & RooStats::AsymptoticCalculator::GetBestFitParams ( ) const

inline

return best fit value for all parameters

Definition at line 100 of file AsymptoticCalculator.h.

◆ GetBestFitPoi()

const RooArgSet & RooStats::AsymptoticCalculator::GetBestFitPoi ( ) const

inline

return snapshot of the best fit parameter

Definition at line 96 of file AsymptoticCalculator.h.

◆ GetExpectedPValues()

double AsymptoticCalculator::GetExpectedPValues	(	double	pnull,
		double	palt,
		double	nsigma,
		bool	usecls,
		bool	oneSided = `true`
	)

static

function given the null and the alt p value - return the expected one given the N - sigma value

Definition at line 808 of file AsymptoticCalculator.cxx.

◆ GetHypoTest()

HypoTestResult * AsymptoticCalculator::GetHypoTest ( ) const

virtual

re-implement HypoTest computation using the asymptotic

It performs an hypothesis tests using the likelihood function and computes the p values for the null and the alternate using the asymptotic formulae for the profile likelihood ratio.

See G. Cowan, K. Cranmer, E. Gross and O. Vitells. Asymptotic formulae for likelihood- based tests of new physics. Eur. Phys. J., C71:1–19, 2011. The formulae are valid only for one POI. If more than one POI exists consider as POI only the first one

Reimplemented from RooStats::HypoTestCalculatorGeneric.

Definition at line 476 of file AsymptoticCalculator.cxx.

◆ GetMuHat()

const RooRealVar * RooStats::AsymptoticCalculator::GetMuHat ( ) const

inline

return best fit parameter (firs of poi)

Definition at line 98 of file AsymptoticCalculator.h.

◆ Initialize()

bool AsymptoticCalculator::Initialize ( ) const

initialize the calculator by performing a global fit and make the Asimov data set

Initialize the calculator The initialization will perform a global fit of the model to the data and build an Asimov data set.

It will then also fit the model to the Asimov data set to find the likelihood value of the Asimov data set nominalAsimov is an option for using Asimov data set obtained using nominal nuisance parameter values By default the nuisance parameters are fitted to the data NOTE: If a fit has been done before, one for speeding up could set all the initial parameters to the fit value and in addition set the null snapshot to the best fit

Definition at line 132 of file AsymptoticCalculator.cxx.

◆ IsOneSidedDiscovery()

bool RooStats::AsymptoticCalculator::IsOneSidedDiscovery ( ) const

inline

Definition at line 89 of file AsymptoticCalculator.h.

◆ IsTwoSided()

bool RooStats::AsymptoticCalculator::IsTwoSided ( ) const

inline

Definition at line 88 of file AsymptoticCalculator.h.

◆ MakeAsimovData() [1/2]

RooAbsData * AsymptoticCalculator::MakeAsimovData	(	const ModelConfig &	model,
		const RooArgSet &	allParamValues,
		RooArgSet &	asimovGlobObs
	)

static

Make a nominal Asimov data set from a model.

Parameters

	model	ModelConfig that contains the model pdf and the model parameters
	allParamValues	The parameters fo the model will be set to the values given in this set
[out]	asimovGlobObs	Global observables set to values satisfying the constraints

Returns: Asimov data set. The user takes ownership.

The parameter values (including the nuisance parameter) can result from a fit to data or be at the nominal values.

Definition at line 1337 of file AsymptoticCalculator.cxx.

◆ MakeAsimovData() [2/2]

RooAbsData * AsymptoticCalculator::MakeAsimovData	(	RooAbsData &	realData,
		const ModelConfig &	model,
		const RooArgSet &	paramValues,
		RooArgSet &	asimovGlobObs,
		const RooArgSet *	genPoiValues = `0`
	)

static

Make Asimov data.

Make the Asimov data from the ModelConfig and list of poi.

Parameters

	realData	Real data
	model	Model config defining the pdf and the parameters
	paramValues	The snapshot of POI and parameters used for finding the best nuisance parameter values (conditioned at these values)
[out]	asimovGlobObs	Global observables set to values satisfying the constraints
	genPoiValues	Optional. A different set of POI values used for generating. By default the same POI are used for generating and for finding the nuisance parameters given an observed data set, a model and a snapshot of the poi.

Returns: The asimov data set. The user takes ownership.

Definition at line 1216 of file AsymptoticCalculator.cxx.

◆ SetAlternateModel()

virtual void RooStats::AsymptoticCalculator::SetAlternateModel ( const ModelConfig & altModel )

inlinevirtual

Set the model for the alternate hypothesis (S+B)

Reimplemented from RooStats::HypoTestCalculatorGeneric.

Definition at line 78 of file AsymptoticCalculator.h.

◆ SetData()

virtual void RooStats::AsymptoticCalculator::SetData ( RooAbsData & data )

inlinevirtual

Set the DataSet.

Reimplemented from RooStats::HypoTestCalculatorGeneric.

Definition at line 82 of file AsymptoticCalculator.h.

◆ SetNullModel()

virtual void RooStats::AsymptoticCalculator::SetNullModel ( const ModelConfig & nullModel )

inlinevirtual

re-implementation of setters since they are needed to re-initialize the calculator

Reimplemented from RooStats::HypoTestCalculatorGeneric.

Definition at line 74 of file AsymptoticCalculator.h.

◆ SetObsToExpected() [1/2]

bool AsymptoticCalculator::SetObsToExpected	(	RooAbsPdf &	pdf,
		const RooArgSet &	obs
	)

staticprotected

set observed value to the expected one works for Gaussian, Poisson or LogNormal assumes mean parameter value is the argument not constant and not depending on observables (if more than two arguments are not constant will use first one but print a warning !) need to iterate on the components of the Poisson to get n and nu (nu can be a RooAbsReal) (code from G.

Petrucciani and extended by L.M.)

Definition at line 962 of file AsymptoticCalculator.cxx.

◆ SetObsToExpected() [2/2]

bool AsymptoticCalculator::SetObsToExpected	(	RooProdPdf &	prod,
		const RooArgSet &	obs
	)

staticprotected

Inpspect a product pdf to find all the Poisson or Gaussian parts to set the observed values to expected ones.

Definition at line 923 of file AsymptoticCalculator.cxx.

◆ SetOneSided()

void RooStats::AsymptoticCalculator::SetOneSided ( bool on )

inline

set test statistic for one sided (upper limits)

Definition at line 64 of file AsymptoticCalculator.h.

◆ SetOneSidedDiscovery()

void RooStats::AsymptoticCalculator::SetOneSidedDiscovery ( bool on )

inline

set the test statistics for one-sided discovery

Definition at line 71 of file AsymptoticCalculator.h.

◆ SetPrintLevel()

void AsymptoticCalculator::SetPrintLevel ( int level )

static

set print level (static function)

0 minimal,
1 normal,
2 debug

Definition at line 87 of file AsymptoticCalculator.cxx.

◆ SetQTilde()

void RooStats::AsymptoticCalculator::SetQTilde ( bool on )

inline

set using of qtilde, by default is controlled if RoORealVar is limited or not

Definition at line 93 of file AsymptoticCalculator.h.

◆ SetTwoSided()

void RooStats::AsymptoticCalculator::SetTwoSided ( )

inline

set the test statistics for two sided (in case of upper limits for discovery does not make really sense)

Definition at line 68 of file AsymptoticCalculator.h.

Member Data Documentation

◆ fAsimovData

RooAbsData* RooStats::AsymptoticCalculator::fAsimovData

mutableprivate

Definition at line 141 of file AsymptoticCalculator.h.

◆ fAsimovGlobObs

RooArgSet RooStats::AsymptoticCalculator::fAsimovGlobObs

mutableprivate

Definition at line 142 of file AsymptoticCalculator.h.

◆ fBestFitParams

RooArgSet RooStats::AsymptoticCalculator::fBestFitParams

mutableprivate

Definition at line 144 of file AsymptoticCalculator.h.

◆ fBestFitPoi

RooArgSet RooStats::AsymptoticCalculator::fBestFitPoi

mutableprivate

Definition at line 143 of file AsymptoticCalculator.h.

◆ fgPrintLevel

int AsymptoticCalculator::fgPrintLevel = 1

staticprivate

Definition at line 137 of file AsymptoticCalculator.h.

◆ fIsInitialized

bool RooStats::AsymptoticCalculator::fIsInitialized

mutableprivate

Definition at line 135 of file AsymptoticCalculator.h.

◆ fNLLAsimov

double RooStats::AsymptoticCalculator::fNLLAsimov

mutableprivate

Definition at line 139 of file AsymptoticCalculator.h.

◆ fNLLObs

double RooStats::AsymptoticCalculator::fNLLObs

mutableprivate

Definition at line 138 of file AsymptoticCalculator.h.

◆ fNominalAsimov

bool RooStats::AsymptoticCalculator::fNominalAsimov

private

Definition at line 134 of file AsymptoticCalculator.h.

◆ fOneSided

bool RooStats::AsymptoticCalculator::fOneSided

private

Definition at line 132 of file AsymptoticCalculator.h.

◆ fOneSidedDiscovery

bool RooStats::AsymptoticCalculator::fOneSidedDiscovery

mutableprivate

Definition at line 133 of file AsymptoticCalculator.h.

◆ fUseQTilde

int RooStats::AsymptoticCalculator::fUseQTilde

mutableprivate

flag to check if calculator is initialized

Definition at line 136 of file AsymptoticCalculator.h.

Libraries for RooStats::AsymptoticCalculator:

[legend]

Public Member Functions

Static Public Member Functions

Static Protected Member Functions

Private Attributes

Static Private Attributes

Additional Inherited Members

Constructor & Destructor Documentation

◆ AsymptoticCalculator()

◆ ~AsymptoticCalculator()

Member Function Documentation

◆ EvaluateNLL()

◆ FillBins()

◆ GenerateAsimovData()

◆ GenerateAsimovDataSinglePdf()

◆ GenerateCountingAsimovData()

◆ GetBestFitParams()

◆ GetBestFitPoi()

◆ GetExpectedPValues()

◆ GetHypoTest()

◆ GetMuHat()

◆ Initialize()

◆ IsOneSidedDiscovery()

◆ IsTwoSided()

◆ MakeAsimovData() [1/2]

◆ MakeAsimovData() [2/2]

◆ SetAlternateModel()

◆ SetData()

◆ SetNullModel()

◆ SetObsToExpected() [1/2]

◆ SetObsToExpected() [2/2]

◆ SetOneSided()

◆ SetOneSidedDiscovery()

◆ SetPrintLevel()

◆ SetQTilde()

◆ SetTwoSided()

Member Data Documentation

◆ fAsimovData

◆ fAsimovGlobObs

◆ fBestFitParams

◆ fBestFitPoi

◆ fgPrintLevel

◆ fIsInitialized

◆ fNLLAsimov

◆ fNLLObs

◆ fNominalAsimov

◆ fOneSided

◆ fOneSidedDiscovery

◆ fUseQTilde