ROOT 6.08/07 Reference Guide |
HypoTestResult is a base class for results from hypothesis tests.
Any tool inheriting from HypoTestCalculator can return a HypoTestResult. As such, it stores a p-value for the null-hypothesis (eg. background-only) and an alternate hypothesis (eg. signal+background). The p-values can also be transformed into confidence levels ( \(CL_{b}\), \(CL_{s+b}\)) in a trivial way. The ratio of the \(CL_{s+b}\) to \(CL_{b}\) is often called \(CL_{s}\), and is considered useful, though it is not a probability. Finally, the p-value of the null can be transformed into a number of equivalent Gaussian sigma using the Significance method.
The p-value of the null for a given test statistic is rigorously defined and this is the starting point for the following conventions.
The p-value for the null and alternate are on the same side of the observed value of the test statistic. This is the more standard convention and avoids confusion when doing inverted tests.
For exclusion, we also want the formula \(CL_{s} = CL_{s+b} / CL_{b}\) to hold which therefore defines our conventions for \(CL_{s+b}\) and \(CL_{b}\). \(CL_{s}\) was specifically invented for exclusion and therefore all quantities need be related through the assignments as they are for exclusion: \(CL_{s+b} = p_{s+b}\); \(CL_{b} = p_{b}\). This is derived by considering the scenarios of a powerful and not powerful inverted test, where for the not so powerful test, \(CL_{s}\) must be close to one.
For results of Hypothesis tests, \(CL_{s}\) has no similar direct interpretation as for exclusion and can be larger than one.
Definition at line 71 of file HypoTestResult.h.
Public Member Functions | |
HypoTestResult (const char *name=0) | |
default constructor More... | |
HypoTestResult (const HypoTestResult &other) | |
copy constructor More... | |
HypoTestResult (const char *name, Double_t nullp, Double_t altp) | |
constructor from name, null and alternate p values More... | |
virtual | ~HypoTestResult () |
destructor More... | |
virtual Double_t | AlternatePValue () const |
Return p-value for alternate hypothesis. More... | |
virtual void | Append (const HypoTestResult *other) |
add values from another HypoTestResult More... | |
virtual Double_t | CLb () const |
Convert NullPValue into a "confidence level". More... | |
Double_t | CLbError () const |
The error on the "confidence level" of the null hypothesis. More... | |
virtual Double_t | CLs () const |
\(CL_{s}\) is simply \(CL_{s+b}/CL_{b}\) (not a method, but a quantity) More... | |
Double_t | CLsError () const |
The error on the ratio \(CL_{s+b}/CL_{b}\). More... | |
virtual Double_t | CLsplusb () const |
Convert AlternatePValue into a "confidence level". More... | |
Double_t | CLsplusbError () const |
The error on the "confidence level" of the alternative hypothesis. More... | |
const RooArgList * | GetAllTestStatisticsData (void) const |
RooDataSet * | GetAltDetailedOutput (void) const |
SamplingDistribution * | GetAltDistribution (void) const |
Bool_t | GetBackGroundIsAlt (void) const |
RooDataSet * | GetFitInfo (void) const |
RooDataSet * | GetNullDetailedOutput (void) const |
SamplingDistribution * | GetNullDistribution (void) const |
Bool_t | GetPValueIsRightTail (void) const |
Double_t | GetTestStatisticData (void) const |
Bool_t | HasTestStatisticData (void) const |
virtual Double_t | NullPValue () const |
Return p-value for null hypothesis. More... | |
Double_t | NullPValueError () const |
The error on the Null p-value. More... | |
HypoTestResult & | operator= (const HypoTestResult &other) |
assignment operator More... | |
void | Print (const Option_t *="") const |
Print out some information about the results Note: use Alt/Null labels for the hypotheses here as the Null might be the s+b hypothesis. More... | |
void | SetAllTestStatisticsData (const RooArgList *tsd) |
void | SetAltDetailedOutput (RooDataSet *d) |
void | SetAltDistribution (SamplingDistribution *alt) |
void | SetBackgroundAsAlt (Bool_t l=kTRUE) |
void | SetFitInfo (RooDataSet *d) |
void | SetNullDetailedOutput (RooDataSet *d) |
void | SetNullDistribution (SamplingDistribution *null) |
void | SetPValueIsRightTail (Bool_t pr) |
void | SetTestStatisticData (const Double_t tsd) |
virtual Double_t | Significance () const |
familiar name for the Null p-value in terms of 1-sided Gaussian significance More... | |
Double_t | SignificanceError () const |
The error on the significance, computed from NullPValueError via error propagation. More... | |
Public Member Functions inherited from TNamed | |
TNamed () | |
TNamed (const char *name, const char *title) | |
TNamed (const TString &name, const TString &title) | |
TNamed (const TNamed &named) | |
TNamed copy ctor. More... | |
virtual | ~TNamed () |
virtual void | Clear (Option_t *option="") |
Set name and title to empty strings (""). More... | |
virtual TObject * | Clone (const char *newname="") const |
Make a clone of an object using the Streamer facility. More... | |
virtual Int_t | Compare (const TObject *obj) const |
Compare two TNamed objects. More... | |
virtual void | Copy (TObject &named) const |
Copy this to obj. More... | |
virtual void | FillBuffer (char *&buffer) |
Encode TNamed into output buffer. More... | |
virtual const char * | GetName () const |
Returns name of object. More... | |
virtual const char * | GetTitle () const |
Returns title of object. More... | |
virtual ULong_t | Hash () const |
Return hash value for this object. More... | |
virtual Bool_t | IsSortable () const |
virtual void | ls (Option_t *option="") const |
List TNamed name and title. More... | |
TNamed & | operator= (const TNamed &rhs) |
TNamed assignment operator. More... | |
virtual void | Print (Option_t *option="") const |
Print TNamed name and title. More... | |
virtual void | SetName (const char *name) |
Set the name of the TNamed. More... | |
virtual void | SetNameTitle (const char *name, const char *title) |
Set all the TNamed parameters (name and title). More... | |
virtual void | SetTitle (const char *title="") |
Set the title of the TNamed. More... | |
virtual Int_t | Sizeof () const |
Return size of the TNamed part of the TObject. More... | |
Public Member Functions inherited from TObject | |
TObject () | |
TObject constructor. More... | |
TObject (const TObject &object) | |
TObject copy ctor. More... | |
virtual | ~TObject () |
TObject destructor. More... | |
void | AbstractMethod (const char *method) const |
Use this method to implement an "abstract" method that you don't want to leave purely abstract. More... | |
virtual void | AppendPad (Option_t *option="") |
Append graphics object to current pad. More... | |
virtual void | Browse (TBrowser *b) |
Browse object. May be overridden for another default action. More... | |
virtual const char * | ClassName () const |
Returns name of class to which the object belongs. More... | |
virtual void | Delete (Option_t *option="") |
Delete this object. More... | |
virtual Int_t | DistancetoPrimitive (Int_t px, Int_t py) |
Computes distance from point (px,py) to the object. More... | |
virtual void | Draw (Option_t *option="") |
Default Draw method for all objects. More... | |
virtual void | DrawClass () const |
Draw class inheritance tree of the class to which this object belongs. More... | |
virtual TObject * | DrawClone (Option_t *option="") const |
Draw a clone of this object in the current pad. More... | |
virtual void | Dump () const |
Dump contents of object on stdout. More... | |
virtual void | Error (const char *method, const char *msgfmt,...) const |
Issue error message. More... | |
virtual void | Execute (const char *method, const char *params, Int_t *error=0) |
Execute method on this object with the given parameter string, e.g. More... | |
virtual void | Execute (TMethod *method, TObjArray *params, Int_t *error=0) |
Execute method on this object with parameters stored in the TObjArray. More... | |
virtual void | ExecuteEvent (Int_t event, Int_t px, Int_t py) |
Execute action corresponding to an event at (px,py). More... | |
virtual void | Fatal (const char *method, const char *msgfmt,...) const |
Issue fatal error message. More... | |
virtual TObject * | FindObject (const char *name) const |
Must be redefined in derived classes. More... | |
virtual TObject * | FindObject (const TObject *obj) const |
Must be redefined in derived classes. More... | |
virtual Option_t * | GetDrawOption () const |
Get option used by the graphics system to draw this object. More... | |
virtual const char * | GetIconName () const |
Returns mime type name of object. More... | |
virtual char * | GetObjectInfo (Int_t px, Int_t py) const |
Returns string containing info about the object at position (px,py). More... | |
virtual Option_t * | GetOption () const |
virtual UInt_t | GetUniqueID () const |
Return the unique object id. More... | |
virtual Bool_t | HandleTimer (TTimer *timer) |
Execute action in response of a timer timing out. More... | |
virtual void | Info (const char *method, const char *msgfmt,...) const |
Issue info message. More... | |
virtual Bool_t | InheritsFrom (const char *classname) const |
Returns kTRUE if object inherits from class "classname". More... | |
virtual Bool_t | InheritsFrom (const TClass *cl) const |
Returns kTRUE if object inherits from TClass cl. More... | |
virtual void | Inspect () const |
Dump contents of this object in a graphics canvas. More... | |
void | InvertBit (UInt_t f) |
virtual Bool_t | IsEqual (const TObject *obj) const |
Default equal comparison (objects are equal if they have the same address in memory). More... | |
virtual Bool_t | IsFolder () const |
Returns kTRUE in case object contains browsable objects (like containers or lists of other objects). More... | |
Bool_t | IsOnHeap () const |
Bool_t | IsZombie () const |
void | MayNotUse (const char *method) const |
Use this method to signal that a method (defined in a base class) may not be called in a derived class (in principle against good design since a child class should not provide less functionality than its parent, however, sometimes it is necessary). More... | |
virtual Bool_t | Notify () |
This method must be overridden to handle object notification. More... | |
void | Obsolete (const char *method, const char *asOfVers, const char *removedFromVers) const |
Use this method to declare a method obsolete. More... | |
void | operator delete (void *ptr) |
Operator delete. More... | |
void | operator delete[] (void *ptr) |
Operator delete []. More... | |
void * | operator new (size_t sz) |
void * | operator new (size_t sz, void *vp) |
void * | operator new[] (size_t sz) |
void * | operator new[] (size_t sz, void *vp) |
TObject & | operator= (const TObject &rhs) |
TObject assignment operator. More... | |
virtual void | Paint (Option_t *option="") |
This method must be overridden if a class wants to paint itself. More... | |
virtual void | Pop () |
Pop on object drawn in a pad to the top of the display list. More... | |
virtual Int_t | Read (const char *name) |
Read contents of object with specified name from the current directory. More... | |
virtual void | RecursiveRemove (TObject *obj) |
Recursively remove this object from a list. More... | |
void | ResetBit (UInt_t f) |
virtual void | SaveAs (const char *filename="", Option_t *option="") const |
Save this object in the file specified by filename. More... | |
virtual void | SavePrimitive (std::ostream &out, Option_t *option="") |
Save a primitive as a C++ statement(s) on output stream "out". More... | |
void | SetBit (UInt_t f, Bool_t set) |
Set or unset the user status bits as specified in f. More... | |
void | SetBit (UInt_t f) |
virtual void | SetDrawOption (Option_t *option="") |
Set drawing option for object. More... | |
virtual void | SetUniqueID (UInt_t uid) |
Set the unique object id. More... | |
virtual void | SysError (const char *method, const char *msgfmt,...) const |
Issue system error message. More... | |
Bool_t | TestBit (UInt_t f) const |
Int_t | TestBits (UInt_t f) const |
virtual void | UseCurrentStyle () |
Set current style settings in this object This function is called when either TCanvas::UseCurrentStyle or TROOT::ForceStyle have been invoked. More... | |
virtual void | Warning (const char *method, const char *msgfmt,...) const |
Issue warning message. More... | |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) |
Write this object to the current directory. More... | |
virtual Int_t | Write (const char *name=0, Int_t option=0, Int_t bufsize=0) const |
Write this object to the current directory. More... | |
Protected Attributes | |
const RooArgList * | fAllTestStatisticsData |
RooDataSet * | fAltDetailedOutput |
SamplingDistribution * | fAltDistr |
Double_t | fAlternatePValue |
Double_t | fAlternatePValueError |
Bool_t | fBackgroundIsAlt |
RooDataSet * | fFitInfo |
RooDataSet * | fNullDetailedOutput |
SamplingDistribution * | fNullDistr |
Double_t | fNullPValue |
Double_t | fNullPValueError |
Bool_t | fPValueIsRightTail |
Double_t | fTestStatisticData |
Protected Attributes inherited from TNamed | |
TString | fName |
TString | fTitle |
Private Member Functions | |
void | UpdatePValue (const SamplingDistribution *distr, Double_t &pvalue, Double_t &perror, Bool_t pIsRightTail) |
updates the pvalue if sufficient data is available More... | |
Additional Inherited Members | |
Public Types inherited from TObject | |
enum | { kIsOnHeap = 0x01000000, kNotDeleted = 0x02000000, kZombie = 0x04000000, kBitMask = 0x00ffffff } |
enum | { kSingleKey = BIT(0), kOverwrite = BIT(1), kWriteDelete = BIT(2) } |
enum | EStatusBits { kCanDelete = BIT(0), kMustCleanup = BIT(3), kObjInCanvas = BIT(3), kIsReferenced = BIT(4), kHasUUID = BIT(5), kCannotPick = BIT(6), kNoContextMenu = BIT(8), kInvalidObject = BIT(13) } |
Static Public Member Functions inherited from TObject | |
static Long_t | GetDtorOnly () |
Return destructor only flag. More... | |
static Bool_t | GetObjectStat () |
Get status of object stat flag. More... | |
static void | SetDtorOnly (void *obj) |
Set destructor only flag. More... | |
static void | SetObjectStat (Bool_t stat) |
Turn on/off tracking of objects in the TObjectTable. More... | |
Protected Member Functions inherited from TObject | |
virtual void | DoError (int level, const char *location, const char *fmt, va_list va) const |
Interface to ErrorHandler (protected). More... | |
void | MakeZombie () |
#include <RooStats/HypoTestResult.h>
|
explicit |
HypoTestResult::HypoTestResult | ( | const HypoTestResult & | other | ) |
copy constructor
Definition at line 72 of file HypoTestResult.cxx.
constructor from name, null and alternate p values
Alternate constructor.
Definition at line 56 of file HypoTestResult.cxx.
|
virtual |
|
inlinevirtual |
Return p-value for alternate hypothesis.
Reimplemented in RooStats::HybridResult.
Definition at line 97 of file HypoTestResult.h.
|
virtual |
add values from another HypoTestResult
Add additional toy-MC experiments to the current results.
Use the data test statistics of the added object if it is not already set (otherwise, ignore the new one).
Definition at line 134 of file HypoTestResult.cxx.
|
inlinevirtual |
Convert NullPValue into a "confidence level".
Definition at line 100 of file HypoTestResult.h.
Double_t HypoTestResult::CLbError | ( | ) | const |
The error on the "confidence level" of the null hypothesis.
compute \(CL_{b}\) error \(CL_{b}\) = 1 - NullPValue() must use opposite condition that routine above
Definition at line 236 of file HypoTestResult.cxx.
|
inlinevirtual |
\(CL_{s}\) is simply \(CL_{s+b}/CL_{b}\) (not a method, but a quantity)
Definition at line 106 of file HypoTestResult.h.
Double_t HypoTestResult::CLsError | ( | ) | const |
The error on the ratio \(CL_{s+b}/CL_{b}\).
Returns an estimate of the error on \(CL_{s}\) through combination of the errors on \(CL_{b}\) and \(CL_{s+b}\):
\[ \sigma_{CL_s} = CL_s \sqrt{\left( \frac{\sigma_{CL_{s+b}}}{CL_{s+b}} \right)^2 + \left( \frac{\sigma_{CL_{b}}}{CL_{b}} \right)^2} \]
.
Definition at line 261 of file HypoTestResult.cxx.
|
inlinevirtual |
Convert AlternatePValue into a "confidence level".
Definition at line 103 of file HypoTestResult.h.
Double_t HypoTestResult::CLsplusbError | ( | ) | const |
The error on the "confidence level" of the alternative hypothesis.
Definition at line 242 of file HypoTestResult.cxx.
|
inline |
Definition at line 125 of file HypoTestResult.h.
|
inline |
Definition at line 122 of file HypoTestResult.h.
|
inline |
Definition at line 120 of file HypoTestResult.h.
Definition at line 140 of file HypoTestResult.h.
|
inline |
Definition at line 123 of file HypoTestResult.h.
|
inline |
Definition at line 121 of file HypoTestResult.h.
|
inline |
Definition at line 119 of file HypoTestResult.h.
Definition at line 137 of file HypoTestResult.h.
Definition at line 124 of file HypoTestResult.h.
Definition at line 220 of file HypoTestResult.cxx.
|
inlinevirtual |
Return p-value for null hypothesis.
Reimplemented in RooStats::HybridResult.
Definition at line 94 of file HypoTestResult.h.
Double_t HypoTestResult::NullPValueError | ( | ) | const |
The error on the Null p-value.
Definition at line 226 of file HypoTestResult.cxx.
HypoTestResult & HypoTestResult::operator= | ( | const HypoTestResult & | other | ) |
assignment operator
Definition at line 103 of file HypoTestResult.cxx.
Print out some information about the results Note: use Alt/Null labels for the hypotheses here as the Null might be the s+b hypothesis.
Definition at line 302 of file HypoTestResult.cxx.
void HypoTestResult::SetAllTestStatisticsData | ( | const RooArgList * | tsd | ) |
Definition at line 196 of file HypoTestResult.cxx.
|
inline |
Definition at line 130 of file HypoTestResult.h.
void HypoTestResult::SetAltDistribution | ( | SamplingDistribution * | alt | ) |
Definition at line 173 of file HypoTestResult.cxx.
Definition at line 139 of file HypoTestResult.h.
|
inline |
Definition at line 132 of file HypoTestResult.h.
|
inline |
Definition at line 131 of file HypoTestResult.h.
void HypoTestResult::SetNullDistribution | ( | SamplingDistribution * | null | ) |
Definition at line 180 of file HypoTestResult.cxx.
Definition at line 211 of file HypoTestResult.cxx.
Definition at line 187 of file HypoTestResult.cxx.
|
inlinevirtual |
familiar name for the Null p-value in terms of 1-sided Gaussian significance
Definition at line 117 of file HypoTestResult.h.
Double_t HypoTestResult::SignificanceError | ( | ) | const |
The error on the significance, computed from NullPValueError via error propagation.
Taylor expansion series approximation for standard deviation (error propagation)
Definition at line 249 of file HypoTestResult.cxx.
|
private |
updates the pvalue if sufficient data is available
Definition at line 279 of file HypoTestResult.cxx.
|
protected |
Definition at line 171 of file HypoTestResult.h.
|
protected |
Definition at line 175 of file HypoTestResult.h.
|
protected |
Definition at line 173 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 167 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 169 of file HypoTestResult.h.
|
protected |
Definition at line 178 of file HypoTestResult.h.
|
protected |
Definition at line 176 of file HypoTestResult.h.
|
protected |
Definition at line 174 of file HypoTestResult.h.
|
protected |
Definition at line 172 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 166 of file HypoTestResult.h.
|
mutableprotected |
Definition at line 168 of file HypoTestResult.h.
|
protected |
Definition at line 177 of file HypoTestResult.h.
|
protected |
Definition at line 170 of file HypoTestResult.h.