Logo ROOT   6.14/05
Reference Guide
rs101_limitexample.C File Reference
Tutorials » RooStats Tutorials

Detailed Description

View in nbviewer Open in SWAN 'Limit Example' RooStats tutorial macro #101 This tutorial shows an example of creating a simple model for a number counting experiment with uncertainty on both the background rate and signal efficiency.

We then use a Confidence Interval Calculator to set a limit on the signal.

pict1_rs101_limitexample.C.png
Processing /mnt/build/workspace/root-makedoc-v614/rootspi/rdoc/src/v6-14-00-patches/tutorials/roostats/rs101_limitexample.C...
RooFit v3.60 -- Developed by Wouter Verkerke and David Kirkby
Copyright (C) 2000-2013 NIKHEF, University of California & Stanford University
All rights reserved, please read http://roofit.sourceforge.net/license.txt
RooWorkspace() contents
variables
---------
(b,gSigBkg,gSigEff,obs,ratioBkgEff,ratioSigEff,s)
p.d.f.s
-------
RooGaussian::bkgConstraint[ x=gSigBkg mean=ratioBkgEff sigma=0.2 ] = 1
RooPoisson::countingModel[ x=obs mean=countingModel_2 ] = 0.0325554
RooProdPdf::modelWithConstraints[ countingModel * sigConstraint * bkgConstraint ] = 0.0325554
RooGaussian::sigConstraint[ x=gSigEff mean=ratioSigEff sigma=0.05 ] = 1
functions
--------
RooAddition::countingModel_2[ countingModel_2_[s_x_ratioSigEff] + countingModel_2_[b_x_ratioBkgEff] ] = 150
RooProduct::countingModel_2_[b_x_ratioBkgEff][ b * ratioBkgEff ] = 100
RooProduct::countingModel_2_[s_x_ratioSigEff][ s * ratioSigEff ] = 50
[#1] INFO:Minization -- createNLL: caching constraint set under name CONSTR_OF_PDF_modelWithConstraints_FOR_OBS_obs with 2 entries
[#1] INFO:Minization -- Including the following contraint terms in minimization: (sigConstraint,bkgConstraint)
[#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_modelWithConstraints_exampleData_with_constr) Summation contains a RooNLLVar, using its error level
[#1] INFO:Minization -- RooMinimizer::optimizeConst: activating const optimization
[#1] INFO:Minization -- The following expressions will be evaluated in cache-and-track mode: (countingModel)
**********
** 1 **SET PRINT 1
**********
**********
** 2 **SET NOGRAD
**********
PARAMETER DEFINITIONS:
NO. NAME VALUE STEP SIZE LIMITS
1 ratioBkgEff 1.00000e+00 3.00000e-01 0.00000e+00 3.00000e+00
2 ratioSigEff 1.00000e+00 3.00000e-01 0.00000e+00 3.00000e+00
3 s 5.00000e+01 1.20000e+01 0.00000e+00 1.20000e+02
**********
** 3 **SET ERR 0.5
**********
**********
** 4 **SET PRINT 1
**********
**********
** 5 **SET STR 1
**********
NOW USING STRATEGY 1: TRY TO BALANCE SPEED AGAINST RELIABILITY
**********
** 6 **MIGRAD 1500 1
**********
FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
START MIGRAD MINIMIZATION. STRATEGY 1. CONVERGENCE WHEN EDM .LT. 1.00e-03
FCN=1.01592 FROM MIGRAD STATUS=INITIATE 12 CALLS 13 TOTAL
EDM= unknown STRATEGY= 1 NO ERROR MATRIX
EXT PARAMETER CURRENT GUESS STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 ratioBkgEff 1.00000e+00 3.00000e-01 2.14402e-01 -9.42809e+00
2 ratioSigEff 1.00000e+00 3.00000e-01 2.14402e-01 -4.71404e+00
3 s 5.00000e+01 1.20000e+01 2.04382e-01 -3.94405e+00
ERR DEF= 0.5
MIGRAD MINIMIZATION HAS CONVERGED.
MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=0.689753 FROM MIGRAD STATUS=CONVERGED 60 CALLS 61 TOTAL
EDM=4.05791e-08 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 ratioBkgEff 1.00002e+00 1.99284e-01 5.69545e-05 2.44292e-04
2 ratioSigEff 9.99992e-01 4.99900e-02 2.59106e-05 -5.01928e-03
3 s 5.99979e+01 2.32251e+01 1.58945e-04 -2.07489e-04
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 3 ERR DEF=0.5
3.998e-02 2.485e-07 -3.998e+00
2.485e-07 2.500e-03 -1.500e-01
-3.998e+00 -1.500e-01 5.687e+02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3
1 0.84507 1.000 0.000 -0.838
2 0.23079 0.000 1.000 -0.126
3 0.84774 -0.838 -0.126 1.000
**********
** 7 **SET ERR 0.5
**********
**********
** 8 **SET PRINT 1
**********
**********
** 9 **HESSE 1500
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=0.689753 FROM HESSE STATUS=OK 16 CALLS 77 TOTAL
EDM=4.05843e-08 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 ratioBkgEff 1.00002e+00 1.99326e-01 1.13909e-05 -3.39822e-01
2 ratioSigEff 9.99992e-01 4.99895e-02 5.18211e-06 -3.39843e-01
3 s 5.99979e+01 2.32296e+01 3.17891e-05 -3.54380e-05
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 3 ERR DEF=0.5
4.000e-02 5.038e-08 -4.000e+00
5.038e-08 2.500e-03 -1.500e-01
-4.000e+00 -1.500e-01 5.690e+02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3
1 0.84514 1.000 0.000 -0.838
2 0.23076 0.000 1.000 -0.126
3 0.84781 -0.838 -0.126 1.000
[#1] INFO:Minization -- RooMinimizer::optimizeConst: deactivating const optimization
[#1] INFO:ObjectHandling -- RooWorkspace::import() importing dataset exampleData
[#1] INFO:Minization -- createNLL picked up cached consraints from workspace with 2 entries
[#1] INFO:Minization -- Including the following contraint terms in minimization: (sigConstraint,bkgConstraint)
[#1] INFO:Minization -- The following global observables have been defined: (gSigEff,gSigBkg)
[#0] PROGRESS:Minization -- ProfileLikelihoodCalcultor::DoGLobalFit - find MLE
[#0] PROGRESS:Minization -- ProfileLikelihoodCalcultor::DoMinimizeNLL - using Minuit / Migrad with strategy 1
[#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_modelWithConstraints_exampleData_with_constr) Summation contains a RooNLLVar, using its error level
[#1] INFO:Minization -- RooMinimizer::optimizeConst: activating const optimization
[#1] INFO:Minization -- The following expressions will be evaluated in cache-and-track mode: (countingModel)
[#1] INFO:Minization --
RooFitResult: minimized FCN value: 0.689753, estimated distance to minimum: 9.56454e-09
covariance matrix quality: Full, accurate covariance matrix
Status : MINIMIZE=0
Floating Parameter FinalValue +/- Error
-------------------- --------------------------
ratioBkgEff 1.0000e+00 +/- 1.99e-01
ratioSigEff 1.0000e+00 +/- 5.00e-02
s 5.9998e+01 +/- 2.32e+01
[#1] INFO:Minization -- RooProfileLL::evaluate(nll_modelWithConstraints_exampleData_with_constr_Profile[s]) Creating instance of MINUIT
[#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_modelWithConstraints_exampleData_with_constr) Summation contains a RooNLLVar, using its error level
[#1] INFO:Minization -- RooProfileLL::evaluate(nll_modelWithConstraints_exampleData_with_constr_Profile[s]) determining minimum likelihood for current configurations w.r.t all observable
[#1] INFO:Minization -- RooProfileLL::evaluate(nll_modelWithConstraints_exampleData_with_constr_Profile[s]) minimum found at (s=59.9982)
.
[#1] INFO:Minization -- RooProfileLL::evaluate(nll_modelWithConstraints_exampleData_with_constr_Profile[s]) Creating instance of MINUIT
[#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_modelWithConstraints_exampleData_with_constr) Summation contains a RooNLLVar, using its error level
[#1] INFO:Minization -- RooProfileLL::evaluate(nll_modelWithConstraints_exampleData_with_constr_Profile[s]) determining minimum likelihood for current configurations w.r.t all observable
[#0] ERROR:InputArguments -- RooArgSet::checkForDup: ERROR argument with name s is already in this set
[#1] INFO:Minization -- RooProfileLL::evaluate(nll_modelWithConstraints_exampleData_with_constr_Profile[s]) minimum found at (s=59.9989)
..........................................................................................................................................................................................................
=== Using the following for ModelConfig ===
Observables: RooArgSet:: = (obs)
Parameters of Interest: RooArgSet:: = (s)
Nuisance Parameters: RooArgSet:: = (ratioSigEff,ratioBkgEff)
Global Observables: RooArgSet:: = (gSigEff,gSigBkg)
PDF: RooProdPdf::modelWithConstraints[ countingModel * sigConstraint * bkgConstraint ] = 0.00460195
FeldmanCousins: ntoys per point: adaptive
FeldmanCousins: nEvents per toy will not fluctuate, will always be 1
FeldmanCousins: Model has nuisance parameters, will do profile construction
FeldmanCousins: # points to test = 100
NeymanConstruction: Prog: 1/100 total MC = 80 this test stat = 3.21178
s=0.6 ratioSigEff=0.999981 ratioBkgEff=1.43644 [-1e+30, 4.68588] in interval = 1
NeymanConstruction: Prog: 2/100 total MC = 80 this test stat = 3.08218
s=1.8 ratioSigEff=1.00019 ratioBkgEff=1.42695 [-1e+30, 3.71884] in interval = 1
NeymanConstruction: Prog: 3/100 total MC = 80 this test stat = 2.95515
s=3 ratioSigEff=1.00055 ratioBkgEff=1.4177 [-1e+30, 3.91238] in interval = 1
NeymanConstruction: Prog: 4/100 total MC = 80 this test stat = 2.83091
s=4.2 ratioSigEff=1.00106 ratioBkgEff=1.4085 [-1e+30, 3.65945] in interval = 1
NeymanConstruction: Prog: 5/100 total MC = 80 this test stat = 2.7093
s=5.4 ratioSigEff=1.00133 ratioBkgEff=1.39956 [-1e+30, 3.10318] in interval = 1
NeymanConstruction: Prog: 6/100 total MC = 80 this test stat = 2.59036
s=6.6 ratioSigEff=1.00159 ratioBkgEff=1.39061 [-1e+30, 2.97611] in interval = 1
NeymanConstruction: Prog: 7/100 total MC = 240 this test stat = 2.47407
s=7.8 ratioSigEff=1.00186 ratioBkgEff=1.38228 [-1e+30, 3.15094] in interval = 1
NeymanConstruction: Prog: 8/100 total MC = 80 this test stat = 2.36054
s=9 ratioSigEff=1.0021 ratioBkgEff=1.37325 [-1e+30, 3.1239] in interval = 1
NeymanConstruction: Prog: 9/100 total MC = 80 this test stat = 2.24968
s=10.2 ratioSigEff=1.00232 ratioBkgEff=1.36423 [-1e+30, 3.40684] in interval = 1
NeymanConstruction: Prog: 10/100 total MC = 80 this test stat = 2.14147
s=11.4 ratioSigEff=1.00253 ratioBkgEff=1.35522 [-1e+30, 3.1714] in interval = 1
NeymanConstruction: Prog: 11/100 total MC = 80 this test stat = 2.03601
s=12.6 ratioSigEff=1.00268 ratioBkgEff=1.3462 [-1e+30, 3.24645] in interval = 1
NeymanConstruction: Prog: 12/100 total MC = 80 this test stat = 1.93321
s=13.8 ratioSigEff=1.00287 ratioBkgEff=1.3372 [-1e+30, 2.53824] in interval = 1
NeymanConstruction: Prog: 13/100 total MC = 80 this test stat = 1.83309
s=15 ratioSigEff=1.00308 ratioBkgEff=1.32821 [-1e+30, 2.70016] in interval = 1
NeymanConstruction: Prog: 14/100 total MC = 80 this test stat = 1.73564
s=16.2 ratioSigEff=1.00323 ratioBkgEff=1.31922 [-1e+30, 2.77008] in interval = 1
NeymanConstruction: Prog: 15/100 total MC = 80 this test stat = 1.64094
s=17.4 ratioSigEff=1.00337 ratioBkgEff=1.31025 [-1e+30, 2.84065] in interval = 1
NeymanConstruction: Prog: 16/100 total MC = 80 this test stat = 1.5489
s=18.6 ratioSigEff=1.00347 ratioBkgEff=1.30119 [-1e+30, 2.44334] in interval = 1
NeymanConstruction: Prog: 17/100 total MC = 80 this test stat = 1.4595
s=19.8 ratioSigEff=1.00359 ratioBkgEff=1.29224 [-1e+30, 2.51006] in interval = 1
NeymanConstruction: Prog: 18/100 total MC = 80 this test stat = 1.37287
s=21 ratioSigEff=1.0037 ratioBkgEff=1.28331 [-1e+30, 2.30853] in interval = 1
NeymanConstruction: Prog: 19/100 total MC = 80 this test stat = 1.28887
s=22.2 ratioSigEff=1.00379 ratioBkgEff=1.27438 [-1e+30, 2.11523] in interval = 1
NeymanConstruction: Prog: 20/100 total MC = 80 this test stat = 1.20756
s=23.4 ratioSigEff=1.00387 ratioBkgEff=1.26546 [-1e+30, 2.09374] in interval = 1
NeymanConstruction: Prog: 21/100 total MC = 80 this test stat = 1.12892
s=24.6 ratioSigEff=1.00394 ratioBkgEff=1.25655 [-1e+30, 2.1559] in interval = 1
NeymanConstruction: Prog: 22/100 total MC = 80 this test stat = 1.05299
s=25.8 ratioSigEff=1.00399 ratioBkgEff=1.24765 [-1e+30, 2.48172] in interval = 1
NeymanConstruction: Prog: 23/100 total MC = 80 this test stat = 0.979748
s=27 ratioSigEff=1.00403 ratioBkgEff=1.23877 [-1e+30, 1.9489] in interval = 1
NeymanConstruction: Prog: 24/100 total MC = 80 this test stat = 0.909174
s=28.2 ratioSigEff=1.00405 ratioBkgEff=1.22989 [-1e+30, 1.92837] in interval = 1
NeymanConstruction: Prog: 25/100 total MC = 80 this test stat = 0.841264
s=29.4 ratioSigEff=1.00408 ratioBkgEff=1.22097 [-1e+30, 1.90797] in interval = 1
NeymanConstruction: Prog: 26/100 total MC = 80 this test stat = 0.776027
s=30.6 ratioSigEff=1.00408 ratioBkgEff=1.21214 [-1e+30, 2.04836] in interval = 1
NeymanConstruction: Prog: 27/100 total MC = 80 this test stat = 0.713453
s=31.8 ratioSigEff=1.00406 ratioBkgEff=1.2033 [-1e+30, 1.42563] in interval = 1
NeymanConstruction: Prog: 28/100 total MC = 80 this test stat = 0.653543
s=33 ratioSigEff=1.00403 ratioBkgEff=1.19446 [-1e+30, 1.92614] in interval = 1
NeymanConstruction: Prog: 29/100 total MC = 80 this test stat = 0.596293
s=34.2 ratioSigEff=1.00399 ratioBkgEff=1.18563 [-1e+30, 1.53064] in interval = 1
NeymanConstruction: Prog: 30/100 total MC = 80 this test stat = 0.541678
s=35.4 ratioSigEff=1.00393 ratioBkgEff=1.17679 [-1e+30, 1.65725] in interval = 1
NeymanConstruction: Prog: 31/100 total MC = 80 this test stat = 0.489749
s=36.6 ratioSigEff=1.00386 ratioBkgEff=1.16796 [-1e+30, 1.42567] in interval = 1
NeymanConstruction: Prog: 32/100 total MC = 80 this test stat = 0.440465
s=37.8 ratioSigEff=1.00378 ratioBkgEff=1.15914 [-1e+30, 1.40855] in interval = 1
NeymanConstruction: Prog: 33/100 total MC = 80 this test stat = 0.393811
s=39 ratioSigEff=1.00369 ratioBkgEff=1.15032 [-1e+30, 1.2593] in interval = 1
NeymanConstruction: Prog: 34/100 total MC = 80 this test stat = 0.349798
s=40.2 ratioSigEff=1.00358 ratioBkgEff=1.14153 [-1e+30, 1.30825] in interval = 1
NeymanConstruction: Prog: 35/100 total MC = 80 this test stat = 0.30842
s=41.4 ratioSigEff=1.00347 ratioBkgEff=1.13275 [-1e+30, 0.874179] in interval = 1
NeymanConstruction: Prog: 36/100 total MC = 80 this test stat = 0.269673
s=42.6 ratioSigEff=1.00334 ratioBkgEff=1.12398 [-1e+30, 1.14949] in interval = 1
NeymanConstruction: Prog: 37/100 total MC = 80 this test stat = 0.233551
s=43.8 ratioSigEff=1.00319 ratioBkgEff=1.11573 [-1e+30, 1.07403] in interval = 1
NeymanConstruction: Prog: 38/100 total MC = 80 this test stat = 0.200042
s=45 ratioSigEff=1.00303 ratioBkgEff=1.10705 [-1e+30, 1.30848] in interval = 1
NeymanConstruction: Prog: 39/100 total MC = 80 this test stat = 0.169154
s=46.2 ratioSigEff=1.00285 ratioBkgEff=1.09838 [-1e+30, 0.987229] in interval = 1
NeymanConstruction: Prog: 40/100 total MC = 80 this test stat = 0.140928
s=47.4 ratioSigEff=1.00267 ratioBkgEff=1.08972 [-1e+30, 0.917557] in interval = 1
NeymanConstruction: Prog: 41/100 total MC = 80 this test stat = 0.115258
s=48.6 ratioSigEff=1.00247 ratioBkgEff=1.08109 [-1e+30, 0.850481] in interval = 1
NeymanConstruction: Prog: 42/100 total MC = 80 this test stat = 0.0921856
s=49.8 ratioSigEff=1.00226 ratioBkgEff=1.07247 [-1e+30, 0.891152] in interval = 1
NeymanConstruction: Prog: 43/100 total MC = 80 this test stat = 0.071704
s=51 ratioSigEff=1.00204 ratioBkgEff=1.06387 [-1e+30, 1.04631] in interval = 1
NeymanConstruction: Prog: 44/100 total MC = 80 this test stat = 0.0538069
s=52.2 ratioSigEff=1.00181 ratioBkgEff=1.05529 [-1e+30, 0.574131] in interval = 1
NeymanConstruction: Prog: 45/100 total MC = 80 this test stat = 0.0384876
s=53.4 ratioSigEff=1.00156 ratioBkgEff=1.04672 [-1e+30, 0.749926] in interval = 1
NeymanConstruction: Prog: 46/100 total MC = 80 this test stat = 0.0257395
s=54.6 ratioSigEff=1.00131 ratioBkgEff=1.03818 [-1e+30, 0.839739] in interval = 1
NeymanConstruction: Prog: 47/100 total MC = 80 this test stat = 0.0155555
s=55.8 ratioSigEff=1.00104 ratioBkgEff=1.02966 [-1e+30, 0.715161] in interval = 1
NeymanConstruction: Prog: 48/100 total MC = 80 this test stat = 0.00792855
s=57 ratioSigEff=1.00075 ratioBkgEff=1.02116 [-1e+30, 0.574145] in interval = 1
NeymanConstruction: Prog: 49/100 total MC = 80 this test stat = 0.00285134
s=58.2 ratioSigEff=1.00046 ratioBkgEff=1.01268 [-1e+30, 0.732099] in interval = 1
NeymanConstruction: Prog: 50/100 total MC = 80 this test stat = 0.000340162
s=59.4 ratioSigEff=1.00048 ratioBkgEff=1.00391 [-1e+30, 0.692058] in interval = 1
NeymanConstruction: Prog: 51/100 total MC = 80 this test stat = 0.000338383
s=60.6 ratioSigEff=0.999508 ratioBkgEff=0.996114 [-1e+30, 0.645986] in interval = 1
NeymanConstruction: Prog: 52/100 total MC = 80 this test stat = 0.00280022
s=61.8 ratioSigEff=0.999513 ratioBkgEff=0.987372 [-1e+30, 0.513396] in interval = 1
NeymanConstruction: Prog: 53/100 total MC = 80 this test stat = 0.00784733
s=63 ratioSigEff=0.999173 ratioBkgEff=0.978982 [-1e+30, 0.477501] in interval = 1
NeymanConstruction: Prog: 54/100 total MC = 80 this test stat = 0.0154172
s=64.2 ratioSigEff=0.998823 ratioBkgEff=0.970615 [-1e+30, 0.721505] in interval = 1
NeymanConstruction: Prog: 55/100 total MC = 80 this test stat = 0.0254769
s=65.4 ratioSigEff=0.998461 ratioBkgEff=0.962272 [-1e+30, 0.628705] in interval = 1
NeymanConstruction: Prog: 56/100 total MC = 80 this test stat = 0.0380399
s=66.6 ratioSigEff=0.998088 ratioBkgEff=0.953953 [-1e+30, 0.636603] in interval = 1
NeymanConstruction: Prog: 57/100 total MC = 80 this test stat = 0.0530874
s=67.8 ratioSigEff=0.997704 ratioBkgEff=0.945659 [-1e+30, 0.596425] in interval = 1
NeymanConstruction: Prog: 58/100 total MC = 80 this test stat = 0.0706058
s=69 ratioSigEff=0.997308 ratioBkgEff=0.93739 [-1e+30, 0.865376] in interval = 1
NeymanConstruction: Prog: 59/100 total MC = 80 this test stat = 0.0905894
s=70.2 ratioSigEff=0.996902 ratioBkgEff=0.929147 [-1e+30, 0.817861] in interval = 1
NeymanConstruction: Prog: 60/100 total MC = 80 this test stat = 0.11303
s=71.4 ratioSigEff=0.996485 ratioBkgEff=0.920929 [-1e+30, 1.06483] in interval = 1
NeymanConstruction: Prog: 61/100 total MC = 80 this test stat = 0.137918
s=72.6 ratioSigEff=0.996058 ratioBkgEff=0.912737 [-1e+30, 0.780337] in interval = 1
NeymanConstruction: Prog: 62/100 total MC = 80 this test stat = 0.165246
s=73.8 ratioSigEff=0.995619 ratioBkgEff=0.904572 [-1e+30, 0.684448] in interval = 1
NeymanConstruction: Prog: 63/100 total MC = 80 this test stat = 0.195001
s=75 ratioSigEff=0.99517 ratioBkgEff=0.896434 [-1e+30, 1.03028] in interval = 1
NeymanConstruction: Prog: 64/100 total MC = 80 this test stat = 0.227176
s=76.2 ratioSigEff=0.994711 ratioBkgEff=0.888323 [-1e+30, 0.978198] in interval = 1
NeymanConstruction: Prog: 65/100 total MC = 80 this test stat = 0.261776
s=77.4 ratioSigEff=0.994241 ratioBkgEff=0.880239 [-1e+30, 1.11253] in interval = 1
NeymanConstruction: Prog: 66/100 total MC = 80 this test stat = 0.298772
s=78.6 ratioSigEff=0.99376 ratioBkgEff=0.872184 [-1e+30, 1.1878] in interval = 1
NeymanConstruction: Prog: 67/100 total MC = 80 this test stat = 0.338144
s=79.8 ratioSigEff=0.99327 ratioBkgEff=0.864156 [-1e+30, 1.48097] in interval = 1
NeymanConstruction: Prog: 68/100 total MC = 80 this test stat = 0.379914
s=81 ratioSigEff=0.992769 ratioBkgEff=0.856158 [-1e+30, 1.56743] in interval = 1
NeymanConstruction: Prog: 69/100 total MC = 80 this test stat = 0.424071
s=82.2 ratioSigEff=0.992258 ratioBkgEff=0.848188 [-1e+30, 1.28528] in interval = 1
NeymanConstruction: Prog: 70/100 total MC = 80 this test stat = 0.470561
s=83.4 ratioSigEff=0.991737 ratioBkgEff=0.840248 [-1e+30, 1.43791] in interval = 1
NeymanConstruction: Prog: 71/100 total MC = 80 this test stat = 0.519388
s=84.6 ratioSigEff=0.991206 ratioBkgEff=0.832337 [-1e+30, 1.84071] in interval = 1
NeymanConstruction: Prog: 72/100 total MC = 80 this test stat = 0.570629
s=85.8 ratioSigEff=0.990665 ratioBkgEff=0.824456 [-1e+30, 1.31507] in interval = 1
NeymanConstruction: Prog: 73/100 total MC = 80 this test stat = 0.624119
s=87 ratioSigEff=0.990114 ratioBkgEff=0.816606 [-1e+30, 1.18907] in interval = 1
NeymanConstruction: Prog: 74/100 total MC = 80 this test stat = 0.679954
s=88.2 ratioSigEff=0.989554 ratioBkgEff=0.808786 [-1e+30, 1.33487] in interval = 1
NeymanConstruction: Prog: 75/100 total MC = 80 this test stat = 0.738064
s=89.4 ratioSigEff=0.988831 ratioBkgEff=0.800144 [-1e+30, 1.56433] in interval = 1
NeymanConstruction: Prog: 76/100 total MC = 80 this test stat = 0.798499
s=90.6 ratioSigEff=0.988237 ratioBkgEff=0.792303 [-1e+30, 1.49944] in interval = 1
NeymanConstruction: Prog: 77/100 total MC = 80 this test stat = 0.861245
s=91.8 ratioSigEff=0.987632 ratioBkgEff=0.784488 [-1e+30, 1.7418] in interval = 1
NeymanConstruction: Prog: 78/100 total MC = 80 this test stat = 0.926247
s=93 ratioSigEff=0.987017 ratioBkgEff=0.7767 [-1e+30, 1.37459] in interval = 1
NeymanConstruction: Prog: 79/100 total MC = 80 this test stat = 0.993496
s=94.2 ratioSigEff=0.986393 ratioBkgEff=0.76894 [-1e+30, 2.1909] in interval = 1
NeymanConstruction: Prog: 80/100 total MC = 80 this test stat = 1.06297
s=95.4 ratioSigEff=0.985758 ratioBkgEff=0.761207 [-1e+30, 1.85545] in interval = 1
NeymanConstruction: Prog: 81/100 total MC = 80 this test stat = 1.13473
s=96.6 ratioSigEff=0.985113 ratioBkgEff=0.753502 [-1e+30, 1.86619] in interval = 1
NeymanConstruction: Prog: 82/100 total MC = 80 this test stat = 1.20872
s=97.8 ratioSigEff=0.984458 ratioBkgEff=0.745825 [-1e+30, 2.41018] in interval = 1
NeymanConstruction: Prog: 83/100 total MC = 80 this test stat = 1.28491
s=99 ratioSigEff=0.983794 ratioBkgEff=0.738176 [-1e+30, 2.4215] in interval = 1
NeymanConstruction: Prog: 84/100 total MC = 80 this test stat = 1.36329
s=100.2 ratioSigEff=0.983119 ratioBkgEff=0.730555 [-1e+30, 2.43281] in interval = 1
NeymanConstruction: Prog: 85/100 total MC = 80 this test stat = 1.44387
s=101.4 ratioSigEff=0.982436 ratioBkgEff=0.722963 [-1e+30, 2.44404] in interval = 1
NeymanConstruction: Prog: 86/100 total MC = 80 this test stat = 1.52662
s=102.6 ratioSigEff=0.981742 ratioBkgEff=0.7154 [-1e+30, 2.09032] in interval = 1
NeymanConstruction: Prog: 87/100 total MC = 80 this test stat = 1.61153
s=103.8 ratioSigEff=0.981039 ratioBkgEff=0.707866 [-1e+30, 2.76135] in interval = 1
NeymanConstruction: Prog: 88/100 total MC = 80 this test stat = 1.69863
s=105 ratioSigEff=0.980326 ratioBkgEff=0.700362 [-1e+30, 2.20046] in interval = 1
NeymanConstruction: Prog: 89/100 total MC = 80 this test stat = 1.78786
s=106.2 ratioSigEff=0.979605 ratioBkgEff=0.692886 [-1e+30, 2.58534] in interval = 1
NeymanConstruction: Prog: 90/100 total MC = 80 this test stat = 1.87921
s=107.4 ratioSigEff=0.978873 ratioBkgEff=0.685441 [-1e+30, 2.89797] in interval = 1
NeymanConstruction: Prog: 91/100 total MC = 80 this test stat = 1.97265
s=108.6 ratioSigEff=0.978133 ratioBkgEff=0.678026 [-1e+30, 3.1204] in interval = 1
NeymanConstruction: Prog: 92/100 total MC = 80 this test stat = 2.06825
s=109.8 ratioSigEff=0.977383 ratioBkgEff=0.670641 [-1e+30, 3.0252] in interval = 1
NeymanConstruction: Prog: 93/100 total MC = 80 this test stat = 2.16592
s=111 ratioSigEff=0.976625 ratioBkgEff=0.663286 [-1e+30, 2.93195] in interval = 1
NeymanConstruction: Prog: 94/100 total MC = 80 this test stat = 2.26567
s=112.2 ratioSigEff=0.975857 ratioBkgEff=0.655962 [-1e+30, 3.15451] in interval = 1
NeymanConstruction: Prog: 95/100 total MC = 80 this test stat = 2.36747
s=113.4 ratioSigEff=0.975049 ratioBkgEff=0.648489 [-1e+30, 3.05911] in interval = 1
NeymanConstruction: Prog: 96/100 total MC = 80 this test stat = 2.47134
s=114.6 ratioSigEff=0.97426 ratioBkgEff=0.641206 [-1e+30, 2.96561] in interval = 1
NeymanConstruction: Prog: 97/100 total MC = 80 this test stat = 2.57725
s=115.8 ratioSigEff=0.973462 ratioBkgEff=0.633952 [-1e+30, 3.5205] in interval = 1
NeymanConstruction: Prog: 98/100 total MC = 720 this test stat = 2.68519
s=117 ratioSigEff=0.972655 ratioBkgEff=0.626728 [-1e+30, 3.19948] in interval = 1
NeymanConstruction: Prog: 99/100 total MC = 80 this test stat = 2.79514
s=118.2 ratioSigEff=0.971839 ratioBkgEff=0.619535 [-1e+30, 3.43015] in interval = 1
NeymanConstruction: Prog: 100/100 total MC = 80 this test stat = 2.90706
s=119.4 ratioSigEff=0.971013 ratioBkgEff=0.612371 [-1e+30, 3.55406] in interval = 1
[#1] INFO:Eval -- 100 points in interval
[#1] INFO:Minization -- createNLL picked up cached consraints from workspace with 2 entries
[#1] INFO:Minization -- Including the following contraint terms in minimization: (sigConstraint,bkgConstraint)
[#1] INFO:Fitting -- RooAddition::defaultErrorLevel(nll_modelWithConstraints_exampleData_with_constr) Summation contains a RooNLLVar, using its error level
[#1] INFO:Minization -- RooMinimizer::optimizeConst: activating const optimization
[#1] INFO:Minization -- The following expressions will be evaluated in cache-and-track mode: (countingModel)
**********
** 1 **SET PRINT 1
**********
**********
** 2 **SET NOGRAD
**********
PARAMETER DEFINITIONS:
NO. NAME VALUE STEP SIZE LIMITS
1 ratioBkgEff 6.12371e-01 1.20943e-01 0.00000e+00 3.00000e+00
2 ratioSigEff 9.71013e-01 4.85593e-02 0.00000e+00 3.00000e+00
3 s 1.19400e+02 3.00000e-01 0.00000e+00 1.20000e+02
**********
** 3 **SET ERR 0.5
**********
**********
** 4 **SET PRINT 1
**********
**********
** 5 **SET STR 1
**********
NOW USING STRATEGY 1: TRY TO BALANCE SPEED AGAINST RELIABILITY
**********
** 6 **MIGRAD 1500 1
**********
FIRST CALL TO USER FUNCTION AT NEW START POINT, WITH IFLAG=4.
START MIGRAD MINIMIZATION. STRATEGY 1. CONVERGENCE WHEN EDM .LT. 1.00e-03
FCN=4.91605 FROM MIGRAD STATUS=INITIATE 27 CALLS 28 TOTAL
EDM= unknown STRATEGY= 1 NO ERROR MATRIX
EXT PARAMETER CURRENT GUESS STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 ratioBkgEff 6.12371e-01 1.20943e-01 0.00000e+00 -1.48191e+01
2 ratioSigEff 9.71013e-01 4.85593e-02 0.00000e+00 -1.97877e+01
3 s 9.75908e+01 3.00000e-01 -7.52185e-01 -1.16462e+00
ERR DEF= 0.5
MIGRAD MINIMIZATION HAS CONVERGED.
MIGRAD WILL VERIFY CONVERGENCE AND ERROR MATRIX.
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=2.81893 FROM MIGRAD STATUS=CONVERGED 79 CALLS 80 TOTAL
EDM=1.35245e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER STEP FIRST
NO. NAME VALUE ERROR SIZE DERIVATIVE
1 ratioBkgEff 1.00007e+00 1.99245e-01 9.52487e-05 8.82164e-03
2 ratioSigEff 1.00005e+00 4.99902e-02 4.32605e-05 3.08408e-02
3 s 5.99980e+01 2.32204e+01 2.65567e-04 2.76686e-03
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 3 ERR DEF=0.5
3.996e-02 9.387e-07 -3.996e+00
9.387e-07 2.500e-03 -1.501e-01
-3.996e+00 -1.501e-01 5.685e+02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3
1 0.84499 1.000 0.000 -0.838
2 0.23083 0.000 1.000 -0.126
3 0.84767 -0.838 -0.126 1.000
**********
** 7 **SET ERR 0.5
**********
**********
** 8 **SET PRINT 1
**********
**********
** 9 **HESSE 1500
**********
COVARIANCE MATRIX CALCULATED SUCCESSFULLY
FCN=2.81893 FROM HESSE STATUS=OK 16 CALLS 96 TOTAL
EDM=1.35245e-06 STRATEGY= 1 ERROR MATRIX ACCURATE
EXT PARAMETER INTERNAL INTERNAL
NO. NAME VALUE ERROR STEP SIZE VALUE
1 ratioBkgEff 1.00007e+00 1.99313e-01 1.90497e-05 -3.39791e-01
2 ratioSigEff 1.00005e+00 4.99898e-02 8.65209e-06 -3.39803e-01
3 s 5.99980e+01 2.32276e+01 5.31134e-05 -3.30629e-05
ERR DEF= 0.5
EXTERNAL ERROR MATRIX. NDIM= 25 NPAR= 3 ERR DEF=0.5
3.999e-02 5.515e-07 -3.999e+00
5.515e-07 2.500e-03 -1.501e-01
-3.999e+00 -1.501e-01 5.689e+02
PARAMETER CORRELATION COEFFICIENTS
NO. GLOBAL 1 2 3
1 0.84511 1.000 0.000 -0.838
2 0.23079 0.000 1.000 -0.126
3 0.84778 -0.838 -0.126 1.000
[#1] INFO:Minization -- RooMinimizer::optimizeConst: deactivating const optimization
[#1] INFO:Minization -- createNLL picked up cached consraints from workspace with 2 entries
[#1] INFO:Minization -- Including the following contraint terms in minimization: (sigConstraint,bkgConstraint)
[#1] INFO:Minization -- The following global observables have been defined: (gSigEff,gSigBkg)
Metropolis-Hastings progress: ....................................................................................................
[#1] INFO:Eval -- Proposal acceptance rate: 47.785%
[#1] INFO:Eval -- Number of steps in chain: 9557
Profile lower limit on s = 13.9497
Profile upper limit on s = 107.95
FC lower limit on s = 0.6
FC upper limit on s = 119.4
MCMC lower limit on s = 19.6124
MCMC upper limit on s = 102.568
MCMC Actual confidence level: 0.949952
[#0] ERROR:InputArguments -- RooArgSet::checkForDup: ERROR argument with name weight_MarkovChain_local_ is already in this set
plotting the chain data - nentries = 9557
plotting the scanned points used in the frequentist construction - npoints = 100
Real time 0:00:13, CP time 13.540
#include "RooProfileLL.h"
#include "RooAbsPdf.h"
#include "RooStats/HypoTestResult.h"
#include "RooRealVar.h"
#include "RooPlot.h"
#include "RooDataSet.h"
#include "RooTreeDataStore.h"
#include "TTree.h"
#include "TCanvas.h"
#include "TLine.h"
#include "TStopwatch.h"
#include "RooStats/ProfileLikelihoodCalculator.h"
#include "RooStats/MCMCCalculator.h"
#include "RooStats/UniformProposal.h"
#include "RooStats/FeldmanCousins.h"
#include "RooStats/NumberCountingPdfFactory.h"
#include "RooStats/ConfInterval.h"
#include "RooStats/PointSetInterval.h"
#include "RooStats/LikelihoodInterval.h"
#include "RooStats/LikelihoodIntervalPlot.h"
#include "RooStats/RooStatsUtils.h"
#include "RooStats/ModelConfig.h"
#include "RooStats/MCMCInterval.h"
#include "RooStats/MCMCIntervalPlot.h"
#include "RooStats/ProposalFunction.h"
#include "RooStats/ProposalHelper.h"
#include "RooFitResult.h"
#include "TGraph2D.h"
#include <cassert>
// use this order for safety on library loading
using namespace RooFit;
using namespace RooStats;
void rs101_limitexample()
{
// --------------------------------------
// An example of setting a limit in a number counting experiment with uncertainty on background and signal
// to time the macro
TStopwatch t;
t.Start();
// --------------------------------------
// The Model building stage
// --------------------------------------
RooWorkspace* wspace = new RooWorkspace();
wspace->factory("Poisson::countingModel(obs[150,0,300], sum(s[50,0,120]*ratioSigEff[1.,0,3.],b[100]*ratioBkgEff[1.,0.,3.]))"); // counting model
// wspace->factory("Gaussian::sigConstraint(ratioSigEff,1,0.05)"); // 5% signal efficiency uncertainty
// wspace->factory("Gaussian::bkgConstraint(ratioBkgEff,1,0.1)"); // 10% background efficiency uncertainty
wspace->factory("Gaussian::sigConstraint(gSigEff[1,0,3],ratioSigEff,0.05)"); // 5% signal efficiency uncertainty
wspace->factory("Gaussian::bkgConstraint(gSigBkg[1,0,3],ratioBkgEff,0.2)"); // 10% background efficiency uncertainty
wspace->factory("PROD::modelWithConstraints(countingModel,sigConstraint,bkgConstraint)"); // product of terms
wspace->Print();
RooAbsPdf* modelWithConstraints = wspace->pdf("modelWithConstraints"); // get the model
RooRealVar* obs = wspace->var("obs"); // get the observable
RooRealVar* s = wspace->var("s"); // get the signal we care about
RooRealVar* b = wspace->var("b"); // get the background and set it to a constant. Uncertainty included in ratioBkgEff
b->setConstant();
RooRealVar* ratioSigEff = wspace->var("ratioSigEff"); // get uncertain parameter to constrain
RooRealVar* ratioBkgEff = wspace->var("ratioBkgEff"); // get uncertain parameter to constrain
RooArgSet constrainedParams(*ratioSigEff, *ratioBkgEff); // need to constrain these in the fit (should change default behavior)
RooRealVar * gSigEff = wspace->var("gSigEff"); // global observables for signal efficiency
RooRealVar * gSigBkg = wspace->var("gSigBkg"); // global obs for background efficiency
gSigEff->setConstant();
gSigBkg->setConstant();
// Create an example dataset with 160 observed events
obs->setVal(160.);
RooDataSet* data = new RooDataSet("exampleData", "exampleData", RooArgSet(*obs));
data->add(*obs);
RooArgSet all(*s, *ratioBkgEff, *ratioSigEff);
// not necessary
modelWithConstraints->fitTo(*data, RooFit::Constrain(RooArgSet(*ratioSigEff, *ratioBkgEff)));
// Now let's make some confidence intervals for s, our parameter of interest
RooArgSet paramOfInterest(*s);
ModelConfig modelConfig(wspace);
modelConfig.SetPdf(*modelWithConstraints);
modelConfig.SetParametersOfInterest(paramOfInterest);
modelConfig.SetNuisanceParameters(constrainedParams);
modelConfig.SetObservables(*obs);
modelConfig.SetGlobalObservables( RooArgSet(*gSigEff,*gSigBkg));
modelConfig.SetName("ModelConfig");
wspace->import(modelConfig);
wspace->import(*data);
wspace->SetName("w");
wspace->writeToFile("rs101_ws.root");
// First, let's use a Calculator based on the Profile Likelihood Ratio
//ProfileLikelihoodCalculator plc(*data, *modelWithConstraints, paramOfInterest);
ProfileLikelihoodCalculator plc(*data, modelConfig);
plc.SetTestSize(.05);
ConfInterval* lrinterval = plc.GetInterval(); // that was easy.
// Let's make a plot
TCanvas* dataCanvas = new TCanvas("dataCanvas");
dataCanvas->Divide(2,1);
dataCanvas->cd(1);
LikelihoodIntervalPlot plotInt((LikelihoodInterval*)lrinterval);
plotInt.SetTitle("Profile Likelihood Ratio and Posterior for S");
plotInt.Draw();
// Second, use a Calculator based on the Feldman Cousins technique
FeldmanCousins fc(*data, modelConfig);
fc.UseAdaptiveSampling(true);
fc.FluctuateNumDataEntries(false); // number counting analysis: dataset always has 1 entry with N events observed
fc.SetNBins(100); // number of points to test per parameter
fc.SetTestSize(.05);
// fc.SaveBeltToFile(true); // optional
ConfInterval* fcint = NULL;
fcint = fc.GetInterval(); // that was easy.
RooFitResult* fit = modelWithConstraints->fitTo(*data, Save(true));
// Third, use a Calculator based on Markov Chain monte carlo
// Before configuring the calculator, let's make a ProposalFunction
// that will achieve a high acceptance rate
ProposalHelper ph;
ph.SetVariables((RooArgSet&)fit->floatParsFinal());
ph.SetCovMatrix(fit->covarianceMatrix());
ph.SetUpdateProposalParameters(true);
ph.SetCacheSize(100);
ProposalFunction* pdfProp = ph.GetProposalFunction(); // that was easy
MCMCCalculator mc(*data, modelConfig);
mc.SetNumIters(20000); // steps to propose in the chain
mc.SetTestSize(.05); // 95% CL
mc.SetNumBurnInSteps(40); // ignore first N steps in chain as "burn in"
mc.SetProposalFunction(*pdfProp);
mc.SetLeftSideTailFraction(0.5); // find a "central" interval
MCMCInterval* mcInt = (MCMCInterval*)mc.GetInterval(); // that was easy
// Get Lower and Upper limits from Profile Calculator
cout << "Profile lower limit on s = " << ((LikelihoodInterval*) lrinterval)->LowerLimit(*s) << endl;
cout << "Profile upper limit on s = " << ((LikelihoodInterval*) lrinterval)->UpperLimit(*s) << endl;
// Get Lower and Upper limits from FeldmanCousins with profile construction
if (fcint != NULL) {
double fcul = ((PointSetInterval*) fcint)->UpperLimit(*s);
double fcll = ((PointSetInterval*) fcint)->LowerLimit(*s);
cout << "FC lower limit on s = " << fcll << endl;
cout << "FC upper limit on s = " << fcul << endl;
TLine* fcllLine = new TLine(fcll, 0, fcll, 1);
TLine* fculLine = new TLine(fcul, 0, fcul, 1);
fcllLine->SetLineColor(kRed);
fculLine->SetLineColor(kRed);
fcllLine->Draw("same");
fculLine->Draw("same");
dataCanvas->Update();
}
// Plot MCMC interval and print some statistics
MCMCIntervalPlot mcPlot(*mcInt);
mcPlot.SetLineColor(kMagenta);
mcPlot.SetLineWidth(2);
mcPlot.Draw("same");
double mcul = mcInt->UpperLimit(*s);
double mcll = mcInt->LowerLimit(*s);
cout << "MCMC lower limit on s = " << mcll << endl;
cout << "MCMC upper limit on s = " << mcul << endl;
cout << "MCMC Actual confidence level: "
<< mcInt->GetActualConfidenceLevel() << endl;
// 3-d plot of the parameter points
dataCanvas->cd(2);
// also plot the points in the markov chain
RooDataSet * chainData = mcInt->GetChainAsDataSet();
assert(chainData);
std::cout << "plotting the chain data - nentries = " << chainData->numEntries() << std::endl;
TTree* chain = RooStats::GetAsTTree("chainTreeData","chainTreeData",*chainData);
assert(chain);
chain->SetMarkerStyle(6);
chain->SetMarkerColor(kRed);
chain->Draw("s:ratioSigEff:ratioBkgEff","nll_MarkovChain_local_","box"); // 3-d box proportional to posterior
// the points used in the profile construction
RooDataSet * parScanData = (RooDataSet*) fc.GetPointsToScan();
assert(parScanData);
std::cout << "plotting the scanned points used in the frequentist construction - npoints = " << parScanData->numEntries() << std::endl;
// getting the tree and drawing it -crashes (very strange....);
// TTree* parameterScan = RooStats::GetAsTTree("parScanTreeData","parScanTreeData",*parScanData);
// assert(parameterScan);
// parameterScan->Draw("s:ratioSigEff:ratioBkgEff","","goff");
TGraph2D *gr = new TGraph2D(parScanData->numEntries());
for (int ievt = 0; ievt < parScanData->numEntries(); ++ievt) {
const RooArgSet * evt = parScanData->get(ievt);
double x = evt->getRealValue("ratioBkgEff");
double y = evt->getRealValue("ratioSigEff");
double z = evt->getRealValue("s");
gr->SetPoint(ievt, x,y,z);
// std::cout << ievt << " " << x << " " << y << " " << z << std::endl;
}
gr->SetMarkerStyle(24);
gr->Draw("P SAME");
delete wspace;
delete lrinterval;
delete mcInt;
delete fcint;
delete data;
// print timing info
t.Stop();
t.Print();
}
Author
Kyle Cranmer

Definition in file rs101_limitexample.C.