HybridOriginalDemo.C: Example on how to use the HybridCalculatorOriginal class | Roostats tutorials | IntervalExamples.C: Example showing confidence intervals with four techniques. |
// A hypothesis testing example based on number counting with background uncertainty. /* HybridStandardForm Authors: Kyle Cranmer, Wouter Verkerke, and Sven Kreiss date May 2010 Part 1-3 date Dec 2010 Part 4-6 A hypothesis testing example based on number counting with background uncertainty. NOTE: This example is like HybridInstructional, but the model is more clearly generalizable to an analysis with shapes. There is a lot of flexability for how one models a problem in RooFit/RooStats. Models come in a few common forms: - standard form: extended PDF of some discriminating variable m: eg: P(m) ~ S*fs(m) + B*fb(m), with S+B events expected in this case the dataset has N rows corresponding to N events and the extended term is Pois(N|S+B) - fractional form: non-extended PDF of some discriminating variable m: eg: P(m) ~ s*fs(m) + (1-s)*fb(m), where s is a signal fraction in this case the dataset has N rows corresponding to N events and there is no extended term - number counting form: in which there is no discriminating variable and the counts are modeled directly (see HybridInstructional) eg: P(N) = Pois(N|S+B) in this case the dataset has 1 row corresponding to N events and the extended term is the PDF itself. Here we convert the number counting form into the standard form by introducing a dummy discriminating variable m with a uniform distribution. This example: - demonstrates the usage of the HybridCalcultor (Part 4-6) - demonstrates the numerical integration of RooFit (Part 2) - validates the RooStats against an example with a known analytic answer - demonstrates usage of different test statistics - explains subtle choices in the prior used for hybrid methods - demonstrates usage of different priors for the nuisance parameters - demonstrates usage of PROOF The basic setup here is that a main measurement has observed x events with an expectation of s+b. One can choose an ad hoc prior for the uncertainty on b, or try to base it on an auxiliary measurement. In this case, the auxiliary measurement (aka control measurement, sideband) is another counting experiment with measurement y and expectation tau*b. With an 'original prior' on b, called \eta(b) then one can obtain a posterior from the auxiliary measurement \pi(b) = \eta(b) * Pois(y|tau*b). This is a principled choice for a prior on b in the main measurement of x, which can then be treated in a hybrid Bayesian/Frequentist way. Additionally, one can try to treat the two measurements simultaneously, which is detailed in Part 6 of the tutorial. This tutorial is related to the FourBin.C tutorial in the modeling, but focuses on hypothesis testing instead of interval estimation. More background on this 'prototype problem' can be found in the following papers: Evaluation of three methods for calculating statistical significance when incorporating a systematic uncertainty into a test of the background-only hypothesis for a Poisson process Authors: Robert D. Cousins, James T. Linnemann, Jordan Tucker http://arxiv.org/abs/physics/0702156 NIM A 595 (2008) 480--501 Statistical Challenges for Searches for New Physics at the LHC Authors: Kyle Cranmer http://arxiv.org/abs/physics/0511028 Measures of Significance in HEP and Astrophysics Authors: J. T. Linnemann http://arxiv.org/abs/physics/0312059 */ #include "RooGlobalFunc.h" #include "RooRealVar.h" #include "RooProdPdf.h" #include "RooWorkspace.h" #include "RooDataSet.h" #include "RooDataHist.h" #include "TCanvas.h" #include "TStopwatch.h" #include "TH1.h" #include "RooPlot.h" #include "RooMsgService.h" #include "RooStats/NumberCountingUtils.h" #include "RooStats/HybridCalculator.h" #include "RooStats/ToyMCSampler.h" #include "RooStats/HypoTestPlot.h" #include "RooStats/NumEventsTestStat.h" #include "RooStats/ProfileLikelihoodTestStat.h" #include "RooStats/SimpleLikelihoodRatioTestStat.h" #include "RooStats/RatioOfProfiledLikelihoodsTestStat.h" #include "RooStats/MaxLikelihoodEstimateTestStat.h" using namespace RooFit; using namespace RooStats; ////////////////////////////////////////////////// // A New Test Statistic Class for this example. // It simply returns the sum of the values in a particular // column of a dataset. // You can ignore this class and focus on the macro below ////////////////////////////////////////////////// class BinCountTestStat : public TestStatistic { public: BinCountTestStat(void) : fColumnName("tmp") {} BinCountTestStat(string columnName) : fColumnName(columnName) {} virtual Double_t Evaluate(RooAbsData& data, RooArgSet& /*nullPOI*/) { // This is the main method in the interface Double_t value = 0.0; for(int i=0; i < data.numEntries(); i++) { value += data.get(i)->getRealValue(fColumnName.c_str()); } return value; } virtual const TString GetVarName() const { return fColumnName; } private: string fColumnName; protected: ClassDef(BinCountTestStat,1) }; ClassImp(BinCountTestStat) ////////////////////////////////////////////////// // The Actual Tutorial Macro ////////////////////////////////////////////////// void HybridStandardForm() { // This tutorial has 6 parts // Table of Contents // Setup // 1. Make the model for the 'prototype problem' // Special cases // 2. NOT RELEVANT HERE // 3. Use RooStats analytic solution for this problem // RooStats HybridCalculator -- can be generalized // 4. RooStats ToyMC version of 2. & 3. // 5. RooStats ToyMC with an equivalent test statistic // 6. RooStats ToyMC with simultaneous control & main measurement // Part 4 takes ~4 min without PROOF. // Part 5 takes about ~2 min with PROOF on 4 cores. // Of course, everything looks nicer with more toys, which takes longer. TStopwatch t; t.Start(); TCanvas *c = new TCanvas; c->Divide(2,2); /////////////////////////////////////////////////////// // P A R T 1 : D I R E C T I N T E G R A T I O N ////////////////////////////////////////////////////// // Make model for prototype on/off problem // Pois(x | s+b) * Pois(y | tau b ) // for Z_Gamma, use uniform prior on b. RooWorkspace* w = new RooWorkspace("w"); // replace the pdf in 'number couting form' //w->factory("Poisson::px(x[150,0,500],sum::splusb(s[0,0,100],b[100,0,300]))"); // with one in standard form. Now x is encoded in event count w->factory("Uniform::f(m[0,1])");//m is a dummy discriminanting variable w->factory("ExtendPdf::px(f,sum::splusb(s[0,0,100],b[100,0,300]))"); w->factory("Poisson::py(y[100,0,500],prod::taub(tau[1.],b))"); w->factory("PROD::model(px,py)"); w->factory("Uniform::prior_b(b)"); // We will control the output level in a few places to avoid // verbose progress messages. We start by keeping track // of the current threshold on messages. RooFit::MsgLevel msglevel = RooMsgService::instance().globalKillBelow(); // Use PROOF-lite on multi-core machines ProofConfig* pc = NULL; // uncomment below if you want to use PROOF pc = new ProofConfig(*w, 4, "workers=4", kFALSE); // machine with 4 cores // pc = new ProofConfig(*w, 2, "workers=2", kFALSE); // machine with 2 cores ///////////////////////////////////////////////// // P A R T 3 : A N A L Y T I C R E S U L T ///////////////////////////////////////////////// // In this special case, the integrals are known analytically // and they are implemented in RooStats::NumberCountingUtils // analytic Z_Bi double p_Bi = NumberCountingUtils::BinomialWithTauObsP(150, 100, 1); double Z_Bi = NumberCountingUtils::BinomialWithTauObsZ(150, 100, 1); cout << "-----------------------------------------"<<endl; cout << "Part 3" << endl; std::cout << "Z_Bi p-value (analytic): " << p_Bi << std::endl; std::cout << "Z_Bi significance (analytic): " << Z_Bi << std::endl; t.Stop(); t.Print(); t.Reset(); t.Start(); //////////////////////////////////////////////////////////////// // P A R T 4 : U S I N G H Y B R I D C A L C U L A T O R //////////////////////////////////////////////////////////////// // Now we demonstrate the RooStats HybridCalculator. // // Like all RooStats calculators it needs the data and a ModelConfig // for the relevant hypotheses. Since we are doing hypothesis testing // we need a ModelConfig for the null (background only) and the alternate // (signal+background) hypotheses. We also need to specify the PDF, // the parameters of interest, and the observables. Furthermore, since // the parameter of interest is floating, we need to specify which values // of the parameter corresponds to the null and alternate (eg. s=0 and s=50) // // define some sets of variables obs={x} and poi={s} // note here, x is the only observable in the main measurement // and y is treated as a separate measurement, which is used // to produce the prior that will be used in this calculation // to randomize the nuisance parameters. w->defineSet("obs","m"); w->defineSet("poi","s"); // create a toy dataset with the x=150 // RooDataSet *data = new RooDataSet("d", "d", *w->set("obs")); // data->add(*w->set("obs")); RooDataSet* data = w->pdf("px")->generate(*w->set("obs"),150); ////////////////////////////////////////////////////////// // Part 3a : Setup ModelConfigs // create the null (background-only) ModelConfig with s=0 ModelConfig b_model("B_model", w); b_model.SetPdf(*w->pdf("px")); b_model.SetObservables(*w->set("obs")); b_model.SetParametersOfInterest(*w->set("poi")); w->var("s")->setVal(0.0); // important! b_model.SetSnapshot(*w->set("poi")); // create the alternate (signal+background) ModelConfig with s=50 ModelConfig sb_model("S+B_model", w); sb_model.SetPdf(*w->pdf("px")); sb_model.SetObservables(*w->set("obs")); sb_model.SetParametersOfInterest(*w->set("poi")); w->var("s")->setVal(50.0); // important! sb_model.SetSnapshot(*w->set("poi")); ////////////////////////////////////////////////////////// // Part 3b : Choose Test Statistic // To make an equivalent calculation we need to use x as the test // statistic. This is not a built-in test statistic in RooStats // so we define it above. The new class inherits from the // RooStats::TestStatistic interface, and simply returns the value // of x in the dataset. NumEventsTestStat eventCount(*w->pdf("px")); ////////////////////////////////////////////////////////// // Part 3c : Define Prior used to randomize nuisance parameters // // The prior used for the hybrid calculator is the posterior // from the auxiliary measurement y. The model for the aux. // measurement is Pois(y|tau*b), thus the likleihood function // is proportional to (has the form of) a Gamma distribution. // if the 'original prior' \eta(b) is uniform, then from // Bayes's theorem we have the posterior: // \pi(b) = Pois(y|tau*b) * \eta(b) // If \eta(b) is flat, then we arrive at a Gamma distribution. // Since RooFit will normalize the PDF we can actually supply // py=Pois(y,tau*b) that will be equivalent to multiplying by a uniform. // // Alternatively, we could explicitly use a gamma distribution: // w->factory("Gamma::gamma(b,sum::temp(y,1),1,0)"); // // or we can use some other ad hoc prior that do not naturally // follow from the known form of the auxiliary measurement. // The common choice is the equivlaent Gaussian: w->factory("Gaussian::gauss_prior(b,y, expr::sqrty('sqrt(y)',y))"); // this corresponds to the "Z_N" calculation. // // or one could use the analogous log-normal prior w->factory("Lognormal::lognorm_prior(b,y, expr::kappa('1+1./sqrt(y)',y))"); // // Ideally, the HybridCalculator would be able to inspect the full // model Pois(x | s+b) * Pois(y | tau b ) and be given the original // prior \eta(b) to form \pi(b) = Pois(y|tau*b) * \eta(b). // This is not yet implemented because in the general case // it is not easy to identify the terms in the PDF that correspond // to the auxiliary measurement. So for now, it must be set // explicitly with: // - ForcePriorNuisanceNull() // - ForcePriorNuisanceAlt() // the name "ForcePriorNuisance" was chosen because we anticipate // this to be auto-detected, but will leave the option open // to force to a different prior for the nuisance parameters. ////////////////////////////////////////////////////////// // Part 3d : Construct and configure the HybridCalculator HybridCalculator hc1(*data, sb_model, b_model); ToyMCSampler *toymcs1 = (ToyMCSampler*)hc1.GetTestStatSampler(); // toymcs1->SetNEventsPerToy(1); // because the model is in number counting form toymcs1->SetTestStatistic(&eventCount); // set the test statistic // toymcs1->SetGenerateBinned(); hc1.SetToys(30000,1000); hc1.ForcePriorNuisanceAlt(*w->pdf("py")); hc1.ForcePriorNuisanceNull(*w->pdf("py")); // if you wanted to use the ad hoc Gaussian prior instead // hc1.ForcePriorNuisanceAlt(*w->pdf("gauss_prior")); // hc1.ForcePriorNuisanceNull(*w->pdf("gauss_prior")); // if you wanted to use the ad hoc log-normal prior instead // hc1.ForcePriorNuisanceAlt(*w->pdf("lognorm_prior")); // hc1.ForcePriorNuisanceNull(*w->pdf("lognorm_prior")); // enable proof // proof not enabled for this test statistic // if(pc) toymcs1->SetProofConfig(pc); // these lines save current msg level and then kill any messages below ERROR RooMsgService::instance().setGlobalKillBelow(RooFit::ERROR); // Get the result HypoTestResult *r1 = hc1.GetHypoTest(); RooMsgService::instance().setGlobalKillBelow(msglevel); // set it back cout << "-----------------------------------------"<<endl; cout << "Part 4" << endl; r1->Print(); t.Stop(); t.Print(); t.Reset(); t.Start(); c->cd(2); HypoTestPlot *p1 = new HypoTestPlot(*r1,30); // 30 bins, TS is discrete p1->Draw(); return; // keep the running time sort by default //////////////////////////////////////////////////////////////////////////// // P A R T 5 : U S I N G H Y B R I D C A L C U L A T O R W I T H // A N A L T E R N A T I V E T E S T S T A T I S T I C ///////////////////////////////////////////////////////////////////////////// // // A likelihood ratio test statistics should be 1-to-1 with the count x // when the value of b is fixed in the likelihood. This is implemented // by the SimpleLikelihoodRatioTestStat SimpleLikelihoodRatioTestStat slrts(*b_model.GetPdf(),*sb_model.GetPdf()); slrts.SetNullParameters(*b_model.GetSnapshot()); slrts.SetAltParameters(*sb_model.GetSnapshot()); // HYBRID CALCULATOR HybridCalculator hc2(*data, sb_model, b_model); ToyMCSampler *toymcs2 = (ToyMCSampler*)hc2.GetTestStatSampler(); // toymcs2->SetNEventsPerToy(1); toymcs2->SetTestStatistic(&slrts); // toymcs2->SetGenerateBinned(); hc2.SetToys(20000,1000); hc2.ForcePriorNuisanceAlt(*w->pdf("py")); hc2.ForcePriorNuisanceNull(*w->pdf("py")); // if you wanted to use the ad hoc Gaussian prior instead // hc2.ForcePriorNuisanceAlt(*w->pdf("gauss_prior")); // hc2.ForcePriorNuisanceNull(*w->pdf("gauss_prior")); // if you wanted to use the ad hoc log-normal prior instead // hc2.ForcePriorNuisanceAlt(*w->pdf("lognorm_prior")); // hc2.ForcePriorNuisanceNull(*w->pdf("lognorm_prior")); // enable proof if(pc) toymcs2->SetProofConfig(pc); // these lines save current msg level and then kill any messages below ERROR RooMsgService::instance().setGlobalKillBelow(RooFit::ERROR); // Get the result HypoTestResult *r2 = hc2.GetHypoTest(); cout << "-----------------------------------------"<<endl; cout << "Part 5" << endl; r2->Print(); t.Stop(); t.Print(); t.Reset(); t.Start(); RooMsgService::instance().setGlobalKillBelow(msglevel); c->cd(3); HypoTestPlot *p2 = new HypoTestPlot(*r2,30); // 30 bins p2->Draw(); return; // so standard tutorial runs faster /////////////////////////////////////////////////////////// // OUTPUT W/O PROOF (2.66 GHz Intel Core i7) /////////////////////////////////////////////////////////// /* ----------------------------------------- Part 3 Z_Bi p-value (analytic): 0.00094165 Z_Bi significance (analytic): 3.10804 Real time 0:00:00, CP time 0.610 Results HybridCalculator_result: - Null p-value = 0.00103333 +/- 0.000179406 - Significance = 3.08048 sigma - Number of S+B toys: 1000 - Number of B toys: 30000 - Test statistic evaluated on data: 150 - CL_b: 0.998967 +/- 0.000185496 - CL_s+b: 0.495 +/- 0.0158106 - CL_s: 0.495512 +/- 0.0158272 Real time 0:04:43, CP time 283.780 */ /* With PROOF ----------------------------------------- Part 5 Results HybridCalculator_result: - Null p-value = 0.00105 +/- 0.000206022 - Significance = 3.07571 sigma - Number of S+B toys: 1000 - Number of B toys: 20000 - Test statistic evaluated on data: 10.8198 - CL_b: 0.99895 +/- 0.000229008 - CL_s+b: 0.491 +/- 0.0158088 - CL_s: 0.491516 +/- 0.0158258 Real time 0:02:22, CP time 0.990 */ ////////////////////////////////////////// // Comparison /////////////////////////////////////////// // LEPStatToolsForLHC // https://plone4.fnal.gov:4430/P0/phystat/packages/0703002 // Uses Gaussian prior // CL_b = 6.218476e-04, Significance = 3.228665 sigma // ////////////////////////////////////////// // Comparison /////////////////////////////////////////// // Asymptotics // From the value of the profile likelihood ratio (5.0338) // The significance can be estimated using Wilks's theorem // significance = sqrt(2*profileLR) = 3.1729 sigma } HybridStandardForm.C:1 HybridStandardForm.C:2 HybridStandardForm.C:3 HybridStandardForm.C:4 HybridStandardForm.C:5 HybridStandardForm.C:6 HybridStandardForm.C:7 HybridStandardForm.C:8 HybridStandardForm.C:9 HybridStandardForm.C:10 HybridStandardForm.C:11 HybridStandardForm.C:12 HybridStandardForm.C:13 HybridStandardForm.C:14 HybridStandardForm.C:15 HybridStandardForm.C:16 HybridStandardForm.C:17 HybridStandardForm.C:18 HybridStandardForm.C:19 HybridStandardForm.C:20 HybridStandardForm.C:21 HybridStandardForm.C:22 HybridStandardForm.C:23 HybridStandardForm.C:24 HybridStandardForm.C:25 HybridStandardForm.C:26 HybridStandardForm.C:27 HybridStandardForm.C:28 HybridStandardForm.C:29 HybridStandardForm.C:30 HybridStandardForm.C:31 HybridStandardForm.C:32 HybridStandardForm.C:33 HybridStandardForm.C:34 HybridStandardForm.C:35 HybridStandardForm.C:36 HybridStandardForm.C:37 HybridStandardForm.C:38 HybridStandardForm.C:39 HybridStandardForm.C:40 HybridStandardForm.C:41 HybridStandardForm.C:42 HybridStandardForm.C:43 HybridStandardForm.C:44 HybridStandardForm.C:45 HybridStandardForm.C:46 HybridStandardForm.C:47 HybridStandardForm.C:48 HybridStandardForm.C:49 HybridStandardForm.C:50 HybridStandardForm.C:51 HybridStandardForm.C:52 HybridStandardForm.C:53 HybridStandardForm.C:54 HybridStandardForm.C:55 HybridStandardForm.C:56 HybridStandardForm.C:57 HybridStandardForm.C:58 HybridStandardForm.C:59 HybridStandardForm.C:60 HybridStandardForm.C:61 HybridStandardForm.C:62 HybridStandardForm.C:63 HybridStandardForm.C:64 HybridStandardForm.C:65 HybridStandardForm.C:66 HybridStandardForm.C:67 HybridStandardForm.C:68 HybridStandardForm.C:69 HybridStandardForm.C:70 HybridStandardForm.C:71 HybridStandardForm.C:72 HybridStandardForm.C:73 HybridStandardForm.C:74 HybridStandardForm.C:75 HybridStandardForm.C:76 HybridStandardForm.C:77 HybridStandardForm.C:78 HybridStandardForm.C:79 HybridStandardForm.C:80 HybridStandardForm.C:81 HybridStandardForm.C:82 HybridStandardForm.C:83 HybridStandardForm.C:84 HybridStandardForm.C:85 HybridStandardForm.C:86 HybridStandardForm.C:87 HybridStandardForm.C:88 HybridStandardForm.C:89 HybridStandardForm.C:90 HybridStandardForm.C:91 HybridStandardForm.C:92 HybridStandardForm.C:93 HybridStandardForm.C:94 HybridStandardForm.C:95 HybridStandardForm.C:96 HybridStandardForm.C:97 HybridStandardForm.C:98 HybridStandardForm.C:99 HybridStandardForm.C:100 HybridStandardForm.C:101 HybridStandardForm.C:102 HybridStandardForm.C:103 HybridStandardForm.C:104 HybridStandardForm.C:105 HybridStandardForm.C:106 HybridStandardForm.C:107 HybridStandardForm.C:108 HybridStandardForm.C:109 HybridStandardForm.C:110 HybridStandardForm.C:111 HybridStandardForm.C:112 HybridStandardForm.C:113 HybridStandardForm.C:114 HybridStandardForm.C:115 HybridStandardForm.C:116 HybridStandardForm.C:117 HybridStandardForm.C:118 HybridStandardForm.C:119 HybridStandardForm.C:120 HybridStandardForm.C:121 HybridStandardForm.C:122 HybridStandardForm.C:123 HybridStandardForm.C:124 HybridStandardForm.C:125 HybridStandardForm.C:126 HybridStandardForm.C:127 HybridStandardForm.C:128 HybridStandardForm.C:129 HybridStandardForm.C:130 HybridStandardForm.C:131 HybridStandardForm.C:132 HybridStandardForm.C:133 HybridStandardForm.C:134 HybridStandardForm.C:135 HybridStandardForm.C:136 HybridStandardForm.C:137 HybridStandardForm.C:138 HybridStandardForm.C:139 HybridStandardForm.C:140 HybridStandardForm.C:141 HybridStandardForm.C:142 HybridStandardForm.C:143 HybridStandardForm.C:144 HybridStandardForm.C:145 HybridStandardForm.C:146 HybridStandardForm.C:147 HybridStandardForm.C:148 HybridStandardForm.C:149 HybridStandardForm.C:150 HybridStandardForm.C:151 HybridStandardForm.C:152 HybridStandardForm.C:153 HybridStandardForm.C:154 HybridStandardForm.C:155 HybridStandardForm.C:156 HybridStandardForm.C:157 HybridStandardForm.C:158 HybridStandardForm.C:159 HybridStandardForm.C:160 HybridStandardForm.C:161 HybridStandardForm.C:162 HybridStandardForm.C:163 HybridStandardForm.C:164 HybridStandardForm.C:165 HybridStandardForm.C:166 HybridStandardForm.C:167 HybridStandardForm.C:168 HybridStandardForm.C:169 HybridStandardForm.C:170 HybridStandardForm.C:171 HybridStandardForm.C:172 HybridStandardForm.C:173 HybridStandardForm.C:174 HybridStandardForm.C:175 HybridStandardForm.C:176 HybridStandardForm.C:177 HybridStandardForm.C:178 HybridStandardForm.C:179 HybridStandardForm.C:180 HybridStandardForm.C:181 HybridStandardForm.C:182 HybridStandardForm.C:183 HybridStandardForm.C:184 HybridStandardForm.C:185 HybridStandardForm.C:186 HybridStandardForm.C:187 HybridStandardForm.C:188 HybridStandardForm.C:189 HybridStandardForm.C:190 HybridStandardForm.C:191 HybridStandardForm.C:192 HybridStandardForm.C:193 HybridStandardForm.C:194 HybridStandardForm.C:195 HybridStandardForm.C:196 HybridStandardForm.C:197 HybridStandardForm.C:198 HybridStandardForm.C:199 HybridStandardForm.C:200 HybridStandardForm.C:201 HybridStandardForm.C:202 HybridStandardForm.C:203 HybridStandardForm.C:204 HybridStandardForm.C:205 HybridStandardForm.C:206 HybridStandardForm.C:207 HybridStandardForm.C:208 HybridStandardForm.C:209 HybridStandardForm.C:210 HybridStandardForm.C:211 HybridStandardForm.C:212 HybridStandardForm.C:213 HybridStandardForm.C:214 HybridStandardForm.C:215 HybridStandardForm.C:216 HybridStandardForm.C:217 HybridStandardForm.C:218 HybridStandardForm.C:219 HybridStandardForm.C:220 HybridStandardForm.C:221 HybridStandardForm.C:222 HybridStandardForm.C:223 HybridStandardForm.C:224 HybridStandardForm.C:225 HybridStandardForm.C:226 HybridStandardForm.C:227 HybridStandardForm.C:228 HybridStandardForm.C:229 HybridStandardForm.C:230 HybridStandardForm.C:231 HybridStandardForm.C:232 HybridStandardForm.C:233 HybridStandardForm.C:234 HybridStandardForm.C:235 HybridStandardForm.C:236 HybridStandardForm.C:237 HybridStandardForm.C:238 HybridStandardForm.C:239 HybridStandardForm.C:240 HybridStandardForm.C:241 HybridStandardForm.C:242 HybridStandardForm.C:243 HybridStandardForm.C:244 HybridStandardForm.C:245 HybridStandardForm.C:246 HybridStandardForm.C:247 HybridStandardForm.C:248 HybridStandardForm.C:249 HybridStandardForm.C:250 HybridStandardForm.C:251 HybridStandardForm.C:252 HybridStandardForm.C:253 HybridStandardForm.C:254 HybridStandardForm.C:255 HybridStandardForm.C:256 HybridStandardForm.C:257 HybridStandardForm.C:258 HybridStandardForm.C:259 HybridStandardForm.C:260 HybridStandardForm.C:261 HybridStandardForm.C:262 HybridStandardForm.C:263 HybridStandardForm.C:264 HybridStandardForm.C:265 HybridStandardForm.C:266 HybridStandardForm.C:267 HybridStandardForm.C:268 HybridStandardForm.C:269 HybridStandardForm.C:270 HybridStandardForm.C:271 HybridStandardForm.C:272 HybridStandardForm.C:273 HybridStandardForm.C:274 HybridStandardForm.C:275 HybridStandardForm.C:276 HybridStandardForm.C:277 HybridStandardForm.C:278 HybridStandardForm.C:279 HybridStandardForm.C:280 HybridStandardForm.C:281 HybridStandardForm.C:282 HybridStandardForm.C:283 HybridStandardForm.C:284 HybridStandardForm.C:285 HybridStandardForm.C:286 HybridStandardForm.C:287 HybridStandardForm.C:288 HybridStandardForm.C:289 HybridStandardForm.C:290 HybridStandardForm.C:291 HybridStandardForm.C:292 HybridStandardForm.C:293 HybridStandardForm.C:294 HybridStandardForm.C:295 HybridStandardForm.C:296 HybridStandardForm.C:297 HybridStandardForm.C:298 HybridStandardForm.C:299 HybridStandardForm.C:300 HybridStandardForm.C:301 HybridStandardForm.C:302 HybridStandardForm.C:303 HybridStandardForm.C:304 HybridStandardForm.C:305 HybridStandardForm.C:306 HybridStandardForm.C:307 HybridStandardForm.C:308 HybridStandardForm.C:309 HybridStandardForm.C:310 HybridStandardForm.C:311 HybridStandardForm.C:312 HybridStandardForm.C:313 HybridStandardForm.C:314 HybridStandardForm.C:315 HybridStandardForm.C:316 HybridStandardForm.C:317 HybridStandardForm.C:318 HybridStandardForm.C:319 HybridStandardForm.C:320 HybridStandardForm.C:321 HybridStandardForm.C:322 HybridStandardForm.C:323 HybridStandardForm.C:324 HybridStandardForm.C:325 HybridStandardForm.C:326 HybridStandardForm.C:327 HybridStandardForm.C:328 HybridStandardForm.C:329 HybridStandardForm.C:330 HybridStandardForm.C:331 HybridStandardForm.C:332 HybridStandardForm.C:333 HybridStandardForm.C:334 HybridStandardForm.C:335 HybridStandardForm.C:336 HybridStandardForm.C:337 HybridStandardForm.C:338 HybridStandardForm.C:339 HybridStandardForm.C:340 HybridStandardForm.C:341 HybridStandardForm.C:342 HybridStandardForm.C:343 HybridStandardForm.C:344 HybridStandardForm.C:345 HybridStandardForm.C:346 HybridStandardForm.C:347 HybridStandardForm.C:348 HybridStandardForm.C:349 HybridStandardForm.C:350 HybridStandardForm.C:351 HybridStandardForm.C:352 HybridStandardForm.C:353 HybridStandardForm.C:354 HybridStandardForm.C:355 HybridStandardForm.C:356 HybridStandardForm.C:357 HybridStandardForm.C:358 HybridStandardForm.C:359 HybridStandardForm.C:360 HybridStandardForm.C:361 HybridStandardForm.C:362 HybridStandardForm.C:363 HybridStandardForm.C:364 HybridStandardForm.C:365 HybridStandardForm.C:366 HybridStandardForm.C:367 HybridStandardForm.C:368 HybridStandardForm.C:369 HybridStandardForm.C:370 HybridStandardForm.C:371 HybridStandardForm.C:372 HybridStandardForm.C:373 HybridStandardForm.C:374 HybridStandardForm.C:375 HybridStandardForm.C:376 HybridStandardForm.C:377 HybridStandardForm.C:378 HybridStandardForm.C:379 HybridStandardForm.C:380 HybridStandardForm.C:381 HybridStandardForm.C:382 HybridStandardForm.C:383 HybridStandardForm.C:384 HybridStandardForm.C:385 HybridStandardForm.C:386 HybridStandardForm.C:387 HybridStandardForm.C:388 HybridStandardForm.C:389 HybridStandardForm.C:390 HybridStandardForm.C:391 HybridStandardForm.C:392 HybridStandardForm.C:393 HybridStandardForm.C:394 HybridStandardForm.C:395 HybridStandardForm.C:396 HybridStandardForm.C:397 HybridStandardForm.C:398 HybridStandardForm.C:399 HybridStandardForm.C:400 HybridStandardForm.C:401 HybridStandardForm.C:402 HybridStandardForm.C:403 HybridStandardForm.C:404 HybridStandardForm.C:405 HybridStandardForm.C:406 HybridStandardForm.C:407 HybridStandardForm.C:408 HybridStandardForm.C:409 HybridStandardForm.C:410 HybridStandardForm.C:411 HybridStandardForm.C:412 HybridStandardForm.C:413 HybridStandardForm.C:414 HybridStandardForm.C:415 HybridStandardForm.C:416 HybridStandardForm.C:417 HybridStandardForm.C:418 HybridStandardForm.C:419 HybridStandardForm.C:420 HybridStandardForm.C:421 HybridStandardForm.C:422 HybridStandardForm.C:423 HybridStandardForm.C:424 HybridStandardForm.C:425 HybridStandardForm.C:426 HybridStandardForm.C:427 HybridStandardForm.C:428 HybridStandardForm.C:429 HybridStandardForm.C:430 HybridStandardForm.C:431 HybridStandardForm.C:432 HybridStandardForm.C:433 HybridStandardForm.C:434 HybridStandardForm.C:435 HybridStandardForm.C:436 HybridStandardForm.C:437 HybridStandardForm.C:438 HybridStandardForm.C:439 HybridStandardForm.C:440 HybridStandardForm.C:441 HybridStandardForm.C:442 HybridStandardForm.C:443 HybridStandardForm.C:444 |
|