MaCh3  2.6.1
Reference Guide
Public Member Functions | Protected Member Functions | Protected Attributes | List of all members
FitterBase Class Referenceabstract

Base class for implementing fitting algorithms. More...

#include <Fitters/FitterBase.h>

Inheritance diagram for FitterBase:
[legend]
Collaboration diagram for FitterBase:
[legend]

Public Member Functions

 FitterBase (Manager *const fitMan)
 Constructor. More...
 
virtual ~FitterBase ()
 Destructor for the FitterBase class. More...
 
void AddSampleHandler (SampleHandlerInterface *sample)
 This function adds a sample PDF object to the analysis framework. The sample PDF object will be utilized in fitting procedures or likelihood scans. More...
 
void AddSystObj (ParameterHandlerBase *cov)
 This function adds a Covariance object to the analysis framework. The Covariance object will be utilized in fitting procedures or likelihood scans. More...
 
virtual void RunMCMC ()=0
 The specific fitting algorithm implemented in this function depends on the derived class. It could be Markov Chain Monte Carlo (MCMC), MinuitFit, or another algorithm. More...
 
void DragRace (const int NLaps=100)
 Calculates the required time for each sample or covariance object in a drag race simulation. Inspired by Dan's feature. More...
 
void RunLLHScan ()
 Perform a 1D likelihood scan. More...
 
void RunLLHMap ()
 Perform a general multi-dimensional likelihood scan. More...
 
void GetStepScaleBasedOnLLHScan (const std::string &filename="")
 LLH scan is good first estimate of step scale. More...
 
void Run2DLLHScan ()
 Perform a 2D likelihood scan. More...
 
void RunSigmaVar ()
 Perform a 1D/2D sigma var for all samples. More...
 
virtual void StartFromPreviousFit (const std::string &FitName)
 Allow to start from previous fit/chain. More...
 
std::string GetName () const
 Get name of class. More...
 

Protected Member Functions

void ProcessMCMC ()
 Process MCMC output. More...
 
void PrepareOutput ()
 Prepare the output file. More...
 
void SaveOutput ()
 Save output and close files. More...
 
void SanitiseInputs ()
 Remove obsolete memory and make other checks before fit starts. More...
 
void SaveSettings ()
 Save the settings that the MCMC was run with. More...
 
bool GetScanRange (std::map< std::string, std::vector< double >> &scanRanges) const
 YSP: Set up a mapping to store parameters with user-specified ranges, suggested by D. Barrow. More...
 
void GetParameterScanRange (const ParameterHandlerBase *cov, const int i, double &CentralValue, double &lower, double &upper, const int n_points, const std::string &suffix="") const
 Helper function to get parameter scan range, central value. More...
 
bool CheckSkipParameter (const std::vector< std::string > &SkipVector, const std::string &ParamName) const
 KS: Check whether we want to skip parameter using skip vector. More...
 
void CustomRange (const std::string &ParName, const double sigma, double &ParamShiftValue) const
 For comparison with other fitting frameworks (like P-Theta) we usually have to apply different parameter values then usual 1, 3 sigma. More...
 
std::vector< double > CalculateBinEdges (double lowerlimit, double upperlimit, int n_points) const
 

Protected Attributes

ManagerfitMan
 The manager for configuration handling. More...
 
unsigned int step
 current state More...
 
double logLCurr
 current likelihood More...
 
double logLProp
 proposed likelihood More...
 
double accProb
 current acceptance prob More...
 
int accCount
 counts accepted steps More...
 
unsigned int stepStart
 step start, by default 0 if we start from previous chain then it will be different More...
 
std::vector< double > sample_llh
 store the llh breakdowns More...
 
std::vector< double > syst_llh
 systematic llh breakdowns More...
 
std::vector< SampleHandlerInterface * > samples
 Sample holder. More...
 
unsigned int TotalNSamples
 Total number of samples used, single SampleHandler can store more than one analysis sample! More...
 
std::vector< ParameterHandlerBase * > systematics
 Systematic holder. More...
 
std::unique_ptr< TStopwatch > clock
 tells global time how long fit took More...
 
std::unique_ptr< TStopwatch > stepClock
 tells how long single step/fit iteration took More...
 
double stepTime
 Time of single step. More...
 
std::unique_ptr< TRandom3 > random
 Random number. More...
 
TFile * outputFile
 Output. More...
 
TDirectory * CovFolder
 Output cov folder. More...
 
TDirectory * SampleFolder
 Output sample folder. More...
 
TTree * outTree
 Output tree with posteriors. More...
 
int auto_save
 auto save every N steps More...
 
bool fTestLikelihood
 Necessary for some fitting algorithms like PSO. More...
 
bool FileSaved
 Checks if file saved not repeat some operations. More...
 
bool SettingsSaved
 Checks if setting saved not repeat some operations. More...
 
bool OutputPrepared
 Checks if output prepared not repeat some operations. More...
 
std::string AlgorithmName
 Name of fitting algorithm that is being used. More...
 

Detailed Description

Base class for implementing fitting algorithms.

This class wraps MaCh3 classes like SampleHandlerInterface and ParameterHandlerBase. It serves as a base for different fitting algorithms and for validation techniques such as LLH scans.

Author
Asher Kaboth
Kamil Skwarczynski

Definition at line 29 of file FitterBase.h.

Constructor & Destructor Documentation

◆ FitterBase()

_MaCh3_Safe_Include_Start_ _MaCh3_Safe_Include_End_ FitterBase::FitterBase ( Manager *const  fitMan)

Constructor.

Parameters
fitManA pointer to a manager object, which will handle all settings.

Definition at line 18 of file FitterBase.cpp.

18  : fitMan(man) {
19 // *************************
20  AlgorithmName = "";
21  //Get mach3 modes from Manager
22  random = std::make_unique<TRandom3>(Get<int>(fitMan->raw()["General"]["Seed"], __FILE__, __LINE__));
23 
24  // Counter of the accepted # of steps
25  accCount = 0;
26  step = 0;
27  stepStart = 0;
28 
29  clock = std::make_unique<TStopwatch>();
30  stepClock = std::make_unique<TStopwatch>();
31  #ifdef MACH3_DEBUG
32  // Fit summary and debug info
33  debug = GetFromManager<bool>(fitMan->raw()["General"]["Debug"], false, __FILE__ , __LINE__);
34  #endif
35 
36  auto outfile = Get<std::string>(fitMan->raw()["General"]["OutputFile"], __FILE__ , __LINE__);
37  // Save output every auto_save steps
38  //you don't want this too often https://root.cern/root/html606/TTree_8cxx_source.html#l01229
39  auto_save = Get<int>(fitMan->raw()["General"]["MCMC"]["AutoSave"], __FILE__ , __LINE__);
40 
41  // Set the output file
42  outputFile = M3::Open(outfile, "RECREATE", __FILE__, __LINE__);
43  outputFile->cd();
44  // Set output tree
45  outTree = new TTree("posteriors", "Posterior_Distributions");
46  // Auto-save every 200MB, the bigger the better https://root.cern/root/html606/TTree_8cxx_source.html#l01229
47  outTree->SetAutoSave(-200E6);
48 
49  FileSaved = false;
50  SettingsSaved = false;
51  OutputPrepared = false;
52 
53  //Create TDirectory
54  CovFolder = outputFile->mkdir("CovarianceFolder");
55  outputFile->cd();
56  SampleFolder = outputFile->mkdir("SampleFolder");
57  outputFile->cd();
58 
59  #ifdef MACH3_DEBUG
60  // Prepare the output log file
61  if (debug) debugFile.open((outfile+".log").c_str());
62  #endif
63 
64  TotalNSamples = 0;
65  fTestLikelihood = GetFromManager<bool>(fitMan->raw()["General"]["Fitter"]["FitTestLikelihood"], false, __FILE__ , __LINE__);
66 }
std::unique_ptr< TRandom3 > random
Random number.
Definition: FitterBase.h:153
int accCount
counts accepted steps
Definition: FitterBase.h:128
bool OutputPrepared
Checks if output prepared not repeat some operations.
Definition: FitterBase.h:174
TFile * outputFile
Output.
Definition: FitterBase.h:156
unsigned int step
current state
Definition: FitterBase.h:120
bool SettingsSaved
Checks if setting saved not repeat some operations.
Definition: FitterBase.h:172
bool FileSaved
Checks if file saved not repeat some operations.
Definition: FitterBase.h:170
std::string AlgorithmName
Name of fitting algorithm that is being used.
Definition: FitterBase.h:177
std::unique_ptr< TStopwatch > clock
tells global time how long fit took
Definition: FitterBase.h:146
Manager * fitMan
The manager for configuration handling.
Definition: FitterBase.h:117
unsigned int stepStart
step start, by default 0 if we start from previous chain then it will be different
Definition: FitterBase.h:130
std::unique_ptr< TStopwatch > stepClock
tells how long single step/fit iteration took
Definition: FitterBase.h:148
TDirectory * CovFolder
Output cov folder.
Definition: FitterBase.h:158
TDirectory * SampleFolder
Output sample folder.
Definition: FitterBase.h:160
unsigned int TotalNSamples
Total number of samples used, single SampleHandler can store more than one analysis sample!
Definition: FitterBase.h:140
int auto_save
auto save every N steps
Definition: FitterBase.h:164
bool fTestLikelihood
Necessary for some fitting algorithms like PSO.
Definition: FitterBase.h:167
TTree * outTree
Output tree with posteriors.
Definition: FitterBase.h:162
YAML::Node const & raw() const
Return config.
Definition: Manager.h:47
TFile * Open(const std::string &Name, const std::string &Type, const std::string &File, const int Line)
Opens a ROOT file with the given name and mode.

◆ ~FitterBase()

FitterBase::~FitterBase ( )
virtual

Destructor for the FitterBase class.

Definition at line 70 of file FitterBase.cpp.

70  {
71 // *************************
72  SaveOutput();
73  if(outputFile != nullptr) delete outputFile;
74  outputFile = nullptr;
75  MACH3LOG_DEBUG("Closing MaCh3 Fitter Engine");
76 }
#define MACH3LOG_DEBUG
Definition: MaCh3Logger.h:34
void SaveOutput()
Save output and close files.
Definition: FitterBase.cpp:232

Member Function Documentation

◆ AddSampleHandler()

void FitterBase::AddSampleHandler ( SampleHandlerInterface sample)

This function adds a sample PDF object to the analysis framework. The sample PDF object will be utilized in fitting procedures or likelihood scans.

Parameters
sampleA pointer to a sample PDF object derived from ParameterHandlerBase.

Definition at line 263 of file FitterBase.cpp.

263  {
264 // *************************
265  // Check if any subsample name collides with already-registered subsamples
266  for (const auto &s : samples) {
267  for (int iExisting = 0; iExisting < s->GetNSamples(); ++iExisting) {
268  for (int iNew = 0; iNew < sample->GetNSamples(); ++iNew) {
269  if (s->GetSampleTitle(iExisting) == sample->GetSampleTitle(iNew)) {
271  "Duplicate sample title '{}' in handler {} detected: "
272  "same title exist in handler ", sample->GetSampleTitle(iNew),
273  sample->GetName(), s->GetName());
274  throw MaCh3Exception(__FILE__, __LINE__);
275  }
276  }
277  }
278  }
279 
280  for (const auto &s : samples) {
281  if (s->GetName() == sample->GetName()) {
282  MACH3LOG_ERROR("SampleHandler with name '{}' already exists!", sample->GetName());
283  MACH3LOG_ERROR("Is it intended?");
284  throw MaCh3Exception(__FILE__ , __LINE__ );
285  }
286  }
287  // Save additional info from samples
288  SampleFolder->cd();
289 
291  TotalNSamples += sample->GetNSamples();
292  MACH3LOG_INFO("Adding {} object, with {} samples", sample->GetName(), sample->GetNSamples());
293  samples.push_back(sample);
294  outputFile->cd();
295 }
#define MACH3LOG_ERROR
Definition: MaCh3Logger.h:37
#define MACH3LOG_INFO
Definition: MaCh3Logger.h:35
std::vector< SampleHandlerInterface * > samples
Sample holder.
Definition: FitterBase.h:138
Custom exception class used throughout MaCh3.
virtual std::string GetName() const =0
Get name for Sample Handler.
virtual void SaveAdditionalInfo([[maybe_unused]] TDirectory *Dir)
Store additional info in a chain.
virtual M3::int_t GetNSamples()
returns total number of samples
virtual std::string GetSampleTitle(const int iSample) const =0
Get fancy title for specified samples.

◆ AddSystObj()

void FitterBase::AddSystObj ( ParameterHandlerBase cov)

This function adds a Covariance object to the analysis framework. The Covariance object will be utilized in fitting procedures or likelihood scans.

Parameters
covA pointer to a Covariance object derived from ParameterHandlerBase.

Definition at line 299 of file FitterBase.cpp.

299  {
300 // *************************
301  MACH3LOG_INFO("Adding systematic object {}, with {} params", cov->GetName(), cov->GetNumParams());
302  // KS: Need to make sure we don't have params with same name, otherwise ROOT I/O and parts of MaCh3 will be terribly confused...
303  for (size_t s = 0; s < systematics.size(); ++s)
304  {
305  for (int iPar = 0; iPar < systematics[s]->GetNumParams(); ++iPar)
306  {
307  for (int i = 0; i < cov->GetNumParams(); ++i)
308  {
309  if(systematics[s]->GetParName(iPar) == cov->GetParName(i)){
310  MACH3LOG_ERROR("ParameterHandler {} has param '{}' which already exists in in {}, with name {}",
311  cov->GetName(), cov->GetParName(i), systematics[s]->GetName(), systematics[s]->GetParName(iPar));
312  throw MaCh3Exception(__FILE__ , __LINE__ );
313  }
314  // Same for fancy name
315  if(systematics[s]->GetParFancyName(iPar) == cov->GetParFancyName(i)){
316  MACH3LOG_ERROR("ParameterHandler {} has param '{}' which already exists in {}, with name {}",
317  cov->GetName(), cov->GetParFancyName(i), systematics[s]->GetName(), systematics[s]->GetParFancyName(iPar));
318  throw MaCh3Exception(__FILE__ , __LINE__ );
319  }
320  }
321  }
322  }
323 
324  systematics.push_back(cov);
325 
326  CovFolder->cd();
327  std::vector<double> n_vec(cov->GetNumParams());
328  for (int i = 0; i < cov->GetNumParams(); ++i) {
329  n_vec[i] = cov->GetParPreFit(i);
330  }
331  cov->GetCovMatrix()->Write(cov->GetName().c_str());
332 
333  TH2D* CorrMatrix = cov->GetCorrelationMatrix();
334  CorrMatrix->Write((cov->GetName() + std::string("_Corr")).c_str());
335  delete CorrMatrix;
336 
337  // If we have yaml config file for covariance let's save it
338  YAML::Node Config = cov->GetConfig();
339  if(!Config.IsNull())
340  {
341  TMacro ConfigSave = YAMLtoTMacro(Config, (std::string("Config_") + cov->GetName()));
342  ConfigSave.Write();
343  }
344 
345  outputFile->cd();
346 }
TMacro YAMLtoTMacro(const YAML::Node &yaml_node, const std::string &name)
Convert a YAML node to a ROOT TMacro object.
Definition: YamlHelper.h:167
std::vector< ParameterHandlerBase * > systematics
Systematic holder.
Definition: FitterBase.h:143
int GetNumParams() const
Get total number of parameters.
TH2D * GetCorrelationMatrix() const
KS: Convert covariance matrix to correlation matrix and return TH2D which can be used for fancy plott...
std::string GetParName(const int i) const
Get name of parameter.
TMatrixDSym * GetCovMatrix() const
Return covariance matrix.
std::string GetParFancyName(const int i) const
Get fancy name of the Parameter.
std::string GetName() const
Get name of covariance.
double GetParPreFit(const int i) const
Get prior parameter value.
YAML::Node GetConfig() const
Getter to return a copy of the YAML node.

◆ CalculateBinEdges()

std::vector< double > FitterBase::CalculateBinEdges ( double  lowerlimit,
double  upperlimit,
int  n_points 
) const
protected

Definition at line 606 of file FitterBase.cpp.

606  {
607 // *************************
608  std::vector<double> binEdges(n_points + 1);
609 
610  double logLower = std::log10(lowerlimit);
611  double logUpper = std::log10(upperlimit);
612 
613  for (int j = 0; j <= n_points; ++j) {
614  binEdges[j] = std::pow(10.0, logLower + (logUpper - logLower) * double(j) / double(n_points));
615  }
616 
617  return binEdges;
618 }

◆ CheckSkipParameter()

bool FitterBase::CheckSkipParameter ( const std::vector< std::string > &  SkipVector,
const std::string &  ParamName 
) const
protected

KS: Check whether we want to skip parameter using skip vector.

Definition at line 536 of file FitterBase.cpp.

536  {
537 // *************************
538  return M3::CaseInsensitiveMatchAny(ParamName, SkipVector);
539 }
bool CaseInsensitiveMatchAny(std::string Text, const std::vector< std::string > &Patterns)
Matches a string against a simple wildcard Pattern using regex. Is not case sensitive.

◆ CustomRange()

void FitterBase::CustomRange ( const std::string &  ParName,
const double  sigma,
double &  ParamShiftValue 
) const
protected

For comparison with other fitting frameworks (like P-Theta) we usually have to apply different parameter values then usual 1, 3 sigma.

Example YAML format:

SigmaVar:
Q2_norm_7: { "3": 2.0 }
SRC_Norm_O: { "-1": 0.5, "1": 1.5, "3": 2.0 }

Definition at line 1351 of file FitterBase.cpp.

1351  {
1352 // *************************
1353  if(!fitMan->raw()["SigmaVar"]["CustomRange"]) return;
1354 
1355  auto Config = fitMan->raw()["SigmaVar"]["CustomRange"];
1356 
1357  const auto sigmaStr = std::to_string(static_cast<int>(std::round(sigma)));
1358 
1359  if (Config[ParName] && Config[ParName][sigmaStr]) {
1360  ParamShiftValue = Config[ParName][sigmaStr].as<double>();
1361  MACH3LOG_INFO(" ::: setting custom range from config ::: {} -> {}", ParName, ParamShiftValue);
1362  }
1363 }

◆ DragRace()

void FitterBase::DragRace ( const int  NLaps = 100)

Calculates the required time for each sample or covariance object in a drag race simulation. Inspired by Dan's feature.

Parameters
NLapsnumber of laps, every part of Fitter will be tested with given number of laps and you will get total and average time

Definition at line 454 of file FitterBase.cpp.

454  {
455 // *************************
456  MACH3LOG_INFO("Let the Race Begin!");
457  MACH3LOG_INFO("All tests will be performed with {} threads", M3::GetNThreads());
458 
459  // Reweight the MC
460  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs)
461  {
462  TStopwatch clockRace;
463  clockRace.Start();
464  for(int Lap = 0; Lap < NLaps; ++Lap) {
465  samples[ivs]->Reweight();
466  }
467  clockRace.Stop();
468  MACH3LOG_INFO("It took {:.4f} s to reweights {} times sample: {}", clockRace.RealTime(), NLaps, samples[ivs]->GetName());
469  MACH3LOG_INFO("On average {:.6f}", clockRace.RealTime()/NLaps);
470  }
471 
472  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs)
473  {
474  TStopwatch clockRace;
475  clockRace.Start();
476  for(int Lap = 0; Lap < NLaps; ++Lap) {
477  samples[ivs]->GetLikelihood();
478  }
479  clockRace.Stop();
480  MACH3LOG_INFO("It took {:.4f} s to calculate GetLikelihood {} times sample: {}", clockRace.RealTime(), NLaps, samples[ivs]->GetName());
481  MACH3LOG_INFO("On average {:.6f}", clockRace.RealTime()/NLaps);
482  }
483  // Get vector of proposed steps. If we want to run LLH scan or something else after we need to revert changes after proposing steps multiple times
484  std::vector<std::vector<double>> StepsValuesBefore(systematics.size());
485  for (size_t s = 0; s < systematics.size(); ++s) {
486  StepsValuesBefore[s] = systematics[s]->GetProposed();
487  }
488  for (size_t s = 0; s < systematics.size(); ++s) {
489  TStopwatch clockRace;
490  clockRace.Start();
491  for(int Lap = 0; Lap < NLaps; ++Lap) {
492  systematics[s]->ProposeStep();
493  }
494  clockRace.Stop();
495  MACH3LOG_INFO("It took {:.4f} s to propose step {} times cov: {}", clockRace.RealTime(), NLaps, systematics[s]->GetName());
496  MACH3LOG_INFO("On average {:.6f}", clockRace.RealTime()/NLaps);
497  }
498  for (size_t s = 0; s < systematics.size(); ++s) {
499  systematics[s]->SetParameters(StepsValuesBefore[s]);
500  }
501 
502  for (size_t s = 0; s < systematics.size(); ++s) {
503  TStopwatch clockRace;
504  clockRace.Start();
505  for(int Lap = 0; Lap < NLaps; ++Lap) {
506  systematics[s]->GetLikelihood();
507  }
508  clockRace.Stop();
509  MACH3LOG_INFO("It took {:.4f} s to calculate get likelihood {} times cov: {}", clockRace.RealTime(), NLaps, systematics[s]->GetName());
510  MACH3LOG_INFO("On average {:.6f}", clockRace.RealTime()/NLaps);
511  }
512  MACH3LOG_INFO("End of race");
513 }
int GetNThreads()
number of threads which we need for example for TRandom3
Definition: Monitor.cpp:372

◆ GetName()

std::string FitterBase::GetName ( ) const
inline

Get name of class.

Definition at line 75 of file FitterBase.h.

75 {return AlgorithmName;};

◆ GetParameterScanRange()

void FitterBase::GetParameterScanRange ( const ParameterHandlerBase cov,
const int  i,
double &  CentralValue,
double &  lower,
double &  upper,
const int  n_points,
const std::string &  suffix = "" 
) const
protected

Helper function to get parameter scan range, central value.

Definition at line 543 of file FitterBase.cpp.

544  {
545 // *************************
546  // YSP: Set up a mapping to store parameters with user-specified ranges, suggested by D. Barrow
547  std::map<std::string, std::vector<double>> scanRanges;
548  const bool isScanRanges = GetScanRange(scanRanges);
549 
550  double nSigma = GetFromManager<int>(fitMan->raw()["LLHScan"]["LLHScanSigma"], 1., __FILE__, __LINE__);
551  bool IsPCA = cov->IsPCA();
552 
553  // Get the parameter name
554  std::string name = cov->GetParFancyName(i);
555  if (IsPCA) name += "_PCA";
556 
557  // Get the parameter priors and bounds
558  CentralValue = cov->GetParProp(i);
559  if (IsPCA) CentralValue = cov->GetPCAHandler()->GetParPropPCA(i);
560 
561  double prior = cov->GetParPreFit(i);
562  if (IsPCA) prior = cov->GetPCAHandler()->GetPreFitValuePCA(i);
563 
564  if (std::abs(CentralValue - prior) > 1e-10) {
565  MACH3LOG_INFO("For {} scanning around value {} rather than prior {}", name, CentralValue, prior);
566  }
567 
568  // Get the covariance matrix and do the +/- nSigma
569  // Set lower and upper bounds relative the CentralValue
570  // Set the parameter ranges between which LLH points are scanned
571  lower = CentralValue - nSigma*cov->GetDiagonalError(i);
572  upper = CentralValue + nSigma*cov->GetDiagonalError(i);
573  // If PCA, transform these parameter values to the PCA basis
574  if (IsPCA) {
575  lower = CentralValue - nSigma*std::sqrt((cov->GetPCAHandler()->GetEigenValues())(i));
576  upper = CentralValue + nSigma*std::sqrt((cov->GetPCAHandler()->GetEigenValues())(i));
577  MACH3LOG_INFO("eval {} = {:.2f}", i, cov->GetPCAHandler()->GetEigenValues()(i));
578  MACH3LOG_INFO("CV {} = {:.2f}", i, CentralValue);
579  MACH3LOG_INFO("lower {} = {:.2f}", i, lower);
580  MACH3LOG_INFO("upper {} = {:.2f}", i, upper);
581  MACH3LOG_INFO("nSigma = {:.2f}", nSigma);
582  }
583 
584  // Implementation suggested by D. Barrow
585  // If param ranges are specified in scanRanges node, extract it from there
586  if(isScanRanges){
587  // Find matching entries through std::maps
588  auto it = scanRanges.find(name);
589  if (it != scanRanges.end() && it->second.size() == 2) { //Making sure the range is has only two entries
590  lower = it->second[0];
591  upper = it->second[1];
592  MACH3LOG_INFO("Found matching param name for setting specified range for {}", name);
593  MACH3LOG_INFO("Range for {} = [{:.2f}, {:.2f}]", name, lower, upper);
594  }
595  }
596 
597  // Cross-section and flux parameters have boundaries that we scan between, check that these are respected in setting lower and upper variables
598  // This also applies for other parameters like osc, etc.
599  lower = std::max(lower, cov->GetLowerBound(i));
600  upper = std::min(upper, cov->GetUpperBound(i));
601  MACH3LOG_INFO("Scanning {} {} with {} steps, from [{:.2f} , {:.2f}], CV = {:.2f}", suffix, name, n_points, lower, upper, CentralValue);
602 }
bool GetScanRange(std::map< std::string, std::vector< double >> &scanRanges) const
YSP: Set up a mapping to store parameters with user-specified ranges, suggested by D....
Definition: FitterBase.cpp:516
double GetParPropPCA(const int i) const
Get current parameter value using PCA.
Definition: PCAHandler.h:150
double GetPreFitValuePCA(const int i) const
Get current parameter value using PCA.
Definition: PCAHandler.h:156
const TVectorD GetEigenValues() const
Get eigen values for all parameters, if you want for decomposed only parameters use GetEigenValuesMas...
Definition: PCAHandler.h:172
double GetUpperBound(const int i) const
Get upper parameter bound in which it is physically valid.
PCAHandler * GetPCAHandler() const
Get pointer for PCAHandler.
double GetLowerBound(const int i) const
Get lower parameter bound in which it is physically valid.
bool IsPCA() const
is PCA, can use to query e.g. LLH scans
M3::float_t GetParProp(const int i) const
Get proposed parameter value.
double GetDiagonalError(const int i) const
Get diagonal error for ith parameter.

◆ GetScanRange()

bool FitterBase::GetScanRange ( std::map< std::string, std::vector< double >> &  scanRanges) const
protected

YSP: Set up a mapping to store parameters with user-specified ranges, suggested by D. Barrow.

Parameters
scanRangesA map with user specified parameter ranges

Definition at line 516 of file FitterBase.cpp.

516  {
517 // *************************
518  bool isScanRanges = false;
519  // YSP: Set up a mapping to store parameters with user-specified ranges, suggested by D. Barrow
520  if(fitMan->raw()["LLHScan"]["ScanRanges"]){
521  YAML::Node scanRangesList = fitMan->raw()["LLHScan"]["ScanRanges"];
522  for (auto it = scanRangesList.begin(); it != scanRangesList.end(); ++it) {
523  std::string itname = it->first.as<std::string>();
524  std::vector<double> itrange = it->second.as<std::vector<double>>();
525  // Set the mapping as param_name:param_range
526  scanRanges[itname] = itrange;
527  }
528  isScanRanges = true;
529  } else {
530  MACH3LOG_INFO("There are no user-defined parameter ranges, so I'll use default param bounds for LLH Scans");
531  }
532  return isScanRanges;
533 }

◆ GetStepScaleBasedOnLLHScan()

void FitterBase::GetStepScaleBasedOnLLHScan ( const std::string &  filename = "")

LLH scan is good first estimate of step scale.

Parameters
filenameby default empty, however if specified it will allow to load LLH scan from external file

Definition at line 906 of file FitterBase.cpp.

906  {
907 // *************************
908  TFile* outputFileLLH = nullptr;
909  bool ownsfile = false;
910  if(outputFileName != ""){
911  outputFileLLH = M3::Open(outputFileName, "READ", __FILE__, __LINE__);
912  ownsfile = true;
913  } else {
914  outputFileLLH = outputFile;
915  }
916  TDirectory *Sample_LLH = outputFileLLH->Get<TDirectory>("Sample_LLH");
917  MACH3LOG_INFO("Starting Get Step Scale Based On LLHScan");
918 
919  if(!Sample_LLH || Sample_LLH->IsZombie())
920  {
921  MACH3LOG_WARN("Couldn't find Sample_LLH, it looks like LLH scan wasn't run, will do this now");
922  RunLLHScan();
923  Sample_LLH = outputFileLLH->Get<TDirectory>("Sample_LLH");
924  }
925 
926  for (ParameterHandlerBase *cov : systematics)
927  {
928  const int npars = cov->GetNumParams();
929  std::vector<double> StepScale(npars);
930  for (int i = 0; i < npars; ++i)
931  {
932  std::string name = cov->GetParFancyName(i);
933  StepScale[i] = cov->GetIndivStepScale(i);
934  TH1D* LLHScan = Sample_LLH->Get<TH1D>((name+"_sam").c_str());
935  if(LLHScan == nullptr)
936  {
937  MACH3LOG_WARN("Couldn't find LLH scan, for {}, skipping", name);
938  continue;
939  }
940  const double LLH_val = std::max(LLHScan->GetBinContent(1), LLHScan->GetBinContent(LLHScan->GetNbinsX()));
941  //If there is no sensitivity leave it
942  if(LLH_val < 0.001) continue;
943 
944  // EM: assuming that the likelihood is gaussian, approximate sigma value is given by variation/sqrt(-2LLH)
945  // can evaluate this at any point, simple to evaluate it in the first bin of the LLH scan
946  // KS: We assume variation is 1 sigma, each dial has different scale so it becomes faff...
947  const double Var = 1.;
948  const double approxSigma = std::abs(Var)/std::sqrt(LLH_val);
949  const double GlobalScale = cov->GetGlobalStepScale();
950  // Based on Ewan comment I just took the 1sigma width from the LLH, assuming it was Gaussian, but then had to also scale by 2.38/sqrt(N_params)
951  const double TargetStep = approxSigma * 2.38 / std::sqrt(npars);
952  // KS: Need to divide by currently used gloalStepScale
953  const double NewStepScale = TargetStep / GlobalScale;
954 
955  StepScale[i] = NewStepScale;
956  MACH3LOG_DEBUG("Sigma: {}", approxSigma);
957  MACH3LOG_DEBUG("Target Step Size (before accounting for global step size): {}", TargetStep);
958  MACH3LOG_DEBUG("Optimal Step Size: {}", NewStepScale);
959  }
960  cov->SetIndivStepScale(StepScale);
961  cov->SaveUpdatedMatrixConfig();
962  }
963  if(ownsfile && outputFileLLH != nullptr) delete outputFileLLH;
964 }
#define MACH3LOG_WARN
Definition: MaCh3Logger.h:36
void RunLLHScan()
Perform a 1D likelihood scan.
Definition: FitterBase.cpp:622
Base class for handling systematic uncertainty parameters.

◆ PrepareOutput()

void FitterBase::PrepareOutput ( )
protected

Prepare the output file.

Definition at line 154 of file FitterBase.cpp.

154  {
155 // *******************
156  if(OutputPrepared) return;
157  //MS: Check if we are fitting the test likelihood, rather than T2K likelihood, and only setup T2K output if not
158  if(!fTestLikelihood)
159  {
160  // Check that we have added samples
161  if (!samples.size()) {
162  MACH3LOG_CRITICAL("No samples Found! Is this really what you wanted?");
163  //throw MaCh3Exception(__FILE__ , __LINE__ );
164  }
165 
166  // Do we want to save proposal? This will break plotting scripts and is heave for disk space and step time. Only use when debugging
167  bool SaveProposal = GetFromManager<bool>(fitMan->raw()["General"]["SaveProposal"], false, __FILE__ , __LINE__);
168 
169  if(SaveProposal) MACH3LOG_INFO("Will save in the chain proposal parameters and LogL");
170  // Prepare the output trees
171  for (ParameterHandlerBase *cov : systematics) {
172  cov->SetBranches(*outTree, SaveProposal);
173  }
174 
175  outTree->Branch("LogL", &logLCurr, "LogL/D");
176  if(SaveProposal) outTree->Branch("LogLProp", &logLProp, "LogLProp/D");
177  outTree->Branch("accProb", &accProb, "accProb/D");
178  outTree->Branch("step", &step, "step/i");
179  outTree->Branch("stepTime", &stepTime, "stepTime/D");
180 
181  // Store individual likelihood components
182  // Using double means double as large output file!
183  sample_llh.resize(samples.size());
184  syst_llh.resize(systematics.size());
185 
186  for (size_t i = 0; i < samples.size(); ++i) {
187  std::stringstream oss, oss2;
188  oss << "LogL_sample_" << i;
189  oss2 << oss.str() << "/D";
190  outTree->Branch(oss.str().c_str(), &sample_llh[i], oss2.str().c_str());
191  }
192 
193  for (size_t i = 0; i < systematics.size(); ++i) {
194  std::stringstream oss, oss2;
195  oss << "LogL_systematic_" << systematics[i]->GetName();
196  oss2 << oss.str() << "/D";
197  outTree->Branch(oss.str().c_str(), &syst_llh[i], oss2.str().c_str());
198  }
199  }
200  else
201  {
202  outTree->Branch("LogL", &logLCurr, "LogL/D");
203  outTree->Branch("accProb", &accProb, "accProb/D");
204  outTree->Branch("step", &step, "step/i");
205  outTree->Branch("stepTime", &stepTime, "stepTime/D");
206  }
207 
208  MACH3LOG_INFO("-------------------- Starting MCMC --------------------");
209  #ifdef MACH3_DEBUG
210  if (debug) {
211  debugFile << "----- Starting MCMC -----" << std::endl;
212  }
213  #endif
214  // Time the progress
215 
216 
217  clock->Start();
218 
219 
220  OutputPrepared = true;
221 }
#define MACH3LOG_CRITICAL
Definition: MaCh3Logger.h:38
double logLProp
proposed likelihood
Definition: FitterBase.h:124
double accProb
current acceptance prob
Definition: FitterBase.h:126
std::vector< double > sample_llh
store the llh breakdowns
Definition: FitterBase.h:133
double stepTime
Time of single step.
Definition: FitterBase.h:150
double logLCurr
current likelihood
Definition: FitterBase.h:122
std::vector< double > syst_llh
systematic llh breakdowns
Definition: FitterBase.h:135

◆ ProcessMCMC()

void FitterBase::ProcessMCMC ( )
protected

Process MCMC output.

Definition at line 407 of file FitterBase.cpp.

407  {
408 // *******************
409  if (fitMan == nullptr) return;
410 
411  // Process the MCMC
412  if (GetFromManager<bool>(fitMan->raw()["General"]["ProcessMCMC"], false, __FILE__ , __LINE__)){
413  // Make the processor
414  MCMCProcessor Processor(std::string(outputFile->GetName()));
415 
416  Processor.Initialise();
417  // Make the TVectorD pointers which hold the processed output
418  TVectorD *Central = nullptr;
419  TVectorD *Errors = nullptr;
420  TVectorD *Central_Gauss = nullptr;
421  TVectorD *Errors_Gauss = nullptr;
422  TVectorD *Peaks = nullptr;
423 
424  // Make the postfit
425  Processor.GetPostfit(Central, Errors, Central_Gauss, Errors_Gauss, Peaks);
426  Processor.DrawPostfit();
427 
428  // Make the TMatrix pointers which hold the processed output
429  TMatrixDSym *Covariance = nullptr;
430  TMatrixDSym *Correlation = nullptr;
431 
432  // Make the covariance matrix
433  Processor.GetCovariance(Covariance, Correlation);
434  Processor.DrawCovariance();
435 
436  std::vector<TString> BranchNames = Processor.GetBranchNames();
437 
438  // Re-open the TFile
439  if (!outputFile->IsOpen()) {
440  MACH3LOG_INFO("Opening output again to update with means..");
441  outputFile = new TFile(Get<std::string>(fitMan->raw()["General"]["OutputFile"], __FILE__, __LINE__).c_str(), "UPDATE");
442  }
443  Central->Write("PDF_Means");
444  Errors->Write("PDF_Errors");
445  Central_Gauss->Write("Gauss_Means");
446  Errors_Gauss->Write("Errors_Gauss");
447  Covariance->Write("Covariance");
448  Correlation->Write("Correlation");
449  }
450 }
Class responsible for processing MCMC chains, performing diagnostics, generating plots,...
Definition: MCMCProcessor.h:61

◆ Run2DLLHScan()

void FitterBase::Run2DLLHScan ( )

Perform a 2D likelihood scan.

Warning
This operation may take a significant amount of time, especially for complex models.

Definition at line 968 of file FitterBase.cpp.

968  {
969 // *************************
970  // Save the settings into the output file
971  SaveSettings();
972 
973  MACH3LOG_INFO("Starting 2D LLH Scan");
974 
975  TDirectory *Sample_2DLLH = outputFile->mkdir("Sample_2DLLH");
976  auto SkipVector = GetFromManager<std::vector<std::string>>(fitMan->raw()["LLHScan"]["LLHScanSkipVector"], {}, __FILE__ , __LINE__);;
977 
978  //Do we want a logarithmic LLH scan
979  bool LLHLogarithmic = GetFromManager<bool>(fitMan->raw()["LLHScan"]["LLHLogarithmic"], false, __FILE__ , __LINE__);
980 
981  // Number of points we do for each LLH scan
982  const int n_points = GetFromManager<int>(fitMan->raw()["LLHScan"]["2DLLHScanPoints"], 20, __FILE__ , __LINE__);
983  // We print 5 reweights
984  const int countwidth = int(double(n_points)/double(5));
985 
986  // Loop over the covariance classes
987  for (ParameterHandlerBase *cov : systematics)
988  {
989  // Scan over all the parameters
990  // Get the number of parameters
991  int npars = cov->GetNumParams();
992  bool IsPCA = cov->IsPCA();
993  if (IsPCA) npars = cov->GetNParameters();
994 
995  for (int i = 0; i < npars; ++i)
996  {
997  std::string name_x = cov->GetParFancyName(i);
998  if (IsPCA) name_x += "_PCA";
999  // Get the parameter central and bounds
1000  double central_x, lower_x, upper_x;
1001  GetParameterScanRange(cov, i, central_x, lower_x, upper_x, n_points, "X");
1002  std::vector<double> binEdges_x;
1003 
1004  //Logarithmic scan
1005  if(LLHLogarithmic){
1006  if (lower_x < 0.0 || upper_x < 0.0){
1007  MACH3LOG_WARN("Cannot perform logarithmic scan for {} "" with range [{}, {}], falling back to linear scan", name_x, lower_x, upper_x);
1008  } else {
1009  binEdges_x = CalculateBinEdges(lower_x, upper_x, n_points);
1010  }
1011  }
1012  // KS: Check if we want to skip this parameter
1013  if(CheckSkipParameter(SkipVector, name_x)) continue;
1014 
1015  for (int j = 0; j < i; ++j)
1016  {
1017  std::string name_y = cov->GetParFancyName(j);
1018  if (IsPCA) name_y += "_PCA";
1019  // KS: Check if we want to skip this parameter
1020  if(CheckSkipParameter(SkipVector, name_y)) continue;
1021 
1022  std::unique_ptr<TH2D> hScanSam;
1023  // Get the parameter central and bounds
1024  double central_y, lower_y, upper_y;
1025  GetParameterScanRange(cov, j, central_y, lower_y, upper_y, n_points, "Y");
1026  if(LLHLogarithmic){
1027  //negative values break it
1028  if (lower_x < 0.0 || upper_x < 0.0 || lower_y < 0.0 || upper_y < 0.0 ) {
1029  MACH3LOG_WARN("Cannot perform logarithmic scan for {} and {} "" with range [{}, {}], [{}, {}], falling back to linear scan", name_x, lower_x, upper_x, name_y, lower_y, upper_y);
1030  hScanSam = std::make_unique<TH2D>((name_x + "_" + name_y + "_sam").c_str(), (name_x + "_" + name_y + "_sam").c_str(), n_points, lower_x, upper_x, n_points, lower_y, upper_y);
1031  hScanSam->SetDirectory(nullptr);
1032  hScanSam->GetXaxis()->SetTitle(name_x.c_str());
1033  hScanSam->GetYaxis()->SetTitle(name_y.c_str());
1034  hScanSam->GetZaxis()->SetTitle("2LLH_sam");
1035  } else {
1036  auto binEdges_y = CalculateBinEdges(lower_y, upper_y, n_points);
1037  hScanSam = std::make_unique<TH2D>((name_x + "_" + name_y + "_sam").c_str(), (name_x + "_" + name_y + "_sam").c_str(), n_points, binEdges_x.data(), n_points, binEdges_y.data());
1038  }
1039  }
1040  //If not then just do the normal thing
1041  else {
1042  hScanSam = std::make_unique<TH2D>((name_x + "_" + name_y + "_sam").c_str(), (name_x + "_" + name_y + "_sam").c_str(), n_points, lower_x, upper_x, n_points, lower_y, upper_y);
1043  hScanSam->SetDirectory(nullptr);
1044  hScanSam->GetXaxis()->SetTitle(name_x.c_str());
1045  hScanSam->GetYaxis()->SetTitle(name_y.c_str());
1046  hScanSam->GetZaxis()->SetTitle("2LLH_sam");
1047  }
1048 
1049  // Scan over the parameter space
1050  for (int x = 0; x < n_points; ++x)
1051  {
1052  if (x % countwidth == 0)
1053  M3::Utils::PrintProgressBar(x, n_points);
1054 
1055  for (int y = 0; y < n_points; ++y)
1056  {
1057  // For PCA we have to do it differently
1058  if (IsPCA) {
1059  cov->GetPCAHandler()->SetParPropPCA(i, hScanSam->GetXaxis()->GetBinCenter(x+1));
1060  cov->GetPCAHandler()->SetParPropPCA(j, hScanSam->GetYaxis()->GetBinCenter(y+1));
1061  } else {
1062  // Set the parameter
1063  cov->SetParProp(i, hScanSam->GetXaxis()->GetBinCenter(x+1));
1064  cov->SetParProp(j, hScanSam->GetYaxis()->GetBinCenter(y+1));
1065  }
1066  // Reweight the MC
1067  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs) {
1068  samples[ivs]->Reweight();
1069  }
1070 
1071  // Get the -log L likelihoods
1072  double samplellh = 0;
1073  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs) {
1074  samplellh += samples[ivs]->GetLikelihood();
1075  }
1076  hScanSam->SetBinContent(x+1, y+1, 2*samplellh);
1077  }// end loop over y points
1078  } // end loop over x points
1079 
1080  Sample_2DLLH->cd();
1081  hScanSam->Write();
1082  // Reset the parameters to their central central values
1083  if (IsPCA) {
1084  cov->GetPCAHandler()->SetParPropPCA(i, central_x);
1085  cov->GetPCAHandler()->SetParPropPCA(j, central_y);
1086  } else {
1087  cov->SetParProp(i, central_x);
1088  cov->SetParProp(j, central_y);
1089  }
1090  } //end loop over systematics y
1091  }//end loop over systematics X
1092  }//end loop covariance classes
1093  Sample_2DLLH->Write();
1094  delete Sample_2DLLH;
1095 }
bool CheckSkipParameter(const std::vector< std::string > &SkipVector, const std::string &ParamName) const
KS: Check whether we want to skip parameter using skip vector.
Definition: FitterBase.cpp:536
void SaveSettings()
Save the settings that the MCMC was run with.
Definition: FitterBase.cpp:80
void GetParameterScanRange(const ParameterHandlerBase *cov, const int i, double &CentralValue, double &lower, double &upper, const int n_points, const std::string &suffix="") const
Helper function to get parameter scan range, central value.
Definition: FitterBase.cpp:543
std::vector< double > CalculateBinEdges(double lowerlimit, double upperlimit, int n_points) const
Definition: FitterBase.cpp:606
void PrintProgressBar(const Long64_t Done, const Long64_t All)
KS: Simply print progress bar.
Definition: Monitor.cpp:229

◆ RunLLHMap()

void FitterBase::RunLLHMap ( )

Perform a general multi-dimensional likelihood scan.

Author
Tomas Nosek

Definition at line 1099 of file FitterBase.cpp.

1099  {
1100 // *************************
1101  // Save the settings into the output file
1102  SaveSettings();
1103 
1104  MACH3LOG_INFO("Starting {}", __func__);
1105 
1106  //KS: Turn it on if you want LLH scan for each ND sample separately, which increase time significantly but can be useful for validating new samples or dials.
1107  bool PlotLLHScanBySample = GetFromManager<bool>(fitMan->raw()["LLHScan"]["LLHScanBySample"], false, __FILE__ , __LINE__);
1108  auto ParamsOfInterest = GetFromManager<std::vector<std::string>>(fitMan->raw()["LLHScan"]["LLHParameters"], {}, __FILE__, __LINE__);
1109 
1110  if(ParamsOfInterest.empty()) {
1111  MACH3LOG_WARN("There were no LLH parameters of interest specified to run the LLHMap! LLHMap will not run at all ...");
1112  return;
1113  }
1114 
1115  // Parameters IDs within the covariance objects
1116  // ParamsCovIDs = {Name, CovObj, IDinCovObj}
1117  std::vector<std::tuple<std::string, ParameterHandlerBase*, int>> ParamsCovIDs;
1118  for(auto& p : ParamsOfInterest) {
1119  bool found = false;
1120  for(auto cov : systematics) {
1121  for(int c = 0; c < cov->GetNumParams(); ++c) {
1122  if(cov->GetParName(c) == p || cov->GetParFancyName(c) == p) {
1123  bool add = true;
1124  for(auto& pc : ParamsCovIDs) {
1125  if(std::get<1>(pc) == cov && std::get<2>(pc) == c)
1126  {
1127  MACH3LOG_WARN("Parameter {} as {}({}) listed multiple times for LLHMap, omitting and using only once!", p, cov->GetName(), c);
1128  add = false;
1129  break;
1130  }
1131  }
1132 
1133  if(add)
1134  ParamsCovIDs.push_back(std::make_tuple(p, cov, c));
1135 
1136  found = true;
1137  break;
1138  }
1139  }
1140  if(found)
1141  break;
1142  }
1143  if(found)
1144  MACH3LOG_INFO("Parameter {} found in {} at an index {}.", p, std::get<1>(ParamsCovIDs.back())->GetName(), std::get<2>(ParamsCovIDs.back()));
1145  else
1146  MACH3LOG_WARN("Parameter {} not found in any of the systematic covariance objects. Will not scan over this one!", p);
1147  }
1148 
1149  // ParamsRanges["parameter"] = {nPoints, {low, high}}
1150  std::map<std::string, std::pair<int, std::pair<double, double>>> ParamsRanges;
1151 
1152  MACH3LOG_INFO("======================================================================================");
1153  MACH3LOG_INFO("Performing a general multi-dimensional LogL map scan over following parameters ranges:");
1154  MACH3LOG_INFO("======================================================================================");
1155  unsigned long TotalPoints = 1;
1156 
1157  double nSigma = GetFromManager<int>(fitMan->raw()["LLHScan"]["LLHScanSigma"], 1., __FILE__, __LINE__);
1158 
1159  // TN: Setting up the scan ranges might look like a re-implementation of the
1160  // FitterBase::GetScanRange, but I guess the use-case here is a bit different.
1161  // Anyway, just in case, we can discuss and rewrite to everyone's liking!
1162  for(auto& p : ParamsCovIDs) {
1163  // Auxiliary vars to help readability
1164  std::string name = std::get<0>(p);
1165  int i = std::get<2>(p);
1166  ParameterHandlerBase* cov = std::get<1>(p);
1167 
1168  ParamsRanges[name].first = GetFromManager<int>(fitMan->raw()["LLHScan"]["LLHScanPoints"], 20, __FILE__, __LINE__);
1169  if(CheckNodeExists(fitMan->raw(),"LLHScan","ScanPoints"))
1170  ParamsRanges[name].first = GetFromManager<int>(fitMan->raw()["LLHScan"]["ScanPoints"][name], ParamsRanges[name].first, __FILE__, __LINE__);
1171 
1172  // Get the parameter priors and bounds
1173  double CentralValue = cov->GetParProp(i);
1174 
1175  bool IsPCA = cov->IsPCA();
1176  if (IsPCA)
1177  CentralValue = cov->GetPCAHandler()->GetParPropPCA(i);
1178 
1179  double prior = cov->GetParPreFit(i);
1180  if (IsPCA)
1181  prior = cov->GetPCAHandler()->GetPreFitValuePCA(i);
1182 
1183  if (std::abs(CentralValue - prior) > 1e-10) {
1184  MACH3LOG_INFO("For {} scanning around value {} rather than prior {}", name, CentralValue, prior);
1185  }
1186  // Get the covariance matrix and do the +/- nSigma
1187  // Set lower and upper bounds relative the CentralValue
1188  // Set the parameter ranges between which LLH points are scanned
1189  double lower = CentralValue - nSigma*cov->GetDiagonalError(i);
1190  double upper = CentralValue + nSigma*cov->GetDiagonalError(i);
1191  // If PCA, transform these parameter values to the PCA basis
1192  if (IsPCA) {
1193  lower = CentralValue - nSigma*std::sqrt((cov->GetPCAHandler()->GetEigenValues())(i));
1194  upper = CentralValue + nSigma*std::sqrt((cov->GetPCAHandler()->GetEigenValues())(i));
1195  MACH3LOG_INFO("eval {} = {:.2f}", i, cov->GetPCAHandler()->GetEigenValues()(i));
1196  MACH3LOG_INFO("CV {} = {:.2f}", i, CentralValue);
1197  MACH3LOG_INFO("lower {} = {:.2f}", i, lower);
1198  MACH3LOG_INFO("upper {} = {:.2f}", i, upper);
1199  MACH3LOG_INFO("nSigma = {:.2f}", nSigma);
1200  }
1201 
1202  ParamsRanges[name].second = {lower,upper};
1203 
1204  if(CheckNodeExists(fitMan->raw(),"LLHScan","ScanRanges"))
1205  ParamsRanges[name].second = GetFromManager<std::pair<double,double>>(fitMan->raw()["LLHScan"]["ScanRanges"][name], ParamsRanges[name].second, __FILE__, __LINE__);
1206 
1207  MACH3LOG_INFO("{} from {:.4f} (lower bin edge) to {:.4f} (upper bin edge) with a {:.5f} step ({} points total)",
1208  name, ParamsRanges[name].second.first, ParamsRanges[name].second.second,
1209  (ParamsRanges[name].second.second - ParamsRanges[name].second.first)/(ParamsRanges[name].first),
1210  ParamsRanges[name].first);
1211 
1212  TotalPoints *= ParamsRanges[name].first;
1213  }
1214 
1215  // TN: Waiting for C++ 20 std::format() function
1216  MACH3LOG_INFO("In total, looping over {} points, from {} parameters. Estimates for run time:", TotalPoints, ParamsCovIDs.size());
1217  MACH3LOG_INFO(" 1 s per point = {} hours", double(TotalPoints)/3600.);
1218  MACH3LOG_INFO(" 0.1 s per point = {} hours", double(TotalPoints)/36000.);
1219  MACH3LOG_INFO("0.01 s per point = {} hours", double(TotalPoints)/360000.);
1220  MACH3LOG_INFO("==================================================================================");
1221 
1222  const int countwidth = int(double(TotalPoints)/double(20));
1223 
1224  // Tree to store LogL values
1225  auto LLHMap = new TTree("llhmap", "LLH Map");
1226 
1227  std::vector<double> CovLogL(systematics.size());
1228  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc )
1229  {
1230  std::string NameTemp = systematics[ivc]->GetName();
1231  NameTemp = NameTemp.substr(0, NameTemp.find("_cov")) + "_LLH";
1232  LLHMap->Branch(NameTemp.c_str(), &CovLogL[ivc]);
1233  }
1234 
1235  std::vector<double> SampleClassLogL(samples.size());
1236  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
1237  {
1238  std::string NameTemp = samples[ivs]->GetName()+"_LLH";
1239  LLHMap->Branch(NameTemp.c_str(), &SampleClassLogL[ivs]);
1240  }
1241 
1242  double SampleLogL, TotalLogL;
1243  LLHMap->Branch("Sample_LLH", &SampleLogL);
1244  LLHMap->Branch("Total_LLH", &TotalLogL);
1245 
1246  std::vector<double>SampleSplitLogL;
1247  if(PlotLLHScanBySample)
1248  {
1249  SampleSplitLogL.resize(TotalNSamples);
1250  int SampleIterator = 0;
1251  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
1252  {
1253  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is )
1254  {
1255  std::string NameTemp =samples[ivs]->GetSampleTitle(is)+"_LLH";
1256  LLHMap->Branch(NameTemp.c_str(), &SampleSplitLogL[SampleIterator]);
1257  SampleIterator++;
1258  }
1259  }
1260  }
1261 
1262  std::vector<double> ParamsValues(ParamsCovIDs.size());
1263  for(unsigned int i=0; i < ParamsCovIDs.size(); ++i)
1264  LLHMap->Branch(std::get<0>(ParamsCovIDs[i]).c_str(), &ParamsValues[i]);
1265 
1266  // Setting up the scan
1267  // Starting at index {0,0,0,...}
1268  std::vector<unsigned long> idx(ParamsCovIDs.size(), 0);
1269 
1270  // loop over scanned points sp
1271  for(unsigned long sp = 0; sp < TotalPoints; ++sp)
1272  {
1273  // At each point need to find the indices and test values to calculate LogL
1274  for(unsigned int n = 0; n < ParamsCovIDs.size(); ++n)
1275  {
1276  // Auxiliaries
1277  std::string name = std::get<0>(ParamsCovIDs[n]);
1278  int points = ParamsRanges[name].first;
1279  double low = ParamsRanges[name].second.first;
1280  double high = ParamsRanges[name].second.second;
1281 
1282  // Find the n-th index of the sp-th scanned point
1283  unsigned long dev = 1;
1284  for(unsigned int m = 0; m <= n; ++m)
1285  dev *= ParamsRanges[std::get<0>(ParamsCovIDs[m])].first;
1286 
1287  idx[n] = sp % dev;
1288  if (n > 0)
1289  idx[n] = idx[n] / ( dev / points );
1290 
1291  // Parameter test value = low + ( high - low ) * idx / #points
1292  ParamsValues[n] = low + (2 * double(idx[n]) + 1) * (high-low) / (2 * double(points));
1293 
1294  // Now set the covariance objects
1295  // Auxiliary
1296  ParameterHandlerBase* cov = std::get<1>(ParamsCovIDs[n]);
1297  int i = std::get<2>(ParamsCovIDs[n]);
1298 
1299  if(cov->IsPCA())
1300  cov->GetPCAHandler()->SetParPropPCA(i, ParamsValues[n]);
1301  else
1302  cov->SetParProp(i, ParamsValues[n]);
1303  }
1304 
1305  // Reweight samples and calculate LogL
1306  TotalLogL = .0;
1307  SampleLogL = .0;
1308 
1309  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs)
1310  samples[ivs]->Reweight();
1311 
1312  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs)
1313  {
1314  SampleClassLogL[ivs] = 2.*samples[ivs]->GetLikelihood();
1315  SampleLogL += SampleClassLogL[ivs];
1316  }
1317  TotalLogL += SampleLogL;
1318 
1319  // CovObjs LogL
1320  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc )
1321  {
1322  CovLogL[ivc] = 2.*systematics[ivc]->GetLikelihood();
1323  TotalLogL += CovLogL[ivc];
1324  }
1325 
1326  if(PlotLLHScanBySample)
1327  {
1328  int SampleIterator = 0;
1329  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
1330  {
1331  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is)
1332  {
1333  SampleSplitLogL[SampleIterator] = 2.*samples[ivs]->GetSampleLikelihood(is);
1334  SampleIterator++;
1335  }
1336  }
1337  }
1338 
1339  LLHMap->Fill();
1340 
1341  if (sp % countwidth == 0)
1342  M3::Utils::PrintProgressBar(sp, TotalPoints);
1343  }
1344 
1345  outputFile->cd();
1346  LLHMap->Write();
1347 }
bool CheckNodeExists(const YAML::Node &node, Args... args)
KS: Wrapper function to call the recursive helper.
Definition: YamlHelper.h:60
void SetParPropPCA(const int i, const double value)
Set proposed value for parameter in PCA base.
Definition: PCAHandler.h:133
void SetParProp(const int i, const double val)
Set proposed parameter value.

◆ RunLLHScan()

void FitterBase::RunLLHScan ( )

Perform a 1D likelihood scan.

Definition at line 622 of file FitterBase.cpp.

622  {
623 // *************************
624  // Save the settings into the output file
625  SaveSettings();
626 
627  MACH3LOG_INFO("Starting {}", __func__);
628 
629  //KS: Turn it on if you want LLH scan for each ND sample separately, which increase time significantly but can be useful for validating new samples or dials.
630  bool PlotLLHScanBySample = GetFromManager<bool>(fitMan->raw()["LLHScan"]["LLHScanBySample"], false, __FILE__ , __LINE__);
631  auto SkipVector = GetFromManager<std::vector<std::string>>(fitMan->raw()["LLHScan"]["LLHScanSkipVector"], {}, __FILE__ , __LINE__);
632  //Do we want a logarithmic LLH scan
633  bool LLHLogarithmic = GetFromManager<bool>(fitMan->raw()["LLHScan"]["LLHLogarithmic"], false, __FILE__ , __LINE__);
634 
635  // Now finally get onto the LLH scan stuff
636  // Very similar code to MCMC but never start MCMC; just scan over the parameter space
637  std::vector<TDirectory *> Cov_LLH(systematics.size());
638  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc )
639  {
640  std::string NameTemp = systematics[ivc]->GetName();
641  NameTemp = NameTemp.substr(0, NameTemp.find("_cov")) + "_LLH";
642  Cov_LLH[ivc] = outputFile->mkdir(NameTemp.c_str());
643  }
644 
645  std::vector<TDirectory *> SampleClass_LLH(samples.size());
646  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
647  {
648  std::string NameTemp = samples[ivs]->GetName();
649  SampleClass_LLH[ivs] = outputFile->mkdir(NameTemp.c_str());
650  }
651 
652  TDirectory *Sample_LLH = outputFile->mkdir("Sample_LLH");
653  TDirectory *Total_LLH = outputFile->mkdir("Total_LLH");
654 
655  std::vector<TDirectory *>SampleSplit_LLH;
656  if(PlotLLHScanBySample)
657  {
658  SampleSplit_LLH.resize(TotalNSamples);
659  int SampleIterator = 0;
660  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
661  {
662  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is )
663  {
664  SampleSplit_LLH[SampleIterator] = outputFile->mkdir((samples[ivs]->GetSampleTitle(is)+ "_LLH").c_str());
665  SampleIterator++;
666  }
667  }
668  }
669  // Number of points we do for each LLH scan
670  const int n_points = GetFromManager<int>(fitMan->raw()["LLHScan"]["LLHScanPoints"], 100, __FILE__ , __LINE__);
671 
672  // We print 5 reweights
673  const int countwidth = int(double(n_points)/double(5));
674 
675  // Loop over the covariance classes
676  for (ParameterHandlerBase *cov : systematics)
677  {
678  // Scan over all the parameters
679  // Get the number of parameters
680  int npars = cov->GetNumParams();
681  bool IsPCA = cov->IsPCA();
682  if (IsPCA) npars = cov->GetNParameters();
683  for (int i = 0; i < npars; ++i)
684  {
685  // Get the parameter name
686  std::string name = cov->GetParFancyName(i);
687  if (IsPCA) name += "_PCA";
688  // KS: Check if we want to skip this parameter
689  if(CheckSkipParameter(SkipVector, name)) continue;
690  // Get the parameter central and bounds
691  double CentralValue, lower, upper;
692  GetParameterScanRange(cov, i, CentralValue, lower, upper, n_points);
693  // Define the TH1D
694  std::unique_ptr<TH1D> hScan;
695  // See if we want to do it logarithmically
696  if(LLHLogarithmic){
697  //negative values break it
698  if (lower < 0.0 || upper < 0.0) {
699  MACH3LOG_WARN("Cannot perform logarithmic scan for {} "" with range [{}, {}], falling back to linear scan",name, lower, upper);
700  hScan = std::make_unique<TH1D>((name + "_full").c_str(), (name + "_full").c_str(), n_points, lower, upper);
701  } else {
702  auto binEdges = CalculateBinEdges(lower, upper, n_points);
703  hScan = std::make_unique<TH1D>((name + "_full").c_str(), (name + "_full").c_str(), n_points, binEdges.data());
704  }
705  }
706  //If not then just do the normal thing
707  else {
708  hScan = std::make_unique<TH1D>((name + "_full").c_str(), (name + "_full").c_str(), n_points, lower, upper);
709  }
710 
711  hScan->SetTitle((std::string("2LLH_full, ") + name + ";" + name + "; -2(ln L_{sample} + ln L_{xsec+flux} + ln L_{det})").c_str());
712  hScan->SetDirectory(nullptr);
713 
714  auto hScanSam = std::make_unique<TH1D>((name + "_sam").c_str(), (name + "_sam").c_str(), n_points, lower, upper);
715  hScanSam->SetTitle((std::string("2LLH_sam, ") + name + ";" + name + "; -2(ln L_{sample})").c_str());
716  hScanSam->SetDirectory(nullptr);
717 
718  std::vector<std::unique_ptr<TH1D>> hScanSample(samples.size());
719  std::vector<double> nSamLLH(samples.size(), 0.0);
720  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
721  {
722  std::string NameTemp = samples[ivs]->GetName();
723  hScanSample[ivs] = std::make_unique<TH1D>((name+"_"+NameTemp).c_str(), (name+"_" + NameTemp).c_str(), n_points, lower, upper);
724  hScanSample[ivs]->SetDirectory(nullptr);
725  hScanSample[ivs]->SetTitle(("2LLH_" + NameTemp + ", " + name + ";" + name + "; -2(ln L_{" + NameTemp +"})").c_str());
726  }
727 
728  std::vector<std::unique_ptr<TH1D>> hScanCov(systematics.size());
729  std::vector<double> nCovLLH(systematics.size(), 0.0);
730  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc )
731  {
732  std::string NameTemp = systematics[ivc]->GetName();
733  NameTemp = NameTemp.substr(0, NameTemp.find("_cov"));
734  hScanCov[ivc] = std::make_unique<TH1D>((name + "_" + NameTemp).c_str(), (name + "_" + NameTemp).c_str(), n_points, lower, upper);
735  hScanCov[ivc]->SetDirectory(nullptr);
736  hScanCov[ivc]->SetTitle(("2LLH_" + NameTemp + ", " + name + ";" + name + "; -2(ln L_{" + NameTemp +"})").c_str());
737  }
738 
739  std::vector<std::unique_ptr<TH1D>> hScanSamSplit;
740  std::vector<double> sampleSplitllh;
741  if(PlotLLHScanBySample)
742  {
743  int SampleIterator = 0;
744  hScanSamSplit.resize(TotalNSamples);
745  sampleSplitllh.resize(TotalNSamples);
746  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
747  {
748  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is )
749  {
750  auto histName = name + samples[ivs]->GetSampleTitle(is);
751  auto histTitle = std::string("2LLH_sam, ") + name + ";" + name + "; -2(ln L_{sample})";
752  hScanSamSplit[SampleIterator] = std::make_unique<TH1D>(histName.c_str(), histTitle.c_str(), n_points, lower, upper);
753  hScanSamSplit[SampleIterator]->SetDirectory(nullptr);
754  SampleIterator++;
755  }
756  }
757  }
758 
759  // Scan over the parameter space
760 
761  for (int j = 0; j < n_points; ++j)
762  {
763  if (j % countwidth == 0)
764  M3::Utils::PrintProgressBar(j, n_points);
765 
766  // For PCA we have to do it differently
767  if (IsPCA) {
768  cov->GetPCAHandler()->SetParPropPCA(i, hScan->GetBinCenter(j+1));
769  } else {
770  // Set the parameter
771  cov->SetParProp(i, hScan->GetBinCenter(j+1));
772  }
773 
774  // Reweight the MC
775  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs ){
776  samples[ivs]->Reweight();
777  }
778  //Total LLH
779  double totalllh = 0.;
780 
781  // Get the -log L likelihoods
782  double samplellh = 0.;
783 
784  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs ) {
785  nSamLLH[ivs] = samples[ivs]->GetLikelihood();
786  samplellh += nSamLLH[ivs];
787  }
788 
789  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc ) {
790  nCovLLH[ivc] = systematics[ivc]->GetLikelihood();
791  totalllh += nCovLLH[ivc];
792  }
793 
794  totalllh += samplellh;
795 
796  if(PlotLLHScanBySample)
797  {
798  int SampleIterator = 0;
799  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
800  {
801  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is)
802  {
803  sampleSplitllh[SampleIterator] = samples[ivs]->GetSampleLikelihood(is);
804  SampleIterator++;
805  }
806  }
807  }
808 
809  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs ) {
810  hScanSample[ivs]->SetBinContent(j+1, 2*nSamLLH[ivs]);
811  }
812  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc ) {
813  hScanCov[ivc]->SetBinContent(j+1, 2*nCovLLH[ivc]);
814  }
815 
816  hScanSam->SetBinContent(j+1, 2*samplellh);
817  hScan->SetBinContent(j+1, 2*totalllh);
818 
819  if(PlotLLHScanBySample)
820  {
821  int SampleIterator = 0;
822  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
823  {
824  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is)
825  {
826  hScanSamSplit[SampleIterator]->SetBinContent(j+1, 2*sampleSplitllh[SampleIterator]);
827  SampleIterator++;
828  }
829  }
830  }
831  }
832  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc )
833  {
834  Cov_LLH[ivc]->cd();
835  hScanCov[ivc]->Write();
836  }
837 
838  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
839  {
840  SampleClass_LLH[ivs]->cd();
841  hScanSample[ivs]->Write();
842  }
843  Sample_LLH->cd();
844  hScanSam->Write();
845  Total_LLH->cd();
846  hScan->Write();
847 
848  if(PlotLLHScanBySample)
849  {
850  int SampleIterator = 0;
851  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
852  {
853  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is)
854  {
855  SampleSplit_LLH[SampleIterator]->cd();
856  hScanSamSplit[SampleIterator]->Write();
857  SampleIterator++;
858  }
859  }
860  }
861 
862  // Reset the parameters to their CentralValue central values
863  if (IsPCA) {
864  cov->GetPCAHandler()->SetParPropPCA(i, CentralValue);
865  } else {
866  cov->SetParProp(i, CentralValue);
867  }
868  }//end loop over systematics
869  }//end loop covariance classes
870 
871  for(unsigned int ivc = 0; ivc < systematics.size(); ++ivc )
872  {
873  Cov_LLH[ivc]->Write();
874  delete Cov_LLH[ivc];
875  }
876 
877  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
878  {
879  SampleClass_LLH[ivs]->Write();
880  delete SampleClass_LLH[ivs];
881  }
882 
883  Sample_LLH->Write();
884  delete Sample_LLH;
885 
886  Total_LLH->Write();
887  delete Total_LLH;
888 
889  if(PlotLLHScanBySample)
890  {
891  int SampleIterator = 0;
892  for(unsigned int ivs = 0; ivs < samples.size(); ++ivs )
893  {
894  for(int is = 0; is < samples[ivs]->GetNSamples(); ++is )
895  {
896  SampleSplit_LLH[SampleIterator]->Write();
897  delete SampleSplit_LLH[SampleIterator];
898  SampleIterator++;
899  }
900  }
901  }
902 }

◆ RunMCMC()

virtual void FitterBase::RunMCMC ( )
pure virtual

The specific fitting algorithm implemented in this function depends on the derived class. It could be Markov Chain Monte Carlo (MCMC), MinuitFit, or another algorithm.

Implemented in PyLikelihoodFit, PyFitterBase, NuDockServerBase, PredictiveThrower, PSO, MinuitFit, and MCMCBase.

◆ RunSigmaVar()

void FitterBase::RunSigmaVar ( )

Perform a 1D/2D sigma var for all samples.

Apply custom range to make easier comparison with p-theta

Definition at line 1445 of file FitterBase.cpp.

1445  {
1446 // *************************
1447  // Save the settings into the output file
1448  SaveSettings();
1449 
1450  bool plot_by_mode = GetFromManager<bool>(fitMan->raw()["SigmaVar"]["PlotByMode"], false, __FILE__ , __LINE__);
1451  bool plot_by_channel = GetFromManager<bool>(fitMan->raw()["SigmaVar"]["PlotByChannel"], false, __FILE__ , __LINE__);
1452  auto SkipVector = GetFromManager<std::vector<std::string>>(fitMan->raw()["SigmaVar"]["SkipVector"], {}, __FILE__ , __LINE__);
1453 
1454  if (plot_by_mode) MACH3LOG_INFO("Plotting by sample and mode");
1455  if (plot_by_channel) MACH3LOG_INFO("Plotting by sample and channel");
1456  if (!plot_by_mode && !plot_by_channel) MACH3LOG_INFO("Plotting by sample only");
1457  if (plot_by_mode && plot_by_channel) MACH3LOG_INFO("Plotting by sample, mode and channel");
1458 
1459  auto SigmaArray = GetFromManager<std::vector<double>>(fitMan->raw()["SigmaVar"]["SigmaArray"], {-3, -1, 0, 1, 3}, __FILE__ , __LINE__);
1460  if (std::find(SigmaArray.begin(), SigmaArray.end(), 0.0) == SigmaArray.end()) {
1461  MACH3LOG_ERROR(":: SigmaArray does not contain 0! Current contents: {} ::", fmt::join(SigmaArray, ", "));
1462  throw MaCh3Exception(__FILE__, __LINE__);
1463  }
1464 
1465  TDirectory* SigmaDir = outputFile->mkdir("SigmaVar");
1466  outputFile->cd();
1467 
1468  for (size_t s = 0; s < systematics.size(); ++s)
1469  {
1470  for(int i = 0; i < systematics[s]->GetNumParams(); i++)
1471  {
1472  std::string ParName = systematics[s]->GetParFancyName(i);
1473  // KS: Check if we want to skip this parameter
1474  if(CheckSkipParameter(SkipVector, ParName)) continue;
1475 
1476  MACH3LOG_INFO(":: Param {} ::", systematics[s]->GetParFancyName(i));
1477 
1478  TDirectory* ParamDir = SigmaDir->mkdir(ParName.c_str());
1479  ParamDir->cd();
1480 
1481  const double ParamCentralValue = systematics[s]->GetParProp(i);
1482  const double Prior = systematics[s]->GetParPreFit(i);
1483  const double ParamLower = systematics[s]->GetLowerBound(i);
1484  const double ParamUpper = systematics[s]->GetUpperBound(i);
1485 
1486  if (std::abs(ParamCentralValue - Prior) > 1e-10) {
1487  MACH3LOG_INFO("For {} scanning around value {} rather than prior {}", ParName, ParamCentralValue, Prior);
1488  }
1489 
1490  for(unsigned int iSample = 0; iSample < samples.size(); ++iSample)
1491  {
1492  auto* MaCh3Sample = samples[iSample];
1493  std::vector<TDirectory*> SampleDir(MaCh3Sample->GetNSamples());
1494  for (int SampleIndex = 0; SampleIndex < MaCh3Sample->GetNSamples(); ++SampleIndex) {
1495  SampleDir[SampleIndex] = ParamDir->mkdir(MaCh3Sample->GetSampleTitle(SampleIndex).c_str());
1496  }
1497 
1498  for (size_t j = 0; j < SigmaArray.size(); ++j) {
1499  double sigma = SigmaArray[j];
1500 
1501  double ParamShiftValue = ParamCentralValue + sigma * std::sqrt((*systematics[s]->GetCovMatrix())(i,i));
1502  ParamShiftValue = std::max(std::min(ParamShiftValue, ParamUpper), ParamLower);
1503 
1505  CustomRange(ParName, sigma, ParamShiftValue);
1506 
1507  MACH3LOG_INFO(" - set to {:<5.2f} ({:<2} sigma shift)", ParamShiftValue, sigma);
1508  systematics[s]->SetParProp(i, ParamShiftValue);
1509 
1510  std::ostringstream valStream;
1511  valStream << std::fixed << std::setprecision(2) << ParamShiftValue;
1512  std::string valueStr = valStream.str();
1513 
1514  std::ostringstream sigmaStream;
1515  sigmaStream << std::fixed << std::setprecision(2) << std::abs(sigma);
1516  std::string sigmaStr = sigmaStream.str();
1517 
1518  std::string suffix;
1519  if (sigma == 0) {
1520  suffix = "_" + ParName + "_nom_val_" + valueStr;
1521  } else {
1522  std::string sign = (sigma > 0) ? "p" : "n";
1523  suffix = "_" + ParName + "_sig_" + sign + sigmaStr + "_val_" + valueStr;
1524  }
1525 
1526  systematics[s]->SetParProp(i, ParamShiftValue);
1527  MaCh3Sample->Reweight();
1528 
1529  WriteHistogramsByMode(MaCh3Sample, suffix, plot_by_mode, plot_by_channel, SampleDir);
1530  }
1531  for (int subSampleIndex = 0; subSampleIndex < MaCh3Sample->GetNSamples(); ++subSampleIndex) {
1532  SampleDir[subSampleIndex]->Close();
1533  delete SampleDir[subSampleIndex];
1534  }
1535  ParamDir->cd();
1536  }
1537 
1538  systematics[s]->SetParProp(i, ParamCentralValue);
1539  MACH3LOG_INFO(" - set back to CV {:<5.2f}", ParamCentralValue);
1540  MACH3LOG_INFO("");
1541  ParamDir->Close();
1542  delete ParamDir;
1543  SigmaDir->cd();
1544  } // end loop over params
1545  } // end loop over systemics
1546  SigmaDir->Close();
1547  delete SigmaDir;
1548 
1549  outputFile->cd();
1550 }
void WriteHistogramsByMode(SampleHandlerInterface *sample, const std::string &suffix, const bool by_mode, const bool by_channel, const std::vector< TDirectory * > &SampleDir)
Generic histogram writer - should make main code more palatable.
void CustomRange(const std::string &ParName, const double sigma, double &ParamShiftValue) const
For comparison with other fitting frameworks (like P-Theta) we usually have to apply different parame...

◆ SanitiseInputs()

void FitterBase::SanitiseInputs ( )
protected

Remove obsolete memory and make other checks before fit starts.

Todo:
consider expanding into ParmaterHandler and add more sanitisers

Definition at line 224 of file FitterBase.cpp.

224  {
225 // *******************
226  for (size_t i = 0; i < samples.size(); ++i) {
227  samples[i]->CleanMemoryBeforeFit();
228  }
229 }

◆ SaveOutput()

void FitterBase::SaveOutput ( )
protected

Save output and close files.

Definition at line 232 of file FitterBase.cpp.

232  {
233 // *******************
234  if(FileSaved) return;
235  //Stop Clock
236  clock->Stop();
237 
238  //KS: Some version of ROOT keep spamming about accessing already deleted object which is wrong and not helpful...
239  int originalErrorLevel = gErrorIgnoreLevel;
240  gErrorIgnoreLevel = kFatal;
241 
242  outputFile->cd();
243  outTree->Write();
244 
245  MACH3LOG_INFO("{} steps took {:.2e} seconds to complete. ({:.2e}s / step).", step - stepStart, clock->RealTime(), clock->RealTime() / static_cast<double>(step - stepStart));
246  MACH3LOG_INFO("{} steps were accepted.", accCount);
247  #ifdef MACH3_DEBUG
248  if (debug)
249  {
250  debugFile << "\n\n" << step << " steps took " << clock->RealTime() << " seconds to complete. (" << clock->RealTime() / step << "s / step).\n" << accCount<< " steps were accepted." << std::endl;
251  debugFile.close();
252  }
253  #endif
254 
255  outputFile->Close();
256  FileSaved = true;
257 
258  gErrorIgnoreLevel = originalErrorLevel;
259 }

◆ SaveSettings()

void FitterBase::SaveSettings ( )
protected

Save the settings that the MCMC was run with.

Definition at line 80 of file FitterBase.cpp.

80  {
81 // *******************
82  if(SettingsSaved) return;
83 
84  outputFile->cd();
85 
86  TDirectory* MaCh3Version = outputFile->mkdir("MaCh3Engine");
87  MaCh3Version->cd();
88 
89  if (std::getenv("MaCh3_ROOT") == nullptr) {
90  MACH3LOG_ERROR("Need MaCh3_ROOT environment variable");
91  MACH3LOG_ERROR("Please remember about source bin/setup.MaCh3.sh");
92  throw MaCh3Exception(__FILE__ , __LINE__ );
93  }
94 
95  if (std::getenv("MACH3") == nullptr) {
96  MACH3LOG_ERROR("Need MACH3 environment variable");
97  throw MaCh3Exception(__FILE__ , __LINE__ );
98  }
99 
100  std::string header_path = std::string(std::getenv("MACH3"));
101  header_path += "/version.h";
102  FILE* file = fopen(header_path.c_str(), "r");
103  //KS: It is better to use experiment specific header file. If given experiment didn't provide it we gonna use one given by Core MaCh3.
104  if (!file) {
105  header_path = std::string(std::getenv("MaCh3_ROOT"));
106  header_path += "/version.h";
107  } else {
108  fclose(file);
109  }
110 
111  // EM: embed the cmake generated version.h file
112  TMacro versionHeader("version_header", "version_header");
113  versionHeader.ReadFile(header_path.c_str());
114  versionHeader.Write();
115 
116  if(GetName() == ""){
117  MACH3LOG_ERROR("Name of currently used algorithm is {}", GetName());
118  MACH3LOG_ERROR("Have you forgotten to modify AlgorithmName?");
119  throw MaCh3Exception(__FILE__ , __LINE__ );
120  }
121  TNamed Engine(GetName(), GetName());
122  Engine.Write(GetName().c_str());
123 
124  MaCh3Version->Write();
125  delete MaCh3Version;
126 
127  outputFile->cd();
128 
130 
131  MACH3LOG_WARN("\033[0;31mCurrent Total RAM usage is {:.2f} GB\033[0m", M3::Utils::getValue("VmRSS") / 1048576.0);
132  MACH3LOG_WARN("\033[0;31mOut of Total available RAM {:.2f} GB\033[0m", M3::Utils::getValue("MemTotal") / 1048576.0);
133 
134  MACH3LOG_INFO("#####Current Setup#####");
135  MACH3LOG_INFO("Number of covariances: {}", systematics.size());
136  for(unsigned int i = 0; i < systematics.size(); ++i)
137  MACH3LOG_INFO("{}: Cov name: {}, it has {} params", i, systematics[i]->GetName(), systematics[i]->GetNumParams());
138  MACH3LOG_INFO("Number of SampleHandlers: {}", samples.size());
139  for(unsigned int i = 0; i < samples.size(); ++i) {
140  MACH3LOG_INFO("{}: SampleHandler name: {}, it has {} samples",i , samples[i]->GetName(), samples[i]->GetNSamples());
141  for(int iSam = 0; iSam < samples[i]->GetNSamples(); ++iSam) {
142  MACH3LOG_INFO(" {}: Sample title: {}, with {} osc channels",iSam , samples[i]->GetSampleTitle(iSam), samples[i]->GetNOscChannels(iSam));
143  }
144  }
145  //TN: Have to close the folder in order to write it to disk before SaveOutput is called in the destructor
146  CovFolder->Close();
147  SampleFolder->Close();
148 
149  SettingsSaved = true;
150 }
std::string GetName() const
Get name of class.
Definition: FitterBase.h:75
void SaveSettings(TFile *const OutputFile) const
Add manager useful information's to TFile, in most cases to Fitter.
Definition: Manager.cpp:40
int getValue(const std::string &Type)
CW: Get info like RAM.
Definition: Monitor.cpp:252

◆ StartFromPreviousFit()

void FitterBase::StartFromPreviousFit ( const std::string &  FitName)
virtual

Allow to start from previous fit/chain.

Parameters
FitNameName of previous chain

Reimplemented in MCMCBase.

Definition at line 349 of file FitterBase.cpp.

349  {
350 // *******************
351  MACH3LOG_INFO("Getting starting position from {}", FitName);
352  TFile *infile = M3::Open(FitName, "READ", __FILE__, __LINE__);
353  TTree *posts = infile->Get<TTree>("posteriors");
354  double log_val = M3::_LARGE_LOGL_;
355  posts->SetBranchAddress("LogL",&log_val);
356 
357  for (size_t s = 0; s < systematics.size(); ++s)
358  {
359  TDirectory* CovarianceFolder = infile->Get<TDirectory>("CovarianceFolder");
360 
361  std::string ConfigName = "Config_" + systematics[s]->GetName();
362  TMacro *ConfigCov = CovarianceFolder->Get<TMacro>(ConfigName.c_str());
363  // KS: Not every covariance uses yaml, if it uses yaml make sure they are identical
364  if (ConfigCov != nullptr) {
365  // Config which was in MCMC from which we are starting
366  YAML::Node CovSettings = TMacroToYAML(*ConfigCov);
367  // Config from currently used cov object
368  YAML::Node ConfigCurrent = systematics[s]->GetConfig();
369 
370  if (!compareYAMLNodes(CovSettings, ConfigCurrent))
371  {
372  MACH3LOG_ERROR("Yaml configs in previous chain (from path {}) and current one are different", FitName);
373  throw MaCh3Exception(__FILE__ , __LINE__ );
374  }
375  delete ConfigCov;
376  }
377 
378  CovarianceFolder->Close();
379  delete CovarianceFolder;
380 
381  std::vector<double> branch_vals;
382  std::vector<std::string> branch_name;
383  systematics[s]->MatchMaCh3OutputBranches(posts, branch_vals, branch_name);
384  posts->GetEntry(posts->GetEntries()-1);
385 
386  systematics[s]->SetParameters(branch_vals);
387  systematics[s]->AcceptStep();
388 
389  MACH3LOG_INFO("Printing new starting values for: {}", systematics[s]->GetName());
390  systematics[s]->PrintPreFitCurrPropValues();
391 
392  // Resetting branch addressed to nullptr as we don't want to write into a deleted vector out of scope...
393  for (int i = 0; i < systematics[s]->GetNumParams(); ++i) {
394  posts->SetBranchAddress(systematics[s]->GetParName(i).c_str(), nullptr);
395  }
396  }
397  logLCurr = log_val;
398  logLProp = log_val;
399 
400  delete posts;
401  infile->Close();
402  delete infile;
403 }
YAML::Node TMacroToYAML(const TMacro &macro)
KS: Convert a ROOT TMacro object to a YAML node.
Definition: YamlHelper.h:152
bool compareYAMLNodes(const YAML::Node &node1, const YAML::Node &node2, bool Mute=false)
Compare if yaml nodes are identical.
Definition: YamlHelper.h:186
constexpr static const double _LARGE_LOGL_
Large Likelihood is used it parameter go out of physical boundary, this indicates in MCMC that such s...
Definition: Core.h:80

Member Data Documentation

◆ accCount

int FitterBase::accCount
protected

counts accepted steps

Definition at line 128 of file FitterBase.h.

◆ accProb

double FitterBase::accProb
protected

current acceptance prob

Definition at line 126 of file FitterBase.h.

◆ AlgorithmName

std::string FitterBase::AlgorithmName
protected

Name of fitting algorithm that is being used.

Definition at line 177 of file FitterBase.h.

◆ auto_save

int FitterBase::auto_save
protected

auto save every N steps

Definition at line 164 of file FitterBase.h.

◆ clock

std::unique_ptr<TStopwatch> FitterBase::clock
protected

tells global time how long fit took

Definition at line 146 of file FitterBase.h.

◆ CovFolder

TDirectory* FitterBase::CovFolder
protected

Output cov folder.

Definition at line 158 of file FitterBase.h.

◆ FileSaved

bool FitterBase::FileSaved
protected

Checks if file saved not repeat some operations.

Definition at line 170 of file FitterBase.h.

◆ fitMan

Manager* FitterBase::fitMan
protected

The manager for configuration handling.

Definition at line 117 of file FitterBase.h.

◆ fTestLikelihood

bool FitterBase::fTestLikelihood
protected

Necessary for some fitting algorithms like PSO.

Definition at line 167 of file FitterBase.h.

◆ logLCurr

double FitterBase::logLCurr
protected

current likelihood

Definition at line 122 of file FitterBase.h.

◆ logLProp

double FitterBase::logLProp
protected

proposed likelihood

Definition at line 124 of file FitterBase.h.

◆ outputFile

TFile* FitterBase::outputFile
protected

Output.

Definition at line 156 of file FitterBase.h.

◆ OutputPrepared

bool FitterBase::OutputPrepared
protected

Checks if output prepared not repeat some operations.

Definition at line 174 of file FitterBase.h.

◆ outTree

TTree* FitterBase::outTree
protected

Output tree with posteriors.

Definition at line 162 of file FitterBase.h.

◆ random

std::unique_ptr<TRandom3> FitterBase::random
protected

Random number.

Definition at line 153 of file FitterBase.h.

◆ sample_llh

std::vector<double> FitterBase::sample_llh
protected

store the llh breakdowns

Definition at line 133 of file FitterBase.h.

◆ SampleFolder

TDirectory* FitterBase::SampleFolder
protected

Output sample folder.

Definition at line 160 of file FitterBase.h.

◆ samples

std::vector<SampleHandlerInterface*> FitterBase::samples
protected

Sample holder.

Definition at line 138 of file FitterBase.h.

◆ SettingsSaved

bool FitterBase::SettingsSaved
protected

Checks if setting saved not repeat some operations.

Definition at line 172 of file FitterBase.h.

◆ step

unsigned int FitterBase::step
protected

current state

Definition at line 120 of file FitterBase.h.

◆ stepClock

std::unique_ptr<TStopwatch> FitterBase::stepClock
protected

tells how long single step/fit iteration took

Definition at line 148 of file FitterBase.h.

◆ stepStart

unsigned int FitterBase::stepStart
protected

step start, by default 0 if we start from previous chain then it will be different

Definition at line 130 of file FitterBase.h.

◆ stepTime

double FitterBase::stepTime
protected

Time of single step.

Definition at line 150 of file FitterBase.h.

◆ syst_llh

std::vector<double> FitterBase::syst_llh
protected

systematic llh breakdowns

Definition at line 135 of file FitterBase.h.

◆ systematics

std::vector<ParameterHandlerBase*> FitterBase::systematics
protected

Systematic holder.

Definition at line 143 of file FitterBase.h.

◆ TotalNSamples

unsigned int FitterBase::TotalNSamples
protected

Total number of samples used, single SampleHandler can store more than one analysis sample!

Definition at line 140 of file FitterBase.h.


The documentation for this class was generated from the following files: