KS: Class handling binning for multiple samples. More...

#include <Samples/BinningHandler.h>

Collaboration diagram for BinningHandler:

Public Member Functions
	BinningHandler ()
	Constructor. More...

virtual	~BinningHandler ()
	destructor More...

int	FindGlobalBin (const int iSample, const std::vector< const double * > &KinVar, const std::vector< int > &NomBin) const
	Find Global bin including. More...

int	FindNominalBin (const int iSample, const int iDim, const double Var) const
	Find the nominal bin for a given variable in a given sample and dimension. More...

int	GetGlobalBinSafe (const int iSample, const std::vector< int > &Bins) const
	Get gloabl bin based on sample, and dimension of each sample with additional checks. More...

int	GetBinSafe (const int iSample, const std::vector< int > &Bins) const
	Get gloabl bin based on sample, and dimension of each sample without any safety checks. More...

int	GetNBins () const
	Get total number of bins over all samples/kinematic bins etc. More...

int	GetNBins (const int iSample) const
	Get total number of bins over for a given sample. More...

int	GetNDim (const int iSample) const
	Get total number of dimensions for sample iSample. More...

std::string	GetBinName (const int GlobalBin) const
	Get fancy name for a given bin, to help match it with global properties. More...

std::string	GetBinName (const int iSample, const int SampleBin) const
	Get fancy name for a given bin, to help match it with global properties. More...

std::string	GetBinName (const int iSample, const std::vector< int > &Bins) const
	Get fancy name for a given bin, to help match it with global properties. More...

std::vector< double >	GetBinEdges (const int iSample, const int iDim) const
	Get N-dim bin edges for a given sample. More...

int	GetNAxisBins (const int iSample, const int iDim) const
	Get Number of N-axis bins for a given sample. More...

bool	IsUniform (const int iSample) const
	Tells whether given sample is using unform binning. More...

int	GetSampleStartBin (const int iSample) const
	Get bin number corresponding to where given sample starts. More...

int	GetSampleEndBin (const int iSample) const
	Get bin number corresponding to where given sample ends. More...

const std::vector< BinInfo >	GetNonUniformBins (const int iSample) const
	Return NonUnifomr bins to for example check extent etc. More...

void	SetGlobalBinNumbers ()
	Sets the GlobalOffset for each SampleBinningInfo to enable linearization of multiple 2D binning samples. More...

void	SetupSampleBinning (const YAML::Node &Settings, SampleInfo &SingleSample)
	Function to setup the binning of your sample histograms and the underlying arrays that get handled in fillArray() and fillArray_MP(). More...

Private Attributes
int	TotalNumberOfBins
	Total number of bins. More...

std::vector< SampleBinningInfo >	SampleBinning
	Binning info for individual sample. More...

Detailed Description

KS: Class handling binning for multiple samples.

Introduction

Each sample can define its own binning in an arbitrary number of dimensions. Internally, every sample's multi-dimensional binning is linearised into a single 1D array. All samples are then concatenated into one global bin index space, allowing the entire analysis to be treated as a single large vector of bins.

The concept of a "global bin" refers to the position of a bin in this linearised, analysis-wide bin index space. Local (sample) bins are always enumerated starting from zero, while global bins span all samples consecutively.

Example layout of global bins with offsets:

Sample 0 (GlobalOffset = 0,  nBins = 4):
  Local bins:   [0] [1] [2] [3]
  Global bins:  [0] [1] [2] [3]
 
Sample 1 (GlobalOffset = 4,  nBins = 3):
  Local bins:   [0] [1] [2]
  Global bins:  [4] [5] [6]
 
Sample 2 (GlobalOffset = 7,  nBins = 2):
  Local bins:   [0] [1]
  Global bins:  [7] [8]
 
Global bin index space:
------------------------------------------------
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
------------------------------------------------

Uniform and Non-Uniform Binning Scheme

MaCh3 supports Uniform and Non-Uniform binning scheme

Uniform binning uses a regular Cartesian grid. Bins are constructed as the Cartesian product of 1D bin edges, forming axis-aligned rectangles (or hyper-rectangles in higher dimensions). Bin widths may be equal or variable, but the grid structure is regular.

In the non-uniform scheme, bin sizes may vary and bins may span different ranges along each dimension. However, all bins must remain axis-aligned hyper-rectangles. Arbitrary or irregular bin shapes (e.g. curved or "banana-shaped" bins) are not supported.

Example of Uniform

+--------+--------+--------+--------+
| Bin 0  | Bin 1  | Bin 2  | Bin 3  |
| (0,0)  | (1,0)  | (2,0)  | (3,0)  |
+--------+--------+--------+--------+
| Bin 4  | Bin 5  | Bin 6  | Bin 7  |
| (0,1)  | (1,1)  | (2,1)  | (3,1)  |
+--------+--------+--------+--------+
| Bin 8  | Bin 9  | Bin 10 | Bin 11 |
| (0,2)  | (1,2)  | (2,2)  | (3,2)  |
+--------+--------+--------+--------+

Example of Non-Uniform

+--------+------------+-------+---------------------+
| Bin 0  |    Bin 1   | Bin 2 |         Bin 3       |
|        |            |       |                     |
+--------+------------+-------+---------------------+
| Bin 4  |            |       |                     |
|        |            | Bin 6 |                     |
+--------+   Bin 5    +-------+                     |
| Bin 7  |            |       |         Bin 9       |
|        |            | Bin 8 |                     |
+--------+------------+-------+---------------------+

Bin Finding Algorithm

Since MaCh3 supports event migration bin finding algorithm must be fast to efficiently be able find bin during running fit. MaCh3 is caching nominal bin with idea that during fit migration should be around this nominal bin. Thus MaCh3 first checks if after shift event falls into Nom-bin and later adjacent. If not backs to binary search.

Uniform

In case of uniform binning above algorithm is easy to test as one performs it for every dimension independently i.e. find X-bin, then Y etc. After which can find bin in flattened 1D space.

Non-Uniform

Internally, non-uniform binning is implemented using two levels:

MegaBins (mapping bins) These form a coarse, uniform grid that spans the full phase space. Each MegaBin acts as a container for one or more non-uniform bins.
Non-Uniform bins The actual analysis bins, defined as hyper-rectangles with arbitrary extents inside a MegaBin.

Bin finding procedure

For a given event, the bin-finding algorithm proceeds as follows:

Locate the MegaBin using the same fast per-dimension logic as in uniform binning.
Once the MegaBin is identified, loop over all non-uniform bins associated with that MegaBin.
The first bin whose extents fully contain the event is selected.
If no bin matches, the event is assigned to the under/overflow bin.

Author: Kamil Skwarczynski; Dan Barrow; Luke Pickering

Definition at line 129 of file BinningHandler.h.

Constructor & Destructor Documentation

◆ BinningHandler()

BinningHandler::BinningHandler ( )

Constructor.

Definition at line 4 of file BinningHandler.cpp.

                                {
 // ************************************************
 }

◆ ~BinningHandler()

virtual BinningHandler::~BinningHandler ( )

inlinevirtual

destructor

Definition at line 136 of file BinningHandler.h.

136 {};

Member Function Documentation

◆ FindGlobalBin()

int BinningHandler::FindGlobalBin	(	const int	iSample,
		const std::vector< const double * > &	KinVar,
		const std::vector< int > &	NomBin
	)		const

Find Global bin including.

Parameters

iSample	index of a given sample
KinVar	Vector of pointers to kinematic variable like Erec
NomBin	Vector of nominal bin indices for this event, one per dimension.

Definition at line 265 of file BinningHandler.cpp.

                                                                       {
 // ************************************************
   //DB Find the relevant bin in the PDF for each event
   const int Dim = static_cast<int>(KinVar.size());
   const SampleBinningInfo& _restrict_ Binning = SampleBinning[NomSample];
   int GlobalBin = 0;
  
   for(int i = 0; i < Dim; ++i) {
     const double Var = *KinVar[i];
     const int Bin = Binning.FindBin(i, Var, NomBin[i]);
     // KS: If we are outside of range in only one dimension this mean out of bounds, we can simply quickly finish
     if(Bin < 0) return M3::UnderOverFlowBin;
     // KS: inline GetBin computation to avoid any memory allocation, which in reweight loop is very costly
     GlobalBin += Bin * Binning.Strides[i];
   }
  
   if(Binning.Uniform) {
     GlobalBin += static_cast<int>(Binning.GlobalOffset);
     return GlobalBin;
   } else {
     const auto& _restrict_ BinMapping = Binning.BinGridMapping[GlobalBin];
     const size_t nNonUniBins = BinMapping.size();
     for(size_t iBin = 0; iBin < nNonUniBins; iBin++) {
       const int BinNumber = BinMapping[iBin];
       const auto& _restrict_ NonUniBin = Binning.Bins[BinNumber];
       if(NonUniBin.IsEventInside(KinVar)){
         return BinNumber + Binning.GlobalOffset;
       }
     }
     MACH3LOG_DEBUG("Didn't find any bin so returning UnderOverFlowBin");
     return M3::UnderOverFlowBin;
   }
 }

◆ FindNominalBin()

int BinningHandler::FindNominalBin	(	const int	iSample,
		const int	iDim,
		const double	Var
	)		const

Find the nominal bin for a given variable in a given sample and dimension.

Parameters

iSample	Sample index
iDim	Dimension index (0 = X, 1 = Y, ...)
Var	Kinematic variable value

Definition at line 302 of file BinningHandler.cpp.

                                            {
 // ************************************************
   const SampleBinningInfo& info = SampleBinning[iSample];
  
   const auto& edges = info.BinEdges[iDim];
  
   // Outside binning range
   if (Var < edges.front() || Var >= edges.back()) {
     return M3::UnderOverFlowBin;
   }
   return static_cast<int>(std::distance(edges.begin(), std::upper_bound(edges.begin(), edges.end(), Var)) - 1);
 }

◆ GetBinEdges()

std::vector<double> BinningHandler::GetBinEdges	(	const int	iSample,
		const int	iDim
	)		const

inline

Get N-dim bin edges for a given sample.

Parameters

iSample	index of a given sample
iDim	dimension for which we extract bin edges

Definition at line 181 of file BinningHandler.h.

181 {return SampleBinning[iSample].BinEdges.at(iDim);};

◆ GetBinName() [1/3]

std::string BinningHandler::GetBinName ( const int GlobalBin ) const

Get fancy name for a given bin, to help match it with global properties.

Parameters

GlobalBin Global Bin integrated over all samples

Definition at line 433 of file BinningHandler.cpp.

                                                               {
 // ************************************************
   int SampleBin = GetSampleFromGlobalBin(SampleBinning, GlobalBin);
   int LocalBin  = GetLocalBinFromGlobalBin(SampleBinning, GlobalBin);
   return GetBinName(SampleBin, LocalBin);
 }

◆ GetBinName() [2/3]

std::string BinningHandler::GetBinName	(	const int	iSample,
		const int	SampleBin
	)		const

Get fancy name for a given bin, to help match it with global properties.

Parameters

iSample	index of a given sample
SampleBin	Global Bin for a given sample

Definition at line 379 of file BinningHandler.cpp.

                                                                                  {
 // ************************************************
   const auto& Binning = SampleBinning[iSample];
  
   // Safety checks
   if (SampleBin < 0 || SampleBin >= static_cast<int>(Binning.nBins)) {
     MACH3LOG_ERROR("Requested bin {} is out of range for sample {}", SampleBin, iSample);
     throw MaCh3Exception(__FILE__, __LINE__);
   }
   std::string BinName;
  
   if(Binning.Uniform) {
     int Dim = static_cast<int>(Binning.Strides.size());
     std::vector<int> Bins(Dim, 0);
     int Remaining = SampleBin;
  
     // Convert the flat/global bin index into per-dimension indices
     // Dim0 is the fastest-changing axis, Dim1 the next, etc.
     //
     // For example (2D):
     //   x = bin % Nx
     //   y = bin / Nx
     //
     // For 3D:
     //   x = bin % Nx
     //   y = (bin / Nx) % Ny
     //   z = bin / (Nx * Ny)
     for (int i = 0; i < Dim; ++i) {
       const int nBinsDim = static_cast<int>(Binning.BinEdges[i].size()) - 1;
       Bins[i] = Remaining % nBinsDim;
       Remaining /= nBinsDim;
     }
  
     for (int i = 0; i < Dim; ++i) {
       if (i > 0) BinName += ", ";
       const double min = Binning.BinEdges[i].at(Bins[i]);
       const double max = Binning.BinEdges[i].at(Bins[i] + 1);
       BinName += fmt::format("Dim{} ({:g}, {:g})", i, min, max);
     }
   } else{
     const BinInfo& bin = Binning.Bins[SampleBin];
     const int Dim = static_cast<int>(bin.Extent.size());
  
     for (int i = 0; i < Dim; ++i) {
       if (i > 0) BinName += ", ";
       const double min = bin.Extent[i][0];
       const double max = bin.Extent[i][1];
       BinName += fmt::format("Dim{} ({:g}, {:g})", i, min, max);
     }
   }
   return BinName;
 }

◆ GetBinName() [3/3]

std::string BinningHandler::GetBinName	(	const int	iSample,
		const std::vector< int > &	Bins
	)		const

Get fancy name for a given bin, to help match it with global properties.

Parameters

iSample	index of a given sample
Bins	Vector of bin indices along each dimension

Definition at line 367 of file BinningHandler.cpp.

                                                                                         {
 // ************************************************
   const auto& Binning = SampleBinning[iSample];
   if(!Binning.Uniform) {
     MACH3LOG_ERROR("When using Non-Uniform binning for sample {} please use One bin instead of Axis bins", iSample);
     throw MaCh3Exception(__FILE__, __LINE__);
   }
   return GetBinName(iSample, GetBinSafe(iSample, Bins));
 }

◆ GetBinSafe()

int BinningHandler::GetBinSafe	(	const int	iSample,
		const std::vector< int > &	Bins
	)		const

Get gloabl bin based on sample, and dimension of each sample without any safety checks.

Parameters

iSample	index of a given sample
Bins	Vector of bin indices along each dimension

Definition at line 318 of file BinningHandler.cpp.

                                                                                  {
 // ************************************************
   const int GlobalBin = SampleBinning[Sample].GetBinSafe(Bins);
   return GlobalBin;
 }

◆ GetGlobalBinSafe()

int BinningHandler::GetGlobalBinSafe	(	const int	iSample,
		const std::vector< int > &	Bins
	)		const

Get gloabl bin based on sample, and dimension of each sample with additional checks.

Parameters

iSample	index of a given sample
Bins	Vector of bin indices along each dimension

Definition at line 325 of file BinningHandler.cpp.

                                                                                        {
 // ************************************************
   const int GlobalBin = SampleBinning[Sample].GetBinSafe(Bins) + static_cast<int>(SampleBinning[Sample].GlobalOffset);
   return GlobalBin;
 }

◆ GetNAxisBins()

int BinningHandler::GetNAxisBins	(	const int	iSample,
		const int	iDim
	)		const

Get Number of N-axis bins for a given sample.

Parameters

iSample	index of a given sample
iDim	dimension for which we extract number of bins

Definition at line 441 of file BinningHandler.cpp.

                                                                         {
 // ************************************************
   const auto& Binning = SampleBinning[iSample];
   if(!Binning.Uniform) {
     MACH3LOG_ERROR("When using Non-Uniform binning for sample {} please use global bin instead of {}", iSample, __func__);
     throw MaCh3Exception(__FILE__, __LINE__);
   } else{
     return static_cast<int>(Binning.AxisNBins.at(iDim));
   }
 }

◆ GetNBins() [1/2]

int BinningHandler::GetNBins ( ) const

inline

Get total number of bins over all samples/kinematic bins etc.

Definition at line 161 of file BinningHandler.h.

161 {return TotalNumberOfBins;};

BinningHandler::TotalNumberOfBins

int TotalNumberOfBins

Total number of bins.

Definition: BinningHandler.h:207

◆ GetNBins() [2/2]

int BinningHandler::GetNBins ( const int iSample ) const

inline

Get total number of bins over for a given sample.

Definition at line 163 of file BinningHandler.h.

163 {return static_cast<int>(SampleBinning[iSample].nBins);};

◆ GetNDim()

int BinningHandler::GetNDim ( const int iSample ) const

inline

Get total number of dimensions for sample iSample.

Definition at line 165 of file BinningHandler.h.

165 { return int(SampleBinning[iSample].BinEdges.size()); }

◆ GetNonUniformBins()

const std::vector< BinInfo > BinningHandler::GetNonUniformBins ( const int iSample ) const

Return NonUnifomr bins to for example check extent etc.

Definition at line 460 of file BinningHandler.cpp.

                                                                                   {
 // ************************************************
   const auto& Binning = SampleBinning[iSample];
   if(!Binning.Uniform) {
     return Binning.Bins;
   } else{
     MACH3LOG_ERROR("{} for sample {} will not work becasue binnin is unfiorm", __func__, iSample);
     throw MaCh3Exception(__FILE__, __LINE__);
   }
 }

◆ GetSampleEndBin()

int BinningHandler::GetSampleEndBin ( const int iSample ) const

Get bin number corresponding to where given sample ends.

Parameters

iSample index of a given sample

Definition at line 338 of file BinningHandler.cpp.

                                                           {
 // ************************************************
   if (Sample == static_cast<int>(SampleBinning.size()) - 1) {
     return GetNBins();
   } else {
     return static_cast<int>(SampleBinning[Sample+1].GlobalOffset);
   }
 }

◆ GetSampleStartBin()

int BinningHandler::GetSampleStartBin ( const int iSample ) const

Get bin number corresponding to where given sample starts.

Parameters

iSample index of a given sample

Definition at line 332 of file BinningHandler.cpp.

                                                             {
 // ************************************************
  return static_cast<int>(SampleBinning[Sample].GlobalOffset);
 }

◆ IsUniform()

bool BinningHandler::IsUniform ( const int iSample ) const

Tells whether given sample is using unform binning.

Parameters

iSample index of a given sample

Definition at line 453 of file BinningHandler.cpp.

                                                       {
 // ************************************************
   const auto& Binning = SampleBinning[iSample];
   return Binning.Uniform;
 }

◆ SetGlobalBinNumbers()

void BinningHandler::SetGlobalBinNumbers ( )

Sets the GlobalOffset for each SampleBinningInfo to enable linearization of multiple 2D binning samples.

Definition at line 349 of file BinningHandler.cpp.

                                          {
 // ************************************************
   if (SampleBinning.empty()) {
     MACH3LOG_ERROR("No binning samples provided.");
     throw MaCh3Exception(__FILE__, __LINE__);
   }
  
   int GlobalOffsetCounter = 0;
   for(size_t iSample = 0; iSample < SampleBinning.size(); iSample++){
     SampleBinning[iSample].GlobalOffset = GlobalOffsetCounter;
     GlobalOffsetCounter += SampleBinning[iSample].nBins;
   }
   // lastly modify total number of bins
   TotalNumberOfBins = GlobalOffsetCounter;
 }

◆ SetupSampleBinning()

void BinningHandler::SetupSampleBinning	(	const YAML::Node &	Settings,
		SampleInfo &	SingleSample
	)

Function to setup the binning of your sample histograms and the underlying arrays that get handled in fillArray() and fillArray_MP().

Definition at line 197 of file BinningHandler.cpp.

                                                                                           {
 // ************************************************
   MACH3LOG_INFO("Setting up Sample Binning");
   //Binning
   SingleSample.VarStr = (Settings["VarStr"] && Settings["VarStr"].IsScalar())
                             ? std::vector<std::string>{Get<std::string>(
                                   Settings["VarStr"], __FILE__, __LINE__)}
                             : Get<std::vector<std::string>>(Settings["VarStr"],
                                                             __FILE__, __LINE__);
   SingleSample.nDimensions = static_cast<int>(SingleSample.VarStr.size());
  
   SampleBinningInfo SingleBinning;
   bool Uniform = Get<bool>(Settings["Uniform"], __FILE__ , __LINE__);
   bool found_range_specifier = false;
   if(Uniform == false) {
     if(Settings["Bins"].IsSequence()) {
         SingleBinning.InitNonUniform(Get<std::vector<std::vector<std::vector<double>>>>(Settings["Bins"], __FILE__, __LINE__));
     } else if(Settings["Bins"].IsMap()){
       // Load binning from external file
       auto file = Get<std::string>(Settings["Bins"]["File"], __FILE__, __LINE__);
       M3::AddPath(file);
       auto key = Get<std::string>(Settings["Bins"]["Key"], __FILE__, __LINE__);
       auto binfile = LoadYamlConfig(file, __FILE__, __LINE__);
       SingleBinning.InitNonUniform(Get<std::vector<std::vector<std::vector<double>>>>(binfile[key], __FILE__, __LINE__));
     } else {
       std::stringstream ss;
       ss << Settings["Bins"];
       MACH3LOG_ERROR(
           "When parsing binning, expected to find a YAML map or sequence, "
           "but found:\n{}",
           ss.str());
       throw MaCh3Exception(__FILE__, __LINE__);
     }
   } else {
     YAML::Node const & bin_edges_node = Settings["BinEdges"] ? Settings["BinEdges"] : Settings["VarBins"];
     if(!bin_edges_node){
       MACH3LOG_ERROR("When setting up Uniform sample binning, didn't find expected key: BinEdges (or VarBins for backward compatibility).");
       throw MaCh3Exception(__FILE__, __LINE__);
     }
     SingleBinning.InitUniform(UniformBinEdgeConfigParser(bin_edges_node, found_range_specifier));
   }
   if(SingleSample.VarStr.size() != SingleBinning.BinEdges.size()) {
     MACH3LOG_ERROR("Number of variables ({}) does not match number of bin edge sets ({}) in sample config '{}'",
                    SingleSample.VarStr.size(), SingleBinning.BinEdges.size(),SingleSample.SampleTitle);
     if(found_range_specifier){
       std::stringstream ss;
       ss << (Settings["BinEdges"] ? Settings["BinEdges"] : Settings["VarBins"]);
       MACH3LOG_ERROR(R"(A bin range specifier was found in node:
  
   {}
  
 Please carefully check the number of square brackets used, a 2D binning
   comprised of just 2 range specifiers must explicitly include the axis
   list specifier like:
  
   BinEdges: [ [ {{linspace: {{nb:10, low:0, up: 10}}}} ], [ {{linspace: {{nb:5, low:10, up: 100}}}} ] ]
 )", ss.str());
     }
     throw MaCh3Exception(__FILE__, __LINE__);
   }
  
   SampleBinning.emplace_back(SingleBinning);
  
   // now setup global numbering
   SetGlobalBinNumbers();
 }

Member Data Documentation

◆ SampleBinning

std::vector<SampleBinningInfo> BinningHandler::SampleBinning

private

Binning info for individual sample.

Definition at line 209 of file BinningHandler.h.

◆ TotalNumberOfBins

int BinningHandler::TotalNumberOfBins

private

Total number of bins.

Definition at line 207 of file BinningHandler.h.

The documentation for this class was generated from the following files:

Samples/BinningHandler.h
Samples/BinningHandler.cpp

Public Member Functions

Private Attributes

Detailed Description

Introduction

Uniform and Non-Uniform Binning Scheme

Bin Finding Algorithm

Uniform

Non-Uniform

Bin finding procedure

Constructor & Destructor Documentation

◆ BinningHandler()

◆ ~BinningHandler()

Member Function Documentation

◆ FindGlobalBin()

◆ FindNominalBin()

◆ GetBinEdges()

◆ GetBinName() [1/3]

◆ GetBinName() [2/3]

◆ GetBinName() [3/3]

◆ GetBinSafe()

◆ GetGlobalBinSafe()

◆ GetNAxisBins()

◆ GetNBins() [1/2]

◆ GetNBins() [2/2]

◆ GetNDim()

◆ GetNonUniformBins()

◆ GetSampleEndBin()

◆ GetSampleStartBin()

◆ IsUniform()

◆ SetGlobalBinNumbers()

◆ SetupSampleBinning()

Member Data Documentation

◆ SampleBinning

◆ TotalNumberOfBins