BinningHandler Class Reference

KS: Class handling binning for multiple samples. More...

#include <Samples/BinningHandler.h>

Collaboration diagram for BinningHandler:

Public Member Functions
	BinningHandler ()
	Constructor. More...
virtual	~BinningHandler ()
	destructor More...
int	FindGlobalBin (const int iSample, const std::vector< const double * > &KinVar, const std::vector< int > &NomBin) const
	Find Global bin including. More...
int	FindNominalBin (const int iSample, const int iDim, const double Var) const
	Find the nominal bin for a given variable in a given sample and dimension. More...
int	GetGlobalBinSafe (const int iSample, const std::vector< int > &Bins) const
	Get gloabl bin based on sample, and dimension of each sample with additional checks. More...
int	GetBinSafe (const int iSample, const std::vector< int > &Bins) const
	Get gloabl bin based on sample, and dimension of each sample without any safety checks. More...
int	GetNBins () const
	Get total number of bins over all samples/kinematic bins etc. More...
int	GetNBins (const int iSample) const
	Get total number of bins over for a given sample. More...
int	GetNDim (const int iSample) const
	Get total number of dimensions for sample iSample. More...
std::string	GetBinName (const int GlobalBin) const
	Get fancy name for a given bin, to help match it with global properties. More...
std::string	GetBinName (const int iSample, const int SampleBin) const
	Get fancy name for a given bin, to help match it with global properties. More...
std::string	GetBinName (const int iSample, const std::vector< int > &Bins) const
	Get fancy name for a given bin, to help match it with global properties. More...
int	GetSampleIndex (const int GlobalBin) const
	Returns sample index based on global bin. More...
std::vector< double >	GetBinEdges (const int iSample, const int iDim) const
	Get N-dim bin edges for a given sample. More...
int	GetNAxisBins (const int iSample, const int iDim) const
	Get Number of N-axis bins for a given sample. More...
bool	IsUniform (const int iSample) const
	Tells whether given sample is using unform binning. More...
int	GetSampleStartBin (const int iSample) const
	Get bin number corresponding to where given sample starts. More...
int	GetSampleEndBin (const int iSample) const
	Get bin number corresponding to where given sample ends. More...
const std::vector< BinInfo >	GetNonUniformBins (const int iSample) const
	Return NonUnifomr bins to for example check extent etc. More...
void	SetGlobalBinNumbers ()
	Sets the GlobalOffset for each SampleBinningInfo to enable linearization of multiple 2D binning samples. More...
void	SetupSampleBinning (const YAML::Node &Settings, SampleInfo &SingleSample)
	Function to setup the binning of your sample histograms and the underlying arrays that get handled in fillArray() and fillArray_MP(). More...

Private Attributes
int	TotalNumberOfBins
	Total number of bins. More...
std::vector< SampleBinningInfo >	SampleBinning
	Binning info for individual sample. More...

Detailed Description

KS: Class handling binning for multiple samples.

Introduction

Each sample can define its own binning in an arbitrary number of dimensions. Internally, every sample's multi-dimensional binning is linearised into a single 1D array. All samples are then concatenated into one global bin index space, allowing the entire analysis to be treated as a single large vector of bins.

The concept of a "global bin" refers to the position of a bin in this linearised, analysis-wide bin index space. Local (sample) bins are always enumerated starting from zero, while global bins span all samples consecutively.

Example layout of global bins with offsets:

Sample 0 (GlobalOffset = 0,  nBins = 4):
  Local bins:   [0] [1] [2] [3]
  Global bins:  [0] [1] [2] [3]
 
Sample 1 (GlobalOffset = 4,  nBins = 3):
  Local bins:   [0] [1] [2]
  Global bins:  [4] [5] [6]
 
Sample 2 (GlobalOffset = 7,  nBins = 2):
  Local bins:   [0] [1]
  Global bins:  [7] [8]
 
Global bin index space:
------------------------------------------------
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 |
------------------------------------------------

Uniform and Non-Uniform Binning Scheme

MaCh3 supports Uniform and Non-Uniform binning scheme

Uniform binning uses a regular Cartesian grid. Bins are constructed as the Cartesian product of 1D bin edges, forming axis-aligned rectangles (or hyper-rectangles in higher dimensions). Bin widths may be equal or variable, but the grid structure is regular.

In the non-uniform scheme, bin sizes may vary and bins may span different ranges along each dimension. However, all bins must remain axis-aligned hyper-rectangles. Arbitrary or irregular bin shapes (e.g. curved or "banana-shaped" bins) are not supported.

Example of Uniform

+--------+--------+--------+--------+
| Bin 0  | Bin 1  | Bin 2  | Bin 3  |
| (0,0)  | (1,0)  | (2,0)  | (3,0)  |
+--------+--------+--------+--------+
| Bin 4  | Bin 5  | Bin 6  | Bin 7  |
| (0,1)  | (1,1)  | (2,1)  | (3,1)  |
+--------+--------+--------+--------+
| Bin 8  | Bin 9  | Bin 10 | Bin 11 |
| (0,2)  | (1,2)  | (2,2)  | (3,2)  |
+--------+--------+--------+--------+

Example of Non-Uniform

+--------+------------+-------+---------------------+
| Bin 0  |    Bin 1   | Bin 2 |         Bin 3       |
|        |            |       |                     |
+--------+------------+-------+---------------------+
| Bin 4  |            |       |                     |
|        |            | Bin 6 |                     |
+--------+   Bin 5    +-------+                     |
| Bin 7  |            |       |         Bin 9       |
|        |            | Bin 8 |                     |
+--------+------------+-------+---------------------+

Bin Finding Algorithm

Since MaCh3 supports event migration bin finding algorithm must be fast to efficiently be able find bin during running fit. MaCh3 is caching nominal bin with idea that during fit migration should be around this nominal bin. Thus MaCh3 first checks if after shift event falls into Nom-bin and later adjacent. If not backs to binary search.

Uniform

In case of uniform binning above algorithm is easy to test as one performs it for every dimension independently i.e. find X-bin, then Y etc. After which can find bin in flattened 1D space.

Non-Uniform

Internally, non-uniform binning is implemented using two levels:

1. MegaBins (mapping bins) These form a coarse, uniform grid that spans the full phase space. Each MegaBin acts as a container for one or more non-uniform bins.
2. Non-Uniform bins The actual analysis bins, defined as hyper-rectangles with arbitrary extents inside a MegaBin.

Bin finding procedure

For a given event, the bin-finding algorithm proceeds as follows:

1. Locate the MegaBin using the same fast per-dimension logic as in uniform binning.
2. Once the MegaBin is identified, loop over all non-uniform bins associated with that MegaBin.
3. The first bin whose extents fully contain the event is selected.
4. If no bin matches, the event is assigned to the under/overflow bin.

Author

Kamil Skwarczynski

Dan Barrow

Luke Pickering

Definition at line 130 of file BinningHandler.h.

Constructor & Destructor Documentation

BinningHandler()

BinningHandler::BinningHandler

(

)

Constructor.

Definition at line 4 of file BinningHandler.cpp.

                               {
// ************************************************
}

~BinningHandler()

virtual BinningHandler::~BinningHandler ( )

inlinevirtual

destructor

Definition at line 137 of file BinningHandler.h.

{};

Member Function Documentation

FindGlobalBin()

int BinningHandler::FindGlobalBin	(	const int	iSample,
		const std::vector< const double * > &	KinVar,
		const std::vector< int > &	NomBin
	)		const

Find Global bin including.

Parameters

iSample	index of a given sample
KinVar	Vector of pointers to kinematic variable like Erec
NomBin	Vector of nominal bin indices for this event, one per dimension.

Definition at line 148 of file BinningHandler.cpp.

                                                                      {
// ************************************************
  //DB Find the relevant bin in the PDF for each event
  const int Dim = static_cast<int>(KinVar.size());
  const SampleBinningInfo& _restrict_ Binning = SampleBinning[NomSample];
  int GlobalBin = 0;
 
  for(int i = 0; i < Dim; ++i) {
    const double Var = *KinVar[i];
    const int Bin = Binning.FindBin(i, Var, NomBin[i]);
    // KS: If we are outside of range in only one dimension this mean out of bounds, we can simply quickly finish
    if(Bin < 0) return M3::UnderOverFlowBin;
    // KS: inline GetBin computation to avoid any memory allocation, which in reweight loop is very costly
    GlobalBin += Bin * Binning.Strides[i];
  }
 
  if(Binning.Uniform) {
    GlobalBin += static_cast<int>(Binning.GlobalOffset);
    return GlobalBin;
  } else {
    const auto& _restrict_ BinMapping = Binning.BinGridMapping[GlobalBin];
    const size_t nNonUniBins = BinMapping.size();
    for(size_t iBin = 0; iBin < nNonUniBins; iBin++) {
      const int BinNumber = BinMapping[iBin];
      const auto& _restrict_ NonUniBin = Binning.Bins[BinNumber];
      if(NonUniBin.IsEventInside(KinVar)){
        return BinNumber + Binning.GlobalOffset;
      }
    }
    MACH3LOG_DEBUG("Didn't find any bin so returning UnderOverFlowBin");
    return M3::UnderOverFlowBin;
  }
}

FindNominalBin()

int BinningHandler::FindNominalBin	(	const int	iSample,
		const int	iDim,
		const double	Var
	)		const

Find the nominal bin for a given variable in a given sample and dimension.

Parameters

iSample	Sample index
iDim	Dimension index (0 = X, 1 = Y, ...)
Var	Kinematic variable value

Definition at line 185 of file BinningHandler.cpp.

                                                           {
// ************************************************
  const SampleBinningInfo& info = SampleBinning[iSample];
 
  const auto& edges = info.BinEdges[iDim];
 
  // Outside binning range
  if (Var < edges.front() || Var >= edges.back()) {
    return M3::UnderOverFlowBin;
  }
  return static_cast<int>(std::distance(edges.begin(), std::upper_bound(edges.begin(), edges.end(), Var)) - 1);
}

GetBinEdges()

std::vector<double> BinningHandler::GetBinEdges ( const int  iSample, const int  iDim  ) const

inline

Get N-dim bin edges for a given sample.

Parameters

iSample	index of a given sample
iDim	dimension for which we extract bin edges

Definition at line 184 of file BinningHandler.h.

{return SampleBinning[iSample].BinEdges.at(iDim);};

GetBinName() [1/3]

std::string BinningHandler::GetBinName

(

const int

GlobalBin

)

const

Get fancy name for a given bin, to help match it with global properties.

Parameters

GlobalBin

Global Bin integrated over all samples

Definition at line 316 of file BinningHandler.cpp.

                                                              {
// ************************************************
  int SampleBin = GetSampleIndex(GlobalBin);
  int LocalBin  = GetLocalBinFromGlobalBin(SampleBinning, GlobalBin);
  return GetBinName(SampleBin, LocalBin);
}

GetBinName() [2/3]

std::string BinningHandler::GetBinName	(	const int	iSample,
		const int	SampleBin
	)		const

Get fancy name for a given bin, to help match it with global properties.

Parameters

iSample	index of a given sample
SampleBin	Global Bin for a given sample

Definition at line 262 of file BinningHandler.cpp.

                                                                                 {
// ************************************************
  const auto& Binning = SampleBinning[iSample];
 
  // Safety checks
  if (SampleBin < 0 || SampleBin >= static_cast<int>(Binning.nBins)) {
    MACH3LOG_ERROR("Requested bin {} is out of range for sample {}", SampleBin, iSample);
    throw MaCh3Exception(__FILE__, __LINE__);
  }
  std::string BinName;
 
  if(Binning.Uniform) {
    int Dim = static_cast<int>(Binning.Strides.size());
    std::vector<int> Bins(Dim, 0);
    int Remaining = SampleBin;
 
    // Convert the flat/global bin index into per-dimension indices
    // Dim0 is the fastest-changing axis, Dim1 the next, etc.
    //
    // For example (2D):
    //   x = bin % Nx
    //   y = bin / Nx
    //
    // For 3D:
    //   x = bin % Nx
    //   y = (bin / Nx) % Ny
    //   z = bin / (Nx * Ny)
    for (int i = 0; i < Dim; ++i) {
      const int nBinsDim = static_cast<int>(Binning.BinEdges[i].size()) - 1;
      Bins[i] = Remaining % nBinsDim;
      Remaining /= nBinsDim;
    }
 
    for (int i = 0; i < Dim; ++i) {
      if (i > 0) BinName += ", ";
      const double min = Binning.BinEdges[i].at(Bins[i]);
      const double max = Binning.BinEdges[i].at(Bins[i] + 1);
      BinName += fmt::format("Dim{} ({:g}, {:g})", i, min, max);
    }
  } else{
    const BinInfo& bin = Binning.Bins[SampleBin];
    const int Dim = static_cast<int>(bin.Extent.size());
 
    for (int i = 0; i < Dim; ++i) {
      if (i > 0) BinName += ", ";
      const double min = bin.Extent[i][0];
      const double max = bin.Extent[i][1];
      BinName += fmt::format("Dim{} ({:g}, {:g})", i, min, max);
    }
  }
  return BinName;
}

GetBinName() [3/3]

std::string BinningHandler::GetBinName	(	const int	iSample,
		const std::vector< int > &	Bins
	)		const

Get fancy name for a given bin, to help match it with global properties.

Parameters

iSample	index of a given sample
Bins	Vector of bin indices along each dimension

Definition at line 250 of file BinningHandler.cpp.

                                                                                        {
// ************************************************
  const auto& Binning = SampleBinning[iSample];
  if(!Binning.Uniform) {
    MACH3LOG_ERROR("When using Non-Uniform binning for sample {} please use One bin instead of Axis bins", iSample);
    throw MaCh3Exception(__FILE__, __LINE__);
  }
  return GetBinName(iSample, GetBinSafe(iSample, Bins));
}

GetBinSafe()

int BinningHandler::GetBinSafe	(	const int	iSample,
		const std::vector< int > &	Bins
	)		const

Get gloabl bin based on sample, and dimension of each sample without any safety checks.

Parameters

iSample	index of a given sample
Bins	Vector of bin indices along each dimension

Definition at line 201 of file BinningHandler.cpp.

                                                                                 {
// ************************************************
  const int GlobalBin = SampleBinning[Sample].GetBinSafe(Bins);
  return GlobalBin;
}

GetGlobalBinSafe()

int BinningHandler::GetGlobalBinSafe	(	const int	iSample,
		const std::vector< int > &	Bins
	)		const

Get gloabl bin based on sample, and dimension of each sample with additional checks.

Parameters

iSample	index of a given sample
Bins	Vector of bin indices along each dimension

Definition at line 208 of file BinningHandler.cpp.

                                                                                       {
// ************************************************
  const int GlobalBin = SampleBinning[Sample].GetBinSafe(Bins) + static_cast<int>(SampleBinning[Sample].GlobalOffset);
  return GlobalBin;
}

GetNAxisBins()

int BinningHandler::GetNAxisBins	(	const int	iSample,
		const int	iDim
	)		const

Get Number of N-axis bins for a given sample.

Parameters

iSample	index of a given sample
iDim	dimension for which we extract number of bins

Definition at line 331 of file BinningHandler.cpp.

                                                                        {
// ************************************************
  const auto& Binning = SampleBinning[iSample];
  if(!Binning.Uniform) {
    MACH3LOG_ERROR("When using Non-Uniform binning for sample {} please use global bin instead of {}", iSample, __func__);
    throw MaCh3Exception(__FILE__, __LINE__);
  } else{
    return static_cast<int>(Binning.AxisNBins.at(iDim));
  }
}

GetNBins() [1/2]

int BinningHandler::GetNBins ( ) const

inline

Get total number of bins over all samples/kinematic bins etc.

Definition at line 162 of file BinningHandler.h.

{return TotalNumberOfBins;};

GetNBins() [2/2]

int BinningHandler::GetNBins ( const int  iSample ) const

inline

Get total number of bins over for a given sample.

Definition at line 164 of file BinningHandler.h.

{return static_cast<int>(SampleBinning[iSample].nBins);};

GetNDim()

int BinningHandler::GetNDim ( const int  iSample ) const

inline

Get total number of dimensions for sample iSample.

Definition at line 166 of file BinningHandler.h.

{ return int(SampleBinning[iSample].BinEdges.size()); }

GetNonUniformBins()

const std::vector< BinInfo > BinningHandler::GetNonUniformBins

(

const int

iSample

)

const

Return NonUnifomr bins to for example check extent etc.

Definition at line 350 of file BinningHandler.cpp.

                                                                                  {
// ************************************************
  const auto& Binning = SampleBinning[iSample];
  if(!Binning.Uniform) {
    return Binning.Bins;
  } else{
    MACH3LOG_ERROR("{} for sample {} will not work becasue binnin is unfiorm", __func__, iSample);
    throw MaCh3Exception(__FILE__, __LINE__);
  }
}

GetSampleEndBin()

int BinningHandler::GetSampleEndBin

(

const int

iSample

)

const

Get bin number corresponding to where given sample ends.

Parameters

iSample

index of a given sample

Definition at line 221 of file BinningHandler.cpp.

                                                          {
// ************************************************
  if (Sample == static_cast<int>(SampleBinning.size()) - 1) {
    return GetNBins();
  } else {
    return static_cast<int>(SampleBinning[Sample+1].GlobalOffset);
  }
}

GetSampleIndex()

int BinningHandler::GetSampleIndex

(

const int

GlobalBin

)

const

Returns sample index based on global bin.

Parameters

GlobalBin

bin for which we will return sample index

Definition at line 324 of file BinningHandler.cpp.

                                                            {
// ************************************************
  int SampleBin = GetSampleFromGlobalBin(SampleBinning, GlobalBin);
  return SampleBin;
}

GetSampleStartBin()

int BinningHandler::GetSampleStartBin

(

const int

iSample

)

const

Get bin number corresponding to where given sample starts.

Parameters

iSample

index of a given sample

Definition at line 215 of file BinningHandler.cpp.

                                                            {
// ************************************************
  return static_cast<int>(SampleBinning[Sample].GlobalOffset);
}

IsUniform()

bool BinningHandler::IsUniform

(

const int

iSample

)

const

Tells whether given sample is using unform binning.

Parameters

iSample

index of a given sample

Definition at line 343 of file BinningHandler.cpp.

                                                      {
// ************************************************
  const auto& Binning = SampleBinning[iSample];
  return Binning.Uniform;
}

SetGlobalBinNumbers()

void BinningHandler::SetGlobalBinNumbers

(

)

Sets the GlobalOffset for each SampleBinningInfo to enable linearization of multiple 2D binning samples.

Definition at line 232 of file BinningHandler.cpp.

                                         {
// ************************************************
  if (SampleBinning.empty()) {
    MACH3LOG_ERROR("No binning samples provided.");
    throw MaCh3Exception(__FILE__, __LINE__);
  }
 
  int GlobalOffsetCounter = 0;
  for(size_t iSample = 0; iSample < SampleBinning.size(); iSample++){
    SampleBinning[iSample].GlobalOffset = GlobalOffsetCounter;
    GlobalOffsetCounter += SampleBinning[iSample].nBins;
  }
  // lastly modify total number of bins
  TotalNumberOfBins = GlobalOffsetCounter;
}

SetupSampleBinning()

void BinningHandler::SetupSampleBinning	(	const YAML::Node &	Settings,
		SampleInfo &	SingleSample
	)

Function to setup the binning of your sample histograms and the underlying arrays that get handled in fillArray() and fillArray_MP().

Definition at line 80 of file BinningHandler.cpp.

                                                                                          {
// ************************************************
  MACH3LOG_INFO("Setting up Sample Binning");
  //Binning
  SingleSample.VarStr = (Settings["VarStr"] && Settings["VarStr"].IsScalar())
    ? std::vector<std::string>{Get<std::string>(
      Settings["VarStr"], __FILE__, __LINE__)}
    : Get<std::vector<std::string>>(Settings["VarStr"],
                                    __FILE__, __LINE__);
  SingleSample.nDimensions = static_cast<int>(SingleSample.VarStr.size());
 
  SampleBinningInfo SingleBinning;
  bool Uniform = Get<bool>(Settings["Uniform"], __FILE__ , __LINE__);
  bool found_range_specifier = false;
  if(Uniform == false) {
    if(Settings["Bins"].IsSequence()) {
      SingleBinning.InitNonUniform(Get<std::vector<std::vector<std::vector<double>>>>(Settings["Bins"], __FILE__, __LINE__));
    } else if(Settings["Bins"].IsMap()){
      // Load binning from external file
      auto file = Get<std::string>(Settings["Bins"]["File"], __FILE__, __LINE__);
      M3::AddPath(file);
      auto key = Get<std::string>(Settings["Bins"]["Key"], __FILE__, __LINE__);
      auto binfile = LoadYamlConfig(file, __FILE__, __LINE__);
      SingleBinning.InitNonUniform(Get<std::vector<std::vector<std::vector<double>>>>(binfile[key], __FILE__, __LINE__));
    } else {
      std::stringstream ss;
      ss << Settings["Bins"];
      MACH3LOG_ERROR(
        "When parsing binning, expected to find a YAML map or sequence, "
        "but found:\n{}",
        ss.str());
      throw MaCh3Exception(__FILE__, __LINE__);
    }
  } else {
    YAML::Node const & bin_edges_node = Settings["BinEdges"] ? Settings["BinEdges"] : Settings["VarBins"];
    if(!bin_edges_node){
      MACH3LOG_ERROR("When setting up Uniform sample binning, didn't find expected key: BinEdges (or VarBins for backward compatibility).");
      throw MaCh3Exception(__FILE__, __LINE__);
    }
    SingleBinning.InitUniform(UniformBinEdgeConfigParser(bin_edges_node, found_range_specifier));
  }
  if(SingleSample.VarStr.size() != SingleBinning.BinEdges.size()) {
    MACH3LOG_ERROR("Number of variables ({}) does not match number of bin edge sets ({}) in sample config '{}'",
                   SingleSample.VarStr.size(), SingleBinning.BinEdges.size(),SingleSample.SampleTitle);
    if(found_range_specifier){
      std::stringstream ss;
      ss << (Settings["BinEdges"] ? Settings["BinEdges"] : Settings["VarBins"]);
      MACH3LOG_ERROR(R"(A bin range specifier was found in node:
 
                     {}
 
                     Please carefully check the number of square brackets used, a 2D binning
                     comprised of just 2 range specifiers must explicitly include the axis
                     list specifier like:
 
                     BinEdges: [ [ {{linspace: {{nb:10, low:0, up: 10}}}} ], [ {{linspace: {{nb:5, low:10, up: 100}}}} ] ]
                     )", ss.str());
    }
    throw MaCh3Exception(__FILE__, __LINE__);
  }
 
  SampleBinning.emplace_back(SingleBinning);
 
  // now setup global numbering
  SetGlobalBinNumbers();
}

Member Data Documentation

SampleBinning

std::vector<SampleBinningInfo> BinningHandler::SampleBinning

private

Binning info for individual sample.

Definition at line 212 of file BinningHandler.h.

TotalNumberOfBins

int BinningHandler::TotalNumberOfBins

private

Total number of bins.

Definition at line 210 of file BinningHandler.h.

---

The documentation for this class was generated from the following files:

• Samples/BinningHandler.h
• Samples/BinningHandler.cpp