.. _library_gradient_boosting_regression:

``gradient_boosting_regression``
================================

Gradient boosting regression supporting continuous and mixed-feature
datasets. Builds an additive ensemble of regression trees by repeatedly
fitting the current residuals under squared-error loss. Starts from the
arithmetic mean of the training targets and then adds a scaled tree
prediction at each boosting stage.

The library implements the ``regressor_protocol`` defined in the
``regression_protocols`` library and learns an additive ensemble of
regression trees by repeatedly fitting the current residuals under
squared-error loss.

API documentation
-----------------

Open the
`../../apis/library_index.html#gradient_boosting_regression <../../apis/library_index.html#gradient_boosting_regression>`__
link in a web browser.

Loading
-------

To load this library, load the ``loader.lgt`` file:

::

   | ?- logtalk_load(gradient_boosting_regression(loader)).

Testing
-------

To test this library predicates, load the ``tester.lgt`` file:

::

   | ?- logtalk_load(gradient_boosting_regression(tester)).

To run the performance benchmark suite, load the
``tester_performance.lgt`` file:

::

   | ?- logtalk_load(gradient_boosting_regression(tester_performance)).

Features
--------

- **Residual Fitting**: Fits each regression tree to the residuals of
  the current additive model under squared-error loss.
- **Shrinkage**: Supports a fixed learning rate to scale stage
  contributions and reduce overfitting.
- **Continuous and Mixed Features**: Supports continuous attributes and
  categorical attributes encoded by the underlying regression-tree
  learner.
- **Tree Configuration**: Exposes the underlying regression-tree depth,
  minimum-leaf, variance-reduction, and scaling options.
- **Diagnostics Metadata**: Learned regressors record model name,
  target, training example count, initial prediction, fitted stage
  count, and effective options, accessible using the shared regression
  diagnostics predicates.
- **Model Export**: Learned regressors can be exported as predicate
  clauses or written to a file.
- **Reference Benchmarks**: Includes a dedicated performance suite
  reporting training time, RMSE, and MAE for representative regression
  datasets.

Regressor representation
------------------------

The learned regressor is represented by default as:

- ``gradient_boosting_regressor(InitialPrediction, WeightedTrees, Diagnostics)``

The exported predicate clauses therefore use the shape:

- ``Functor(InitialPrediction, WeightedTrees, Diagnostics)``

In this representation, ``WeightedTrees`` contains
``weighted_tree(LearningRate, Tree)`` terms and ``Diagnostics`` stores
training metadata including the effective options.

Diagnostics syntax
------------------

The ``diagnostics/2`` predicate returns a list of metadata terms with
the form:

::

   [
       model(gradient_boosting_regression),
       target(Target),
       training_example_count(TrainingExampleCount),
       options(Options),
       initial_prediction(InitialPrediction),
       stage_count(StageCount)
   ]

Where:

- ``model(gradient_boosting_regression)`` identifies the learning
  algorithm that produced the regressor.
- ``target(Target)`` stores the target attribute name declared by the
  training dataset.
- ``training_example_count(TrainingExampleCount)`` stores the number of
  examples used during training.
- ``options(Options)`` stores the effective learning options after
  merging the user options with the library defaults.
- ``initial_prediction(InitialPrediction)`` stores the constant
  prediction used to initialize the additive model before fitting any
  trees.
- ``stage_count(StageCount)`` stores the number of boosting stages that
  were actually fitted.

Use the ``regression_protocols`` ``diagnostic/2`` and
``regressor_options/2`` helper predicates when you only need a single
metadata term or the effective options.

Options
-------

The ``learn/3`` predicate accepts the following options:

- ``number_of_estimators/1``: Maximum number of boosting stages to fit.
  Each stage adds one regression tree to the ensemble. The default is
  ``50``. Training can stop before reaching this limit when the residual
  sum of squares becomes negligible.
- ``learning_rate/1``: Shrinkage factor applied to each stage prediction
  before it is added to the current model. Smaller values usually
  require more stages but can improve generalization. The default is
  ``0.1``.
- ``maximum_depth/1``: Maximum depth allowed for each regression tree
  used as a base learner. Lower values produce weaker, simpler trees;
  higher values allow each stage to model more complex residual
  structure. The default is ``3``.
- ``minimum_samples_leaf/1``: Minimum number of training examples
  required in each leaf of a base learner tree. Increasing this value
  makes the fitted trees more conservative and can reduce overfitting.
  The default is ``1``.
- ``minimum_variance_reduction/1``: Minimum reduction in target variance
  required to accept a split when fitting a base learner tree. Larger
  values make tree growth stricter by rejecting weak splits. The default
  is ``0.0``.
- ``feature_scaling/1``: Controls continuous-feature scaling in the
  underlying regression-tree learner. The accepted values are ``true``
  and ``false``. The default is ``false``.
