List of Metrics#

Metrics that are already defined in BOA:

MeanSquaredError
RootMeanSquaredError
RSquared
Mean
NormalizedRootMeanSquaredError

Any of these Metrics can be used directly in your configuration file.

Examples

# Single objective optimization config
optimization_options:
    objective_options:
        objectives:
            # List all of your metrics here,
            # only list 1 metric for a single objective optimization
            - metric: RootMeanSquaredError

# MultiObjective Optimization config
optimization_options:
    objective_options:
        objectives:
            # List all of your metrics here,
            # only list multiple objectives for a multi objective optimization
            - metric: RMSE
            - metric: R2

If your metric is computed elsewhere (say by your model code or your model wrapping code), then you can use as a value passthrough the metric Mean. BOA will be adding a dedicated passthrough Metric in the near future, but for the moment you can use Mean and in your Wapper.fetch_trial_data you can do something like this

def fetch_trial_data(self, trial, *args, **kwargs):
    value = get_value_somehow()
    return dict(a=[value])

Mean expects to get called with a parameter a being an array or list of values (notice that it isn’t dict(a=value) it is dict(a=[value]) with [value] making it a mean of just value, so it acts as a passthrough, but you have to return it as a dictionary with of a => list[value]

Overview Information Here: boa.metrics

class boa.metrics.metrics.SklearnMetric(metric_to_eval: Optional[str] = None, *args, **kwargs)[source]#

Bases: ModularMetric

A subclass of ModularMetric where you can pass in a string name of a metric from sklrean.metrics, and BOA will grab that metric and create a BOA metric class for you.

See also

sklearn.metrics: for the list of metrics you can use.
ModularMetric: For information on all parameters various metrics in general can be supplied

Parameters:: metric_to_eval (str) –

class boa.metrics.metrics.MeanSquaredError(lower_is_better=True, *args, **kwargs)[source]#

Bases: SklearnMetric

Mean squared error regression loss.

See also

sklearn.metrics.mean_squared_error(): for the function parameters to guide your json attribute-value pairs needed.
ModularMetric: For information on all parameters various metrics in general can be supplied

boa.metrics.metrics.MSE#: alias of MeanSquaredError

boa.metrics.metrics.mean_squared_error#: alias of MeanSquaredError

class boa.metrics.metrics.RootMeanSquaredError(lower_is_better=True, metric_func_kwargs=(('squared', False),), *args, **kwargs)[source]#

Bases: SklearnMetric

Root mean squared error regression loss.

See also

sklearn.metrics.mean_squared_error(): with squared=False for the function parameters to guide your json attribute-value pairs needed.
ModularMetric: For information on all parameters various metrics in general can be supplied

boa.metrics.metrics.RMSE#: alias of RootMeanSquaredError

boa.metrics.metrics.root_mean_squared_error#: alias of RootMeanSquaredError

class boa.metrics.metrics.RSquared(lower_is_better=True, *args, **kwargs)[source]#

Bases: SklearnMetric

\(R^2\) (coefficient of determination) regression score function.

Best possible score is 1.0, and it can be negative (because the model can be arbitrarily worse). In the general case when the true y is non-constant, a constant model that always predicts the average y disregarding the input features would get a \(R^2\) score of 0.0.

See also

sklearn.metrics.r2_score(): for the function parameters to guide your json attribute-value pairs needed.
ModularMetric: For information on all parameters various metrics in general can be supplied

boa.metrics.metrics.r2_score#: alias of RSquared

boa.metrics.metrics.R2#: alias of RSquared

class boa.metrics.metrics.Mean(lower_is_better=True, *args, **kwargs)[source]#

Bases: ModularMetric

Arithmetic mean along the specified axis for your metric, Defaults to minimization, if you want to maximize, specify lower_is_better: False or minimize: False in your configuration

See also

numpy.mean(): for the function parameters to guide your json attribute-value pairs needed.
ModularMetric: For information on all parameters various metrics in general can be supplied

boa.metrics.metrics.mean#: alias of Mean

class boa.metrics.metrics.NormalizedRootMeanSquaredError(lower_is_better=True, *args, **kwargs)[source]#

Bases: ModularMetric

Normalized root mean squared error. Like a normalized version of RMSE. Normalization defaults to IQR (inner quartile range).

normalized_root_mean_squared_error

See also

normalized_root_mean_squared_error(): for the function parameters to guide your json attribute-value pairs needed.
ModularMetric: For information on all parameters various metrics in general can be supplied

boa.metrics.metrics.NRMSE#: alias of NormalizedRootMeanSquaredError

boa.metrics.metrics.normalized_root_mean_squared_error#: alias of NormalizedRootMeanSquaredError

boa.metrics.metrics.get_metric_from_config(config, instantiate=True, **kwargs)[source]#

boa.metrics.metrics.get_metric_by_class_name(metric_name, instantiate=True, sklearn_=False, **kwargs)[source]#

boa.metrics.metrics.get_boa_metric(name) → Type[ModularMetric][source]#

Return type:: Type[ModularMetric]