Loading BOA from JSON and Plotting Results

This notebook demonstrates how to:

Loading a Scheduler from JSON from a previous run (If you want to see the Experiment that this is from, see Running BOA Optimization Directly in Python. We will look at the output and plot some exploratory data analysis.

import pathlib
import os

import numpy as np
from ax.utils.notebook.plotting import init_notebook_plotting
from ax.plot.trace import optimization_trace_single_method_plotly
from ax.service.utils.report_utils import get_standard_plots, exp_to_df
import boa
from botorch.test_functions.synthetic import Cosine8

init_notebook_plotting()

[INFO 12-10 13:48:11] ax.utils.notebook.plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.

Loading the Scheduler

# Filepath to the scheduler.json
scheduler_fp = pathlib.Path(os.path.abspath("")) / "boa_runs/scheduler.json"

scheduler = boa.scheduler_from_json_file(scheduler_fp)
experiment = scheduler.experiment

[WARNING 12-10 13:48:12] Scheduler: Experiment(boa_runs) already has 100 trials associated with it. Total trials setting for this scheduler is 100, so no more trials would be run in this scheduler if `Scheduler.run_all_trials` is called (but you can still use `Scheduler.run_n_trials` to run a fixed number of trials).

Get the Best Trial and Output All Trials

best_trial, best_params, obj = scheduler.get_best_trial()
best_trial

99

exp_to_df(experiment)

	Cosine8	trial_index	arm_name	x0	x1	x2	x3	x4	x5	x6	x7	trial_status	generation_method
0	2.167124	0	0_0	0.774544	0.468805	0.068063	0.362316	0.879579	0.273648	0.832961	0.104289	COMPLETED	Sobol
1	1.332692	1	1_0	0.490980	0.122534	0.286072	0.135533	0.407212	0.516411	0.552975	0.209887	COMPLETED	Sobol
2	2.239779	2	2_0	0.370515	0.437868	0.613205	0.630494	0.135183	0.472351	0.552578	0.686980	COMPLETED	Sobol
3	3.417184	3	3_0	0.136836	0.695278	0.936232	0.844562	0.099137	0.971449	0.467957	0.413035	COMPLETED	Sobol
4	4.009739	4	4_0	0.747055	0.203236	0.984640	0.257893	0.542277	0.742640	0.682041	0.910413	COMPLETED	Sobol
...	...	...	...	...	...	...	...	...	...	...	...	...	...
95	3.157963	95	95_0	0.428170	0.875173	0.948110	0.439605	0.663419	0.438504	0.603967	0.475487	COMPLETED	GPEI
96	1.803210	96	96_0	0.516141	0.635045	0.359797	0.222931	0.587980	0.171713	0.471170	0.425680	COMPLETED	GPEI
97	2.426793	97	97_0	0.385700	0.414825	0.168669	0.768933	0.446303	0.664891	0.797453	0.790683	COMPLETED	GPEI
98	3.770818	98	98_0	0.568660	0.763975	0.816566	0.932060	0.571092	0.517328	0.833606	0.391031	COMPLETED	GPEI
99	3.341448	99	99_0	0.779255	0.245177	0.975580	0.867021	0.315499	0.468669	0.496713	0.597464	COMPLETED	GPEI

100 rows × 13 columns

EDA Plots

plots = get_standard_plots(experiment, scheduler.generation_strategy.model)
for plot in plots:
    plot.show()

optimum = -Cosine8().optimal_value
model_transitions = scheduler.generation_strategy.model_transitions
data = experiment.fetch_data()
metric_name = "Cosine8"

optimization_trace_single_method_plotly(
    y=np.array([data.df[data.df["metric_name"] == metric_name]["mean"]]),
    title=r"$\text{ Cosine Mixture: } f(x) = -0.1 \sum_{i=1}^n (cos(5 \pi x_i) - x_i^2) : x \in [-1, 1]^8$",
    ylabel="Cosine Mixture",
    model_transitions=model_transitions,
    # Try and use the metric's lower_is_better property, but fall back on
    # objective's minimize property if relevent
    optimum=optimum,
    optimization_direction=(
        ("minimize" if experiment.metrics[metric_name].lower_is_better is True else "maximize")
        if experiment.metrics[metric_name].lower_is_better is not None
        else ("minimize" if optimization_config.objective.minimize else "maximize")
    ),
    plot_trial_points=True,
)