Loading BOA from JSON and Plotting Results

This notebook demonstrates how to:

Loading a Scheduler from JSON from a previous run (If you want to see the Experiment that this is from, see :ref:Running BOA Optimization Directly in Python). We will look at the output and plot some exploratory data analysis.

import pathlib
import os

import numpy as np
from ax.utils.notebook.plotting import init_notebook_plotting
from ax.plot.trace import optimization_trace_single_method_plotly
from ax.service.utils.report_utils import get_standard_plots, exp_to_df
import boa
from botorch.test_functions.synthetic import Cosine8

init_notebook_plotting()

[INFO 11-30 12:47:43] ax.utils.notebook.plotting: Injecting Plotly library into cell. Do not overwrite or delete cell.

Loading the Scheduler

# Filepath to the scheduler.json
scheduler_fp = pathlib.Path(os.path.abspath("")) / "boa_runs/scheduler.json"

scheduler = boa.scheduler_from_json_file(scheduler_fp)
experiment = scheduler.experiment

[WARNING 11-30 12:47:44] Scheduler: Experiment(boa_runs) already has 100 trials associated with it. Total trials setting for this scheduler is 100, so no more trials would be run in this scheduler if `Scheduler.run_all_trials` is called (but you can still use `Scheduler.run_n_trials` to run a fixed number of trials).

Get the Best Trial and Output All Trials

best_trial, best_params, obj = scheduler.get_best_trial()
best_trial

99

exp_to_df(experiment)

	Cosine8	trial_index	arm_name	x0	x1	x2	x3	x4	x5	x6	x7	trial_status	generation_method
0	2.167124	0	0_0	0.774544	0.468805	0.068063	0.362316	0.879579	0.273648	0.832961	0.104289	COMPLETED	Sobol
1	1.332692	1	1_0	0.490980	0.122534	0.286072	0.135533	0.407212	0.516411	0.552975	0.209887	COMPLETED	Sobol
2	2.239779	2	2_0	0.370515	0.437868	0.613205	0.630494	0.135183	0.472351	0.552578	0.686980	COMPLETED	Sobol
3	3.417184	3	3_0	0.136836	0.695278	0.936232	0.844562	0.099137	0.971449	0.467957	0.413035	COMPLETED	Sobol
4	4.009739	4	4_0	0.747055	0.203236	0.984640	0.257893	0.542277	0.742640	0.682041	0.910413	COMPLETED	Sobol
...	...	...	...	...	...	...	...	...	...	...	...	...	...
95	3.157963	95	95_0	0.428170	0.875173	0.948110	0.439605	0.663419	0.438504	0.603967	0.475487	COMPLETED	GPEI
96	1.803210	96	96_0	0.516141	0.635045	0.359797	0.222931	0.587980	0.171713	0.471170	0.425680	COMPLETED	GPEI
97	2.426793	97	97_0	0.385700	0.414825	0.168669	0.768933	0.446303	0.664891	0.797453	0.790683	COMPLETED	GPEI
98	3.770818	98	98_0	0.568660	0.763975	0.816566	0.932060	0.571092	0.517328	0.833606	0.391031	COMPLETED	GPEI
99	3.341448	99	99_0	0.779255	0.245177	0.975580	0.867021	0.315499	0.468669	0.496713	0.597464	COMPLETED	GPEI

100 rows × 13 columns

EDA Plots

plots = get_standard_plots(experiment, scheduler.generation_strategy.model)
for plot in plots:
    plot.show()

optimum = -Cosine8().optimal_value
model_transitions = scheduler.generation_strategy.model_transitions
data = experiment.fetch_data()
metric_name = "Cosine8"

optimization_trace_single_method_plotly(
    y=np.array([data.df[data.df["metric_name"] == metric_name]["mean"]]),
    title=r"$\text{ Cosine Mixture: } f(x) = -0.1 \sum_{i=1}^n (cos(5 \pi x_i) - x_i^2) : x \in [-1, 1]^8$",
    ylabel="Cosine Mixture",
    model_transitions=model_transitions,
    # Try and use the metric's lower_is_better property, but fall back on
    # objective's minimize property if relevent
    optimum=optimum,
    optimization_direction=(
        ("minimize" if experiment.metrics[metric_name].lower_is_better is True else "maximize")
        if experiment.metrics[metric_name].lower_is_better is not None
        else ("minimize" if optimization_config.objective.minimize else "maximize")
    ),
    plot_trial_points=True,
)