Optimizers¶
Every optimizer takes a search space, a config and optional starting points, and supports optimize(), ask() and tell(). See Choosing an optimizer for which to use.
choose_optimizer(search_space: SearchSpace | list[Parameter], max_evaluations: int, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None, **config: Any) -> Optimizer
The optimizer the benchmark suggests for this search space and budget, set up to use it.
- Up to 200 evaluations: Bayesian optimization, unless a choice parameter has choose_n.
- More, with choice parameters: the genetic optimizer, since CMA-ES searches numbers only.
- More, with integer parameters: the genetic optimizer, which did better on the benchmark's integer and mixed problems.
- More, with continuous numbers only: CMA-ES.
Other settings, such as seed, maximize, noise or threshold, are passed on to the chosen optimizer's config. The optimizer's class and config show what was chosen.
choose_pareto_optimizer(search_space: SearchSpace | list[Parameter], objectives: dict[str, Any], max_evaluations: int, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None, **config: Any) -> MultiObjectiveOptimizer
The multi-objective optimizer the benchmark suggests for this search space and budget, set up to use it.
- Up to 200 evaluations: multi-objective Bayesian optimization, unless a choice parameter has choose_n.
- More: NSGA2.
objectives names each objective and its direction, as in MultiObjectiveConfig. Other settings are passed on to the chosen optimizer's config.
BayesianOptimizer(search_space: SearchSpace | List[Parameter], config: BayesianConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: Optimizer, _ModelSearch
Bayesian optimization with a Gaussian process and expected improvement.
The first generation is spread over the search space (with any initial points). After that, each candidate is the point where a Gaussian process model of the objective predicts the largest expected improvement on the best value so far. Failed evaluations are modelled as the worst value seen, which steers the search away from them.
Range parameters, integer or not, and choice parameters are supported, but not choice parameters with choose_n. Fixed parameters are passed through.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
search_space
|
SearchSpace | List[Parameter]
|
The parameters to optimize over, as a SearchSpace or a list of parameters. |
required |
config
|
BayesianConfig | None
|
A BayesianConfig. Defaults are used if omitted. |
None
|
initial_points
|
dict[str, Any] | list[dict[str, Any]] | None
|
Points to evaluate first: one dict of parameter values, or a list of them. |
None
|
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
result() -> Result
The best result found so far, with a summary of every completed generation.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
CMAES(search_space: SearchSpace | List[Parameter], config: CMAESConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: Optimizer
Covariance matrix adaptation evolution strategy, for range parameters.
Each generation samples candidates from a multivariate normal distribution, then moves the distribution's centre toward the best of them and reshapes it to stretch along the directions that improved. It learns the scale of each variable and how variables interact, which makes it strong on smooth continuous problems.
Range parameters, continuous or integer, are searched; fixed parameters are passed through. Choice parameters are not supported: use GeneticOptimizer for those. A range a condition switches off keeps its place in the distribution; its value is ignored.
The search starts at the centre of every range. With initial points, it evaluates them first and starts at the best one instead; a smaller initial_step then keeps the search close to it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
search_space
|
SearchSpace | List[Parameter]
|
The parameters to optimize over, as a SearchSpace or a list of parameters. |
required |
config
|
CMAESConfig | None
|
A CMAESConfig. Defaults are used if omitted. |
None
|
initial_points
|
dict[str, Any] | list[dict[str, Any]] | None
|
Points to evaluate first: one dict of parameter values, or a list of them. |
None
|
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
result() -> Result
The best result found so far, with a summary of every completed generation.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
GeneticOptimizer(search_space: SearchSpace | List[Parameter], config: OptimizerConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: Optimizer, Breeding
Function optimizer that employs an implementation of the genetic algorithm. The genetic algorithm is a metaheuristic inspired by the process of natural selection. It is a population-based algorithm that uses the concepts of selection, crossover, and mutation to evolve a population of individuals towards a solution to the objective function.
The first generation is any initial points, then random candidates. Each later generation keeps the previous one's elites unchanged, then fills the rest of the population with children: two parents are selected, crossed over, and mutated. Failed evaluations are never elites or parents.
With blend crossover, each range value of a child is drawn from between its parents' values, and a little beyond, rather than copied from one of them; choices and fixed values are still copied.
With adaptive mutation, each individual carries its own mutation size, as in evolution strategies: a child takes the geometric mean of its parents' sizes, varied at random, so individuals with sizes that suit the problem's current scale breed more.
Without noise, a child that repeats a point already asked for, such as a copy of a parent that no mutation touched, has one gene at a time mutated again until it is new, since evaluating it again would only spend the budget. With noise, repeats stay: they refine the point's average.
With conditional parameters, every individual keeps a value for each parameter, even one switched off at its point, so a child that switches a branch back on inherits settings that once did well there. A gene switched off in one parent but on in the other comes from the parent it is on in, and only genes switched on are mutated.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
search_space
|
SearchSpace | List[Parameter]
|
The parameters to optimize over, as a SearchSpace or a list of parameters. |
required |
config
|
OptimizerConfig | None
|
A GeneticOptimizerConfig. Defaults are used if omitted. |
None
|
Example:
>>> from hone import GeneticOptimizer, GeneticOptimizerConfig, RangeParameter
>>>
>>> def objective_function(x, y):
>>> return x**2 + y**2
>>>
>>> x = RangeParameter("x", -5, 5)
>>> y = RangeParameter("y", -5, 5)
>>>
>>> optimizer = GeneticOptimizer(search_space=[x, y], config=GeneticOptimizerConfig(threshold=0.01))
>>> result = optimizer.optimize(objective_function)
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
result() -> Result
The best result found so far, with a summary of every completed generation.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
RandomSearch(search_space: SearchSpace | List[Parameter], config: OptimizerConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: Optimizer
Evaluates population_size random points per generation and keeps the best.
Useful as a baseline: a smarter optimizer should beat it for the same number of evaluations. Any initial points take the place of random points in the first generation.
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
result() -> Result
The best result found so far, with a summary of every completed generation.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
MultiObjectiveBayesian(search_space: SearchSpace | List[Parameter], config: MultiObjectiveBayesianConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: MultiObjectiveOptimizer, _ModelSearch
Bayesian optimization for several objectives, returning the front of best trade-offs.
The first generation is spread over the search space, as in BayesianOptimizer. Each later candidate aims at a different part of the front: it draws random weights for the objectives, each scaled by the range seen so far, and goes where the model expects the most improvement in their augmented Chebyshev combination, the largest weighted gap from the best values plus a twentieth of their sum. Unlike a weighted sum, this reaches every part of the front, including any dent in it. With method "parego" one Gaussian process models the combination (ParEGO, Knowles 2006); with "models" one models each objective, and the combination is applied to their predictions.
Range and choice parameters are supported, as in BayesianOptimizer, with constraints and conditions. Each candidate takes a fraction of a second or more to choose, so it suits expensive objectives and budgets of a few hundred evaluations.
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
result() -> ParetoResult
The front of everything evaluated so far, with a summary of every completed generation.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
NSGA2(search_space: SearchSpace | List[Parameter], config: OptimizerConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: MultiObjectiveOptimizer, Breeding
The genetic optimizer for several objectives, returning the front of best trade-offs.
Each generation, the previous generation and its children are sorted into fronts: the points nothing dominates, then those only the first front dominates, and so on. The best fronts survive, and where one doesn't fit whole, its most isolated points do, by crowding distance, so the front stays spread out. Parents are chosen from the survivors by tournaments of two, won by the better front and then the more isolated point. Children are bred as in the genetic optimizer, with its crossover, mutation, repeat avoidance, repair for constraints, conditions and niches.
The result's front is that of every point evaluated, not only the survivors, so a trade-off found early is never lost when crowding drops it from the population. Ties are broken by the order points were created, so a seeded run repeats exactly, and its result doesn't depend on the number of workers.
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
result() -> ParetoResult
The front of everything evaluated so far, with a summary of every completed generation.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
MultiObjectiveRandomSearch(search_space: SearchSpace | List[Parameter], config: OptimizerConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: MultiObjectiveOptimizer
Evaluates population_size random points per generation and keeps the front of them all.
A baseline: a multi-objective optimizer should find a better front for the same number of evaluations. Any initial points take the place of random points in the first generation.
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
result() -> ParetoResult
The front of everything evaluated so far, with a summary of every completed generation.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
Base classes¶
Optimizer(search_space: SearchSpace | List[Parameter], config: OptimizerConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: OptimizerCore[Result]
Base class for population-based optimizers of a single objective.
Use it either way round:
- optimize(objective) calls the objective for every candidate and returns the Result.
- ask() and tell() let you evaluate candidates yourself, anywhere and in any order; result() returns the Result so far.
The base class runs the generation loop: it hands out each generation's candidates, records their outcomes, tracks the best so far, stops at the threshold, and builds the Result. A subclass only decides how each generation's population is produced, by implementing _next_population.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
search_space
|
SearchSpace | List[Parameter]
|
The parameters to optimize over, as a SearchSpace or a list of parameters. |
required |
config
|
OptimizerConfig | None
|
The optimizer's configuration. Defaults are used if omitted. |
None
|
initial_points
|
dict[str, Any] | list[dict[str, Any]] | None
|
Points to evaluate first, such as a known good design or the best parameters of an earlier run: one dict of parameter values, or a list of them. Fixed parameters may be left out, and so must parameters a condition switches off. |
None
|
Every individual an optimizer breeds holds a value for every parameter, including those the conditions switch off at its point, so an optimizer can keep them to use again. The base class leaves them out of what it hands out and reports: trials, results and progress.
The objective returns a number. To record an evaluation as failed instead, it can raise EvaluationFailed or return NaN or infinity.
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
MultiObjectiveOptimizer(search_space: SearchSpace | List[Parameter], config: OptimizerConfig | None = None, *, initial_points: dict[str, Any] | list[dict[str, Any]] | None = None)
Bases: OptimizerCore[ParetoResult]
Base class for optimizers of several objectives, which return a ParetoResult.
The objective function returns a dict with one value per objective named in the config's objectives, such as {"accuracy": 0.97, "cost": 12.0}, and tell() takes the same. A dict with missing or extra keys, or a value that isn't a number, raises TypeError: it is a bug in the objective. Raise EvaluationFailed, or return NaN or infinity for any objective, to record an evaluation as failed; failed evaluations are never on the front.
The front is that of every point evaluated, not only the latest generation, so a trade-off found early is never lost. With noise, each point's values are the averages of all its evaluations, and the front comes with their standard errors.
property
¶
done: bool
Whether the run has finished: every generation is complete, or the threshold was reached.
With min_finished below 1, the run isn't done until every late result has been told.
ask(n: int | None = None) -> list[Trial]
Hand out up to n candidates to evaluate, or all that are ready when n is None.
Returns an empty list when the run is done, or when every candidate of the current generation has been handed out but not all have been told: the next generation depends on their results.
tell(trial: Trial | int, value: Any = None, *, failed: bool = False) -> None
Report the outcome of a trial from ask(), by the trial or its id.
Pass the objective's value, or failed=True when the evaluation could not produce one (a crashed simulation, say). A value of NaN or infinity also counts as failed. Failed trials count as evaluations, rank below every successful one, and are never used as parents.
save_checkpoint(path: str | Path) -> None
Save the run's whole state to path, so load_checkpoint() can carry it on later, in another process.
The file is replaced in one step, so a crash while saving leaves the previous checkpoint intact.
load_checkpoint(path: str | Path) -> None
Carry on the run saved in path by save_checkpoint(), from where it was saved.
The checkpoint must come from an optimizer set up like this one: the same class, config, search space and initial points. Trials handed out but not yet told when it was saved are handed out again by ask(), and can still be told by their ids.
A checkpoint is a pickle, which can run code when it is loaded: only load checkpoints you saved yourself.
optimize(objective_function: Callable, *, workers: int = 1, timeout: float | None = None, callback: Callable[[Any], bool | None] | None = None, checkpoint: str | Path | None = None) -> ResultT
Run the optimizer from the start, calling objective_function for every candidate.
The function is called with one keyword argument per parameter. To record an evaluation as failed, raise EvaluationFailed or return NaN or infinity; any other exception stops the run. With a seed, every call with the same number of workers returns the same result. The genetic optimizer and random search return it however many workers are used; CMA-ES and Bayesian optimization do not, because their default population grows with the number of workers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
objective_function
|
Callable
|
The function to optimize. |
required |
workers
|
int
|
Evaluate candidates in this many background processes at once. |
1
|
timeout
|
float | None
|
The longest, in seconds, one evaluation may take. An evaluation that runs longer is stopped and counts as failed, as does one whose process crashes. |
None
|
callback
|
Callable[[Any], bool | None] | None
|
Called with the progress after each generation. Return True from it to stop the run; optimize() then returns the result so far. |
None
|
checkpoint
|
str | Path | None
|
A file to save the run's state to after each generation. If the file exists, the run carries on from it instead of starting over, so a run killed partway through picks up where it left off; one that finished returns its result at once. The result is the same as an uninterrupted run's. |
None
|
Pressing Ctrl+C also stops the run and returns the result so far, marked as stopped, once at least one evaluation has succeeded.
With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.
result() -> ParetoResult
The front of everything evaluated so far, with a summary of every completed generation.
Evaluations¶
pydantic-model
¶
Bases: BaseModel
A candidate handed out by ask(): evaluate params, then report the outcome with tell().
Config:
frozen:True
Fields:
-
id(int) -
params(dict[str, Any]) -
generation(int)
Bases: Exception
Raise this from an objective function to record the evaluation as failed rather than stop the run.
Bases: ValueError
The optimizer cannot search this kind of parameter.