Skip to content

Sensitivity analysis

sensitivity() measures which parameters matter. See Which parameters matter for when to use Morris screening or Sobol indices, and how to fix the parameters that don't matter.

sensitivity 

sensitivity(objective_function: Callable, search_space: SearchSpace | Sequence[Parameter], *, max_evaluations: int = 500, method: Method | None = None, maximize: bool = False, seed: int | None = None, workers: int = 1, timeout: float | None = None, confidence: float = 0.95) -> SensitivityReport

Measure how much each parameter of the search space matters to the objective.

Morris screening needs about 10 × (parameters + 1) evaluations to rank the parameters. Sobol indices need about 64 × (parameters + 2) or more, and say what share of the objective's variance each parameter accounts for. Without method, Sobol is used when max_evaluations allows it, and Morris otherwise. Neither uses more than max_evaluations.

The objective is called as optimize() calls it: with one keyword argument per parameter switched on, raising EvaluationFailed or returning NaN or infinity to record a failed evaluation. Failed evaluations are left out of the indices.

Parameters:

Name Type Description Default
objective_function Callable

The function to measure.

 required
search_space SearchSpace | Sequence[Parameter]

The parameters, with any conditions; constraints aren't supported yet.

 required
max_evaluations int

The most evaluations to make.

 500
method Method | None

"morris" or "sobol", rather than choosing from max_evaluations.

 None
maximize bool

Whether higher values are better, which only decides best_params.

 False
seed int | None

Fixes the points evaluated and the intervals, so a run can be repeated.

 None
workers int

Evaluate this many points at once in background processes.

 1
timeout float | None

The longest, in seconds, one evaluation may take, as in optimize().

 None
confidence float

The confidence level of the intervals, from bootstrapping.

 0.95

With workers > 1 or a timeout, each evaluation runs in a separate process, so the function must be defined at the top level of a module.

SensitivityReport pydantic-model

Bases: BaseModel

What sensitivity() measured: each parameter's importance, most important first.

Config:

  • frozen: True
  • use_attribute_docstrings: True

Fields:

method pydantic-field 

method: Method

How the parameters were measured: "morris" or "sobol".

parameters pydantic-field 

parameters: list[ParameterSensitivity]

Each parameter measured, most important first.

evaluations pydantic-field 

evaluations: int

Evaluations made, failed ones included.

failures pydantic-field 

failures: int

Evaluations that failed, timed out or crashed their worker.

confidence pydantic-field 

confidence: float

The confidence level of the intervals.

best_params pydantic-field 

best_params: dict[str, Any] | None

The best point evaluated, as the objective was called with it; None if every evaluation failed.

best_value pydantic-field 

best_value: float | None

The objective's value at best_params; None if every evaluation failed.

maximize pydantic-field 

maximize: bool

Whether higher values were counted as better in choosing best_params.

ranking property 

ranking: list[str]

The parameters' names, most important first.

unimportant 

unimportant(threshold: float | None = None) -> list[str]

The parameters that can be fixed with little effect: those whose whole interval is below threshold.

For Sobol, threshold is a total index, 0.05 unless given: the parameter accounts for under 5% of the objective's variance, interactions included. For Morris it is a share of the largest mu_star, 0.1 unless given. Only the upper end of each interval is compared, so a parameter whose importance is uncertain is kept. A parameter without an interval, because too few of its evaluations succeeded, is kept too.

reduced 

reduced(search_space: SearchSpace | Sequence[Parameter], *, threshold: float | None = None, keep: int | None = None, at: dict[str, Any] | None = None) -> SearchSpace

The search space with its unimportant parameters fixed, ready for choose_optimizer.

The parameters unimportant() names are fixed, or with keep, every parameter but the keep most important. Each is fixed at its value in at, if given, and otherwise at the best point evaluated; one the conditions switched off there, or every parameter if every evaluation failed, is fixed at the middle of its range, or its middle choice.

A fixed parameter is still passed to the objective, so the objective doesn't change. Conditions and constraints are kept: a parameter one of them tests stays a range or choice parameter, narrowed to its one value, so they still apply.

Parameters:

Name Type Description Default
search_space SearchSpace | Sequence[Parameter]

The search space the report measured.

 required
threshold float | None

Passed to unimportant(). Not with keep.

 None
keep int | None

Keep this many of the most important parameters, and fix the rest.

 None
at dict[str, Any] | None

Values to fix parameters at, by name.

 None

ParameterSensitivity pydantic-model

Bases: BaseModel

How much one parameter matters.

importance ranks the parameters: mu_star for Morris, the total index for Sobol. The other fields are None for the method that wasn't used.

Config:

  • frozen: True
  • use_attribute_docstrings: True

Fields:

name pydantic-field 

name: str

The parameter's name.

importance pydantic-field 

importance: float

mu_star for Morris, the total index for Sobol.

interval pydantic-field 

interval: Interval | None = None

The confidence interval of importance; None when too few evaluations succeeded to estimate it.

mu_star pydantic-field 

mu_star: float | None = None

Morris: the mean size of the parameter's effect, the change in the objective across its whole range, whichever way it goes. Small means the parameter barely matters.

mu pydantic-field 

mu: float | None = None

Morris: the mean effect with its sign. Well below mu_star means the effect changes direction.

sigma pydantic-field 

sigma: float | None = None

Morris: the standard deviation of the effect. Large against mu_star means the effect depends on the other parameters, through interactions or a curved response.

first pydantic-field 

first: float | None = None

Sobol: the share of the objective's variance the parameter accounts for alone.

first_interval pydantic-field 

first_interval: Interval | None = None

Sobol: the confidence interval of first.

total pydantic-field 

total: float | None = None

Sobol: the share of the variance the parameter accounts for with all its interactions; near 0 means it can be fixed at any value with little effect. Estimates can fall slightly below 0.

total_interval pydantic-field 

total_interval: Interval | None = None

Sobol: the confidence interval of total, the same as interval.

Interval pydantic-model

Bases: BaseModel

A confidence interval.

Config:

  • frozen: True
  • use_attribute_docstrings: True

Fields:

low pydantic-field 

low: float

The lower end.

high pydantic-field 

high: float

The upper end.