Probability
•
Probability theory
•
Mathematical optimization
•
Monte Carlo method
•
Control theory
•
Optimization algorithms
Optimization algorithms and methods
Estimation theory
Stochastic algorithms
Monte-Carlo simulation
Stochastic optimization
Stochastic optimization (SO) methods are optimization methods that generate and use random variables. For stochastic problems, the random variables appear in the formulation of the optimization proble...
Metaheuristic
•
Sequential experiments
Metaheuristic
In computer science and mathematical optimization, a metaheuristic is a higher-level procedure or heuristic designed to find, generate, or select a lower-level procedure or heuristic (partial search a...
Metaheuristic - Wikipedia
Sequential experiments
Sequential experiments - Wikipedia
Response surface methodology
In statistics, response surface methodology (RSM) explores the relationships between several explanatory variables and one or more response variables. The method was introduced by G. E. P. Box and K....
Neyer d-optimal test
The Neyer D-Optimal Test is a sensitivity test. It can be used to answer questions such as "How far can a carton of eggs fall, on average, before one breaks?" If these egg cartons are very expensive,...
Simulation Optimization Library: Throughput Maximization
The problem of Throughput Maximization is a family of iterative stochastic optimization algorithms that attempt to find the maximum expected throughput in an n-stage Flow line. According to Pichitlamk...
Bayesian optimization
Bayesian optimization is a sequential design strategyfor global optimization of black-box functions.
The term is generally attributed to Jonas Mockus and is coined in his work from a series of pub...
Multi-armed bandit
In probability theory, the multi-armed bandit problem (sometimes called the K- or N-armed bandit problem) is a problem in which a gambler at a row of slot machines (sometimes known as "one-armed bandi...
BRST algorithm
Boender-Rinnooy-Stougie-Timmer algorithm (BRST) is an optimization algorithm suitable for finding global optimum of black box functions. In their paper Boender et al. describe their method as a stoch...
Thompson sampling
In artificial intelligence, Thompson sampling, named after William R. Thompson, is a heuristic for choosing actions that addresses the exploration-exploitation dilemma in the multi-armed bandit proble...
CMA-ES
CMA-ES stands for Covariance Matrix Adaptation Evolution Strategy. Evolution strategies (ES) are stochastic, derivative-free methods for numerical optimization of non-linear or non-convex continuous ...
CMA-ES - Wikipedia
Pocock boundary
The Pocock boundary is a method for determining whether to stop a clinical trial prematurely. The typical clinical trial compares two groups of patients. One group are given a placebo or conventional ...
Random search
Random search (RS) is a family of numerical optimization methods that do not require the gradient of the problem to be optimized, and RS can hence be used on functions that are not continuous or diffe...
Haybittle–Peto boundary
The Haybittle–Peto boundary is a rule for deciding when to stop a clinical trial prematurely.The typical clinical trial compares two groups of patients. One group are given a placebo or conventional ...
Stochastic gradient descent
Stochastic gradient descent is a gradient descent optimization method for minimizing an objective function that is written as a sum of differentiable functions.
Both statistical estimation and mac...
Stochastic gradient descent - Wikipedia
Biogeography-based optimization
Biogeography-based optimization (BBO) is an evolutionary algorithm (EA) that optimizes a function by stochastically and iteratively improving candidate solutions with regard to a given measure of qual...
Biogeography-based optimization - Wikipedia
Natural evolution strategy
Natural evolution strategies (NES) are a family of numerical optimization algorithms for black-box problems. Similar in spirit to evolution strategies, they iteratively update the (continuous) paramet...
Estimating equations
In statistics, the method of estimating equations is a way of specifying how the parameters of a statistical model should be estimated. This can be thought of as a generalisation of many classical met...
Stochastic approximation
Stochastic approximation methods are a family of iterative stochastic optimization algorithms that attempt to find zeroes or extrema of functions which cannot be computed directly, but only estimated ...
Stochastic programming
In the field of mathematical optimization, stochastic programming is a framework for modeling optimization problems that involve uncertainty. Whereas deterministic optimization problems are formulated...
Quantum annealing
Quantum annealing (QA) is a metaheuristic for finding the global minimum of a given objective function over a given set of candidate solutions (candidate states), by a process using quantum fluctuatio...
Quantum annealing - Wikipedia
Optimal computing budget allocation
Optimal computing budget allocation (OCBA) is a concept first introduced in the mid 1990s by Dr Chun-Hung Chen. This approach intends to maximize the overall simulation efficiency for finding an op...
Optimal computing budget allocation - Wikipedia
Parallel tempering
Parallel tempering, also known as replica exchange MCMC sampling, is a simulation method aimed at improving the dynamic properties of Monte Carlo method simulations of physical systems, and of Markov...
Scenario optimization
The scenario approach or scenario optimization approach is a technique for obtaining solutions to robust optimization and chance-constrained optimization problems based on randomization of the constra...
M-estimation
In statistics, M-estimators are a broad class of estimators, which are obtained as the minima of sums of functions of the data. Least-squares estimators are M-estimators. The definition of M-estimator...
Stochastic tunneling
In numerical analysis, stochastic tunneling (STUN) is an approach to global optimization based on the Monte Carlo method-sampling of the function to be objective minimized in which the function is non...
Stochastic tunneling - Wikipedia
Ant colony optimization algorithms
In computer science and operations research, the ant colony optimization algorithm (ACO) is a probabilistic technique for solving computational problems which can be reduced to finding good paths thro...
Ant colony optimization algorithms - Wikipedia