Guidelines for benchmarking of optimization-based approaches for fitting mathematical models

Table 1 Covariates

Abbreviation	Covariate	Typical possible choices
C1	Application problem	Model equations and the data set(s)
C2	Primary performance criteria	Convergence per computation time, iteration steps
C3	Secondary performance criteria	Documentation, user-friendliness, code quality
C4	Parameter scale	Linear vs. log scale
C5	Global search strategy	Multiple initial guesses, scatter search algorithms, stochastic search
C6	Initial guess strategy	Fixed vs. random, normally distributed vs. uniform vs. latin-hypercube
C7	Parameter constraints	Upper and lower bounds
C8	Prior knowledge	None vs. (weakly) informative priors
C9	ODE integration implementation	SUNDIALS, Matlab, R
C10	ODE integration algorithm	Stiff vs. non-stiff approaches, Adams-Moulton vs. BDF
C11	Integration accuracy	ODE integrator tolerances
C12	Derivative calculation	Finite differences, sensitivity equations, adjoint sensitivities
C13	Stopping rule	Optimization termination criteria
C14	Handling of non-converging ODE integration	Termination of optimization vs. infinite loss
C15	Algorithm-specific configurations	Cross-over rate, annealing temperature, number of particles

The performance of an optimization approaches depend on many decisions and configurations C1–C15. For the comparison of several approaches, these attributes appear as covariates. Performance benefits for individual choices do not necessarily indicate a general advantage because benefits might merely originate from the chosen configurations

ISSN: 1474-760X