Multidimensional Benchmarks Results

This page shows the results obtained by applying a number of Global optimization algorithms to the entire benchmark suite of N-D optimization problems, together with some statistics on the algorithm performances.

Multi-Dimensional (N-D) Test Functions

The following table shows the overall success of all Global Optimization algorithms, considering for every benchmark function 100 random starting points.

So, for example, AMPGO was able to solve, on average, 79.0% of all the test functions for all the 100 random starting points using, on average, 611 functions evaluations.

Optimization algorithms performances (N-dimensional)

Optimization Method	Overall Success (%)	Functions Evaluations
AMPGO	79.027	611
ASA	58.457	3566
BasinHopping	49.478	1220
CMA-ES	57.038	603
CRS2	57.315	963
DE	25.533	1429
DIRECT	55.027	563
Firefly	53.565	1462
Galileo	4.978	125
MLSL	20.082	3096
PSWARM	52.815	1163
SCE	68.005	970
SIMANN	6.446	1390

These results are also depicted in the next figure, which clearly shows that AMPGO is the better-performing optimization algorithms as far as the current benchmark is considered.

AMPGO Optimization algorithms performances (N-dimensional)

It is also interesting to analyze the success of an optimization algorithm based on the fraction (or percentage) of problems solved given a fixed number of allowed function evaluations, let’s say from 100 to 2000. The next figure presents such analysis, and it is clear that, on average, AMPGO outperforms all the other optimization methods using only 200 function evaluations (against the 2000 allowed for all the others).

AMPGO Percentage of problems solved given a fixed number of function evaluations (N-dimensional)

After the following comment from Geoffrey Oxberry in the scipy mailing list:

Note

One of the typical plots used to assess performance is a “performance profile”, which was defined in “Benchmarking Optimization Software with Performance Profiles” by Dolan and More (http://arxiv.org/pdf/cs/0102001.pdf). I didn’t see any plots in this format. Do you plan on presenting performance data in this manner? The solved problems versus function evaluations looks pretty close to this sort of presentation. This sort of format also avoids some of the pitfalls mentioned on your site re: performance comparisons.

I converted the Matlab code used by Dolan and More to Python and produced the following output:

AMPGO Performance profiles for the various optimization algorithms (N-dimensional)

The following table is a split-by-benchmark function of the first table, showing the percentage of successful optimizations per benchmark, considering 100 random starting points.

Optimization algorithms performances (N-dimensional)

Function Name	N	AMPGO	ASA	BasinHopping	CMA-ES	CRS2	DE	DIRECT	Firefly	Galileo	MLSL	PSWARM	SCE	SIMANN
Ackley	2	43	99	1	17	80	0	100	0	0	0	2	100	0
Adjiman	2	100	100	100	99	96	100	0	100	0	0	100	100	0
Alpine01	2	100	100	0	98	61	0	100	1	0	100	64	67	0
Alpine02	2	98	100	31	38	86	100	100	97	0	0	78	99	74
AMGM	2	100	100	100	100	53	100	0	100	59	0	88	100	0
BartelsConn	2	100	100	0	100	92	0	0	13	0	0	23	100	0
Beale	2	100	3	100	71	70	0	100	78	0	0	83	100	0
Bird	2	100	100	44	53	64	100	100	70	0	0	73	97	0
Bohachevsky	2	100	100	100	91	94	0	100	95	0	0	91	100	0
BoxBetts	3	100	0	92	89	88	0	0	86	0	0	8	100	0
Branin01	2	100	100	100	100	81	0	100	100	0	0	70	96	0
Branin02	2	100	18	24	5	21	0	100	6	0	0	5	16	0
Brent	2	100	100	100	100	34	0	100	100	0	0	100	100	0
Brown	2	100	96	100	100	83	0	100	100	0	0	100	100	0
Bukin02	2	100	100	100	100	100	100	100	100	100	0	100	100	0
Bukin04	2	100	100	0	100	80	100	0	61	0	100	100	100	0
Bukin06	2	94	0	0	0	0	0	0	0	0	0	0	0	0
CarromTable	2	100	100	41	13	43	100	100	93	57	0	99	100	0
Chichinadze	2	71	100	18	9	41	0	54	15	0	0	75	57	0
Cigar	2	100	98	0	100	71	0	0	0	0	100	32	100	0
Cola	17	58	0	0	0	0	0	0	0	0	0	0	0	0
Colville	4	100	0	0	100	0	100	100	100	9	100	84	100	1
Corana	4	0	100	55	6	0	0	0	4	0	0	51	31	0
CosineMixture	2	100	100	100	100	0	100	100	100	100	0	100	100	0
CrossInTray	2	79	100	4	12	98	0	100	100	0	0	100	71	0
CrossLegTable	2	0	0	0	0	0	0	0	0	0	0	0	0	9
CrownedCross	2	3	0	0	0	0	0	0	0	0	0	0	0	0
Csendes	2	100	100	100	100	99	0	100	100	12	0	100	100	0
Cube	2	100	0	0	95	64	51	0	19	0	54	48	100	0
Damavandi	2	18	0	0	0	1	0	0	0	0	0	0	0	0
Deb01	2	100	100	100	99	55	100	0	100	0	100	27	0	0
Deb02	2	100	100	100	98	94	0	100	98	0	100	16	9	0
Decanomial	2	93	35	0	97	91	0	0	20	0	37	99	100	0
Deceptive	2	31	98	0	3	7	0	0	0	0	100	2	67	0
DeckkersAarts	2	100	100	99	97	97	100	100	83	0	0	100	100	0
DeflectedCorrugatedSpring	2	96	0	100	0	48	100	0	65	0	0	100	47	0
DeVilliersGlasser01	4	100	0	7	6	0	0	0	0	0	100	0	0	0
DeVilliersGlasser02	5	1	0	0	0	0	0	0	0	0	0	0	0	0
DixonPrice	2	100	97	100	100	82	56	100	100	0	41	100	97	0
Dolan	5	100	0	0	87	0	0	100	4	0	100	17	0	0
DropWave	2	34	20	93	0	43	0	0	57	0	0	5	0	0
Easom	2	5	74	0	4	25	0	0	0	0	62	1	100	0
EggCrate	2	94	100	31	14	78	0	100	87	0	47	100	100	0
EggHolder	2	59	2	2	1	19	0	0	7	0	0	24	0	100
ElAttarVidyasagarDutta	2	100	44	62	52	64	0	100	1	0	0	56	92	0
Exp2	2	100	98	10	7	20	0	0	1	0	0	54	89	0
Exponential	2	100	100	100	100	97	47	100	100	0	0	100	100	0
FreudensteinRoth	2	100	58	25	37	55	0	0	28	0	100	95	95	0
Gear	4	44	100	3	96	97	100	0	100	1	0	91	98	0
Giunta	2	100	100	69	60	96	0	100	93	0	0	100	100	0
GoldsteinPrice	2	100	74	94	82	82	0	100	97	0	0	100	99	0
Griewank	2	3	17	0	0	60	0	0	3	0	0	1	0	0
Gulf	3	59	0	3	1	41	0	0	0	0	0	0	19	0
Hansen	2	80	56	80	17	43	52	51	91	0	0	31	5	0
Hartmann3	3	100	0	100	92	87	0	0	26	0	0	100	100	0
Hartmann6	6	100	0	20	67	1	0	100	54	0	0	63	95	0
HelicalValley	3	100	0	100	100	15	0	52	0	0	81	0	100	0
HimmelBlau	2	100	0	100	100	87	0	100	100	0	0	59	97	0
HolderTable	2	100	100	52	20	91	100	100	95	96	0	100	100	0
Holzman	3	100	100	80	42	0	100	0	100	0	100	0	100	100
Hosaki	2	100	100	87	88	97	100	100	99	0	0	100	100	0
Infinity	2	100	100	100	100	100	0	100	100	10	0	100	100	0
JennrichSampson	2	100	2	0	100	81	0	50	61	0	0	54	100	0
Judge	2	100	8	100	77	86	0	41	100	0	0	98	100	0
Katsuura	2	100	92	100	96	0	0	100	100	0	0	100	99	0
Keane	2	100	100	100	100	60	100	0	100	100	100	9	100	0
Kowalik	4	92	0	4	19	1	0	0	0	0	0	0	67	0
Langermann	2	97	70	71	10	65	100	0	62	0	0	82	100	0
LennardJones	6	100	0	90	85	6	0	100	100	0	100	52	5	0
Leon	2	100	0	0	100	61	0	56	44	0	0	1	100	0
Levy03	2	100	100	96	28	96	58	0	100	0	56	100	100	0
Levy05	2	26	50	74	14	47	100	43	81	0	0	56	68	0
Levy13	2	100	100	100	64	87	58	0	100	0	47	70	100	0
Matyas	2	100	31	100	100	81	44	100	100	0	0	100	100	0
McCormick	2	100	100	52	59	71	0	100	99	0	0	100	100	0
Michalewicz	2	100	100	100	53	86	100	100	90	0	0	100	100	36
MieleCantrell	4	93	1	97	100	98	0	50	51	0	0	28	100	0
Mishra01	2	100	100	100	98	0	100	100	100	0	0	5	100	0
Mishra02	2	100	100	100	100	0	100	100	100	0	0	100	100	0
Mishra03	2	43	7	0	4	20	0	50	0	0	0	24	4	93
Mishra04	2	29	3	0	1	15	0	0	2	0	0	6	2	91
Mishra05	2	97	78	88	14	96	0	44	21	3	0	60	71	0
Mishra06	2	100	54	36	49	79	47	100	91	0	0	85	0	0
Mishra07	2	100	100	100	100	90	53	100	100	0	100	58	77	0
Mishra08	2	94	38	0	98	94	0	0	18	0	88	99	100	0
Mishra09	3	100	100	0	99	58	0	100	41	0	61	15	37	0
Mishra10	2	0	99	20	37	0	100	0	99	34	0	79	99	78
Mishra11	2	100	100	100	100	88	53	100	100	0	47	71	37	5
MultiModal	2	100	100	0	100	88	0	100	97	0	100	100	100	0
NeedleEye	2	100	100	0	100	0	100	0	37	0	0	100	95	0
NewFunction01	2	60	7	0	2	52	0	0	4	0	0	12	29	35
NewFunction02	2	42	1	0	2	19	0	0	1	0	0	5	23	36
NewFunction03	2	100	83	100	19	100	100	57	88	28	0	95	61	0
OddSquare	2	6	18	2	8	92	100	0	100	0	0	55	5	2
Parsopoulos	2	100	100	89	94	61	0	100	100	0	0	41	34	0
Pathological	2	30	15	8	0	0	0	100	0	0	0	92	0	0
Paviani	10	100	0	100	100	0	0	0	0	0	0	0	85	0
Penalty01	2	100	99	38	91	98	0	100	100	0	0	52	100	0
Penalty02	2	52	43	100	29	75	0	0	53	0	0	20	16	0
PenHolder	2	100	100	19	19	96	100	100	98	95	0	100	100	100
PermFunction01	2	100	0	100	100	79	0	100	100	0	100	17	100	0
PermFunction02	2	100	0	100	100	85	0	100	100	0	0	99	100	0
Pinter	2	96	82	97	34	79	0	100	98	0	52	54	92	0
Plateau	2	43	100	53	83	0	100	0	100	93	0	100	100	100
Powell	4	100	0	0	100	43	0	0	2	0	0	6	100	0
PowerSum	4	92	0	0	16	0	0	0	0	0	0	0	1	0
Price01	2	100	100	100	99	71	0	100	6	0	55	100	68	0
Price02	2	65	43	1	2	34	0	0	36	0	0	24	36	0
Price03	2	100	5	55	52	38	0	100	7	0	0	54	54	0
Price04	2	100	18	4	100	81	42	44	65	0	70	19	100	0
Qing	2	100	100	100	100	85	0	100	2	0	0	100	75	0
Quadratic	2	100	100	100	100	96	100	100	97	0	0	100	100	0
Quintic	2	100	95	0	100	65	52	56	0	0	100	50	83	0
Rana	2	96	95	13	7	49	100	54	21	29	0	94	65	95
Rastrigin	2	97	73	81	6	88	0	0	49	0	0	31	48	0
Ripple01	2	100	32	0	3	68	0	0	2	0	0	32	0	0
Ripple25	2	100	93	65	8	89	0	0	7	0	0	87	67	0
Rosenbrock	2	100	0	0	100	67	0	100	51	0	100	0	100	0
RosenbrockModified	2	60	4	0	2	13	55	0	10	0	0	1	9	0
RotatedEllipse01	2	100	98	100	100	69	0	100	1	0	0	100	100	0
RotatedEllipse02	2	100	100	100	98	79	0	100	9	0	0	100	100	0
Salomon	2	5	8	95	0	25	0	0	0	0	0	3	0	0
Sargan	2	100	100	100	100	87	0	100	70	0	0	100	100	0
Schaffer01	2	3	40	1	0	87	0	0	91	0	0	14	0	0
Schaffer02	2	15	36	3	4	93	48	0	100	0	100	24	3	0
Schaffer03	2	23	39	0	7	91	44	0	100	0	0	18	6	0
Schaffer04	2	13	31	2	0	91	32	0	99	0	0	6	3	2
SchmidtVetters	3	100	98	100	90	100	100	100	100	34	0	53	100	98
Schwefel01	2	100	100	100	100	91	0	100	100	0	0	100	100	0
Schwefel02	2	100	100	100	100	94	0	100	100	0	51	100	100	0
Schwefel04	2	100	100	100	100	58	0	100	100	0	100	100	100	0
Schwefel06	2	100	0	0	100	70	0	100	0	0	0	4	100	0
Schwefel20	2	100	94	0	100	61	0	100	0	0	48	10	100	0
Schwefel21	2	100	88	0	100	71	0	100	0	0	0	19	100	0
Schwefel22	2	98	98	0	96	63	0	100	0	0	100	10	100	0
Schwefel26	2	80	84	10	10	55	52	100	51	48	0	79	92	99
Schwefel36	2	0	100	100	24	18	0	100	100	0	0	87	100	0
Shekel05	4	100	0	46	43	25	0	100	48	0	0	48	42	0
Shekel07	4	100	0	52	35	31	0	100	60	0	0	63	70	0
Shekel10	4	100	0	52	35	40	0	100	49	0	0	53	71	0
Shubert01	2	91	100	90	19	57	0	46	57	0	0	28	0	0
Shubert03	2	82	93	17	3	16	0	100	53	0	0	18	0	0
Shubert04	2	88	100	89	8	50	100	49	79	0	0	31	6	0
SineEnvelope	2	9	6	1	0	6	0	0	3	0	0	1	0	0
SixHumpCamel	2	100	100	100	90	96	100	100	100	0	0	100	99	0
Sodp	2	100	100	100	100	94	0	100	100	0	0	100	100	0
Sphere	2	100	100	100	100	84	0	100	100	0	48	100	100	0
Step	2	0	100	0	92	0	100	0	100	1	0	100	100	4
Stochastic	2	0	5	0	20	13	0	97	0	0	0	0	0	0
StretchedV	2	100	0	96	99	76	0	100	100	2	100	39	25	0
StyblinskiTang	2	100	100	55	28	80	0	100	78	0	0	99	98	0
TestTubeHolder	2	91	61	42	3	81	100	0	31	0	0	35	6	0
ThreeHumpCamel	2	100	83	100	78	91	0	100	100	0	0	99	100	0
Treccani	2	100	100	100	100	87	49	100	100	0	64	100	99	0
Trefethen	2	1	4	8	5	9	0	0	1	0	0	4	0	0
Trid	6	100	0	100	100	0	0	100	4	0	0	18	100	0
Trigonometric01	2	100	100	100	100	46	100	100	100	3	100	86	100	0
Trigonometric02	2	9	99	59	70	88	0	49	5	0	0	24	49	0
Tripod	2	99	20	0	49	34	0	100	0	0	50	2	43	0
Ursem01	2	100	100	60	60	100	100	100	99	0	0	100	100	0
Ursem03	2	43	100	3	13	80	0	100	0	0	42	6	100	18
Ursem04	2	100	93	0	62	73	0	100	2	0	100	8	100	0
UrsemWaves	2	100	6	100	51	0	0	0	61	0	0	47	0	0
VenterSobiezcczanskiSobieski	2	61	100	62	2	89	0	100	1	0	0	30	87	0
Vincent	2	100	100	100	100	94	0	100	98	0	0	47	59	0
Watson	6	100	0	0	46	0	0	0	0	0	0	0	77	0
Wavy	2	39	80	8	2	84	0	0	63	0	0	62	30	0
WayburnSeader01	2	100	6	0	100	86	0	38	50	0	94	59	97	0
WayburnSeader02	2	100	4	0	97	90	0	0	5	0	0	2	97	0
Weierstrass	2	0	57	0	92	81	0	0	0	0	0	0	98	0
Whitley	2	5	2	28	3	24	0	0	0	0	0	2	0	0
Wolfe	3	100	0	100	100	0	0	0	100	0	0	100	100	0
XinSheYang01	2	0	24	0	16	62	2	41	7	0	0	0	0	0
XinSheYang02	2	63	35	0	10	49	0	100	0	0	0	5	100	0
XinSheYang03	2	5	3	0	0	0	0	0	5	0	0	0	0	0
XinSheYang04	2	40	53	0	1	65	0	0	0	0	0	4	100	0
Xor	9	100	0	55	79	49	0	0	41	0	0	70	13	7
YaoLiu04	2	100	100	0	100	83	0	100	36	0	100	34	96	1
YaoLiu09	2	97	67	79	5	85	0	0	47	0	0	35	41	0
Zacharov	2	100	100	100	100	86	0	100	100	0	0	100	100	0
ZeroSum	2	0	0	0	69	0	0	0	0	0	0	8	100	0
Zettl	2	100	100	100	99	87	0	100	100	0	0	100	100	0
Zimmerman	2	1	5	0	37	14	0	0	0	0	0	0	47	0
Zirilli	2	100	100	100	83	97	100	100	100	0	0	100	100	0

The following table is a split-by-benchmark function of the first table, showing the average number of functions evaluations for successful optimizations only, considering 100 random starting points.

Optimization algorithms performances (N-dimensional)

Function Name	N	AMPGO	ASA	BasinHopping	CMA-ES	CRS2	DE	DIRECT	Firefly	Galileo	MLSL	PSWARM	SCE	SIMANN
Ackley	2	1515	1288	1691	454	1201	2019	234	2000	85	2586	349	522	1186
Adjiman	2	14	1254	639	239	273	1453	176	392	172	2347	450	394	2001
Alpine01	2	251	1267	2002	614	931	2019	511	1999	99	1126	842	1370	910
Alpine02	2	271	1207	575	298	600	823	218	1156	151	3358	968	379	1188
AMGM	2	15	471	466	187	965	360	2001	28	63	2255	864	35	2001
BartelsConn	2	102	1234	2033	657	795	2019	322	1976	89	1634	345	335	1496
Beale	2	33	857	1645	531	596	2019	600	1452	94	5131	1341	316	1597
Bird	2	85	1224	883	350	646	1377	253	1851	129	3130	1191	492	821
Bohachevsky	2	250	1286	1229	471	755	2019	290	1788	88	2207	339	420	1063
BoxBetts	3	29	4589	1940	573	628	2018	196	870	114	6707	3472	214	2001
Branin01	2	11	1265	875	407	564	969	264	1322	97	2990	1258	494	1221
Branin02	2	290	955	713	386	620	1011	620	1929	99	3974	958	1819	1099
Brent	2	2	1223	241	474	1170	2019	278	1366	102	2069	395	281	1360
Brown	2	14	1197	591	357	465	2019	156	1020	120	2123	340	192	1108
Bukin02	2	8	1242	488	39	371	46	698	23	24	2829	462	14	2001
Bukin04	2	68	1371	2025	705	682	360	2001	1759	59	59	450	316	1836
Bukin06	2	842	1226	2047	965	877	381	2001	2000	91	238	928	2004	877
CarromTable	2	101	1266	495	316	900	213	210	410	81	5512	469	365	2001
Chichinadze	2	1209	1214	1162	517	597	2019	174	1974	112	3679	474	1397	1222
Cigar	2	26	1245	2018	900	926	2019	258	2000	–	942	503	418	1474
Cola	17	1360	60687	2018	2006	2001	2055	2001	2000	925	7979	4020	2039	669
Colville	4	125	7929	2041	242	2003	371	826	196	112	746	1469	151	1167
Corana	4	2026	576	542	776	2001	1447	2001	1948	228	3749	665	1780	1159
CosineMixture	2	3	1284	208	8	2001	46	292	20	29	2868	380	10	2001
CrossInTray	2	924	1239	522	305	644	2019	204	1106	148	6148	347	1159	2001
CrossLegTable	2	2007	1255	1615	870	550	2019	138	2000	192	276	2316	2010	793
CrownedCross	2	2020	1254	2969	933	525	2019	190	2000	97	267	2293	2009	2001
Csendes	2	21	516	1046	183	141	2019	40	65	71	2201	301	49	2001
Cube	2	58	738	2015	1195	925	473	1379	1933	59	977	1432	586	1698
Damavandi	2	1827	1270	505	385	586	360	2001	2000	60	2267	281	2004	1260
Deb01	2	35	1229	1368	534	1132	360	2001	1535	116	1060	2563	2011	1264
Deb02	2	113	1220	1103	471	756	444	110	1355	174	167	3226	1912	1954
Decanomial	2	522	37	2015	730	555	2019	92	1926	–	1086	362	264	251
Deceptive	2	3017	1260	7607	606	636	2019	248	2000	184	775	1078	1446	2001
DeckkersAarts	2	44	1304	1059	302	936	791	490	1269	101	5380	473	196	1272
DeflectedCorrugatedSpring	2	700	–	783	361	734	360	2001	1637	60	4221	286	1758	1679
DeVilliersGlasser01	4	231	14114	1963	1972	1853	1274	2001	2000	–	1783	3913	2014	1934
DeVilliersGlasser02	5	2139	47968	1994	2002	1828	1737	2001	2000	348	2303	3922	2020	1692
DixonPrice	2	17	1330	1106	451	644	1607	431	1482	80	2003	361	380	1598
Dolan	5	86	43089	2047	1423	2001	1842	884	1992	175	866	3313	2019	1047
DropWave	2	1677	1138	770	422	871	720	174	1766	165	4391	305	2008	1950
Easom	2	1949	1226	1034	438	1024	2019	180	2000	78	3199	419	700	1019
EggCrate	2	556	1302	744	379	692	2019	270	1617	95	1388	388	352	909
EggHolder	2	1381	1269	680	500	568	1011	600	1947	165	7069	1220	2008	908
ElAttarVidyasagarDutta	2	109	1314	1306	618	842	1456	728	1998	–	4579	392	686	1975
Exp2	2	164	1301	370	343	580	2019	8	1997	94	2735	1318	1022	877
Exponential	2	6	1264	431	313	449	1999	128	776	162	2103	299	160	2001
FreudensteinRoth	2	89	1126	1554	631	648	2019	719	1937	107	1124	919	637	1934
Gear	4	1555	414	101	649	447	1725	2001	146	141	2543	1030	274	2001
Giunta	2	45	1234	693	327	457	2019	219	921	96	2231	319	187	2001
GoldsteinPrice	2	56	1144	1708	438	677	1537	348	1643	90	4375	329	298	1690
Griewank	2	1990	1277	569	439	1056	845	348	1997	95	2556	412	2007	1092
Gulf	3	2037	4382	127	952	1325	1522	522	2000	101	6266	3978	1917	281
Hansen	2	859	1228	1064	418	734	1140	252	1396	141	6219	2721	1987	734
Hartmann3	3	45	4299	1638	616	896	2018	230	1847	295	6698	579	355	2001
Hartmann6	6	69	19805	1996	1481	1959	2016	750	1800	523	7744	1247	952	2001
HelicalValley	3	33	–	1936	1057	1974	1772	1511	2000	99	864	3385	711	835
HimmelBlau	2	15	–	893	406	675	2019	271	1598	94	2216	1577	481	1199
HolderTable	2	92	1279	599	304	309	89	224	289	47	6272	464	62	2001
Holzman	3	194	0	1245	9	2001	66	2001	20	–	335	4015	14	251
Hosaki	2	162	1252	779	216	550	739	302	800	162	2465	349	164	2001
Infinity	2	16	531	1052	181	139	2019	40	64	71	2197	299	49	2001
JennrichSampson	2	32	1381	2019	557	810	927	524	1906	99	7841	552	407	914
Judge	2	38	1269	1175	358	739	2019	526	1614	79	4009	609	290	1424
Katsuura	2	2	1281	177	1237	2001	2019	994	54	105	115	412	1070	2001
Keane	2	6	977	524	8	911	47	2001	20	24	1585	3547	10	2001
Kowalik	4	814	7965	1984	1718	1373	1966	625	2000	158	7766	3888	1456	1995
Langermann	2	865	1245	1123	393	607	1201	178	1662	113	5781	762	727	1977
LennardJones	6	82	23343	242	1675	1978	2016	508	904	260	724	2071	1964	2001
Leon	2	34	853	2015	616	599	1707	1311	1687	85	2952	2466	318	1473
Levy03	2	198	1255	926	373	571	1056	987	1233	81	2281	289	227	1513
Levy05	2	1717	1268	1306	471	908	1360	218	1745	100	4580	1050	1545	825
Levy13	2	430	1242	1554	544	709	1056	1035	1630	78	1824	285	533	1127
Matyas	2	6	1374	607	410	541	1162	272	1078	64	2193	432	203	1915
McCormick	2	28	1274	690	350	419	2019	242	1127	128	2504	369	213	2001
Michalewicz	2	352	1238	1327	262	596	739	144	1196	136	3666	338	246	942
MieleCantrell	4	180	6000	1801	530	308	2012	269	1259	153	7637	2992	241	1951
Mishra01	2	2	1200	152	11	2001	1201	674	45	80	2091	3657	303	2001
Mishra02	2	2	1270	176	23	2001	1789	480	84	106	2164	387	565	2001
Mishra03	2	1499	1154	2041	905	437	1044	421	2000	93	284	1871	1975	1240
Mishra04	2	1722	1194	2044	905	683	626	357	1982	101	247	2177	1994	1266
Mishra05	2	558	1285	1643	681	362	602	494	1681	94	5786	1026	1152	2001
Mishra06	2	219	1244	899	412	659	1614	250	1527	130	3716	357	2005	1393
Mishra07	2	9	1169	583	401	653	1152	262	824	80	322	1651	958	1414
Mishra08	2	472	29	2016	734	547	2019	91	1938	–	1658	387	267	251
Mishra09	3	177	0	2024	1072	1665	1626	469	1775	–	288	2844	1592	251
Mishra10	2	2139	58	111	145	2001	381	2001	66	65	2565	888	64	850
Mishra11	2	13	1168	667	380	659	791	213	611	87	2612	1336	1609	1672
MultiModal	2	58	1042	2028	465	442	2019	160	808	94	677	338	270	1418
NeedleEye	2	46	649	2020	454	2001	1372	332	1932	97	2634	309	545	1074
NewFunction01	2	1345	1142	1975	908	907	528	324	1946	99	232	1686	1722	1702
NewFunction02	2	1557	1151	1988	906	541	519	279	1988	78	190	2408	1779	1730
NewFunction03	2	432	1337	1118	479	343	881	349	510	85	4323	382	1064	2001
OddSquare	2	1957	1033	245	109	1085	1649	62	1377	87	4554	402	1966	1979
Parsopoulos	2	19	1260	497	335	551	2019	184	1194	95	2501	2512	1619	1969
Pathological	2	1684	1240	751	557	1714	360	142	2000	61	5014	822	2009	1708
Paviani	10	29	79149	1357	1457	2001	2023	2001	2000	1062	8139	2955	1829	258
Penalty01	2	143	1278	783	494	660	2019	289	1395	84	2623	345	352	1754
Penalty02	2	1393	1251	1752	700	792	2019	401	1901	75	2365	371	1895	1533
PenHolder	2	166	1264	574	241	428	76	138	128	53	6024	537	43	291
PermFunction01	2	18	–	1597	428	640	549	380	1173	84	219	2377	284	1709
PermFunction02	2	17	–	1549	445	732	1137	250	1574	97	2954	804	361	1284
Pinter	2	628	1298	1183	460	711	2019	490	1751	85	1908	336	710	925
Plateau	2	1669	17	101	152	2001	360	2001	32	65	2880	242	23	747
Powell	4	36	11842	2014	1086	1908	1069	1014	1989	150	7419	3806	560	1136
PowerSum	4	998	11953	2123	1874	2001	1520	934	2000	159	7915	3971	2012	1426
Price01	2	6	1272	368	591	838	2019	432	1994	83	1092	382	1278	994
Price02	2	1307	1255	496	329	769	1281	164	1771	94	2707	2269	1817	1728
Price03	2	93	1282	1918	585	874	1172	612	1991	87	2336	1563	1177	1367
Price04	2	48	1117	2009	770	715	1520	428	1691	–	1885	2394	359	1864
Qing	2	27	1321	963	615	932	2019	566	1998	–	2414	383	1084	1316
Quadratic	2	6	1283	651	291	792	1378	429	1758	119	2678	327	244	1048
Quintic	2	193	1272	2060	706	1184	1156	1304	2000	75	562	287	1077	1550
Rana	2	581	1273	428	445	1301	445	491	1641	112	6758	726	1152	974
Rastrigin	2	529	1268	1278	453	902	1240	278	1902	106	5477	402	1712	857
Ripple01	2	215	1242	2013	693	953	591	208	1997	159	1759	1064	2008	1650
Ripple25	2	368	1271	1130	483	695	2019	216	1954	157	2187	1136	1408	1928
Rosenbrock	2	53	694	2013	854	863	717	506	1824	86	943	2676	552	1516
RosenbrockModified	2	1357	725	2000	608	831	971	655	1834	89	7674	2968	1884	1372
RotatedEllipse01	2	10	1324	505	612	875	1514	476	1999	72	2215	393	344	1035
RotatedEllipse02	2	6	1322	522	575	819	2019	448	1989	75	2035	385	320	1032
Salomon	2	1955	1163	953	411	1106	1554	152	2000	75	3446	340	2006	971
Sargan	2	6	1311	505	505	757	2019	478	1898	100	2159	363	289	1051
Schaffer01	2	1977	1089	542	1019	811	1756	88	1192	93	3862	378	2008	339
Schaffer02	2	1908	1248	618	1210	708	1777	152	942	73	215	2161	1984	369
Schaffer03	2	1839	1291	662	1219	1015	940	90	959	96	5680	2204	1923	334
Schaffer04	2	1878	1315	549	1150	1062	1811	96	1004	92	6097	2362	1988	289
SchmidtVetters	3	24	37	1134	10	741	67	186	20	24	9572	1909	14	286
Schwefel01	2	31	1167	1062	410	539	2019	176	1169	87	2322	372	200	1256
Schwefel02	2	4	950	352	452	375	2019	234	1072	78	1306	461	194	1538
Schwefel04	2	14	1318	917	415	482	2019	282	1305	104	231	396	231	1707
Schwefel06	2	309	1047	2064	770	1220	2019	755	2000	84	4114	440	493	880
Schwefel20	2	171	1261	2061	715	1043	2019	448	2000	93	3131	366	484	850
Schwefel21	2	255	1275	2061	712	1146	2019	476	2000	77	1819	368	479	844
Schwefel22	2	302	1274	2059	1119	1153	2019	562	2000	102	837	382	532	1009
Schwefel26	2	730	1266	428	429	635	1091	423	1232	74	7057	814	541	785
Schwefel36	2	2003	1252	673	468	1708	2019	464	58	104	2159	1071	425	2001
Shekel05	4	253	8054	1048	653	1907	1971	428	1859	313	6822	1051	1432	2000
Shekel07	4	151	8281	1114	636	1907	1971	280	1491	328	7034	924	930	1991
Shekel10	4	201	8242	1104	635	1861	2012	278	1571	324	6963	1027	913	1994
Shubert01	2	629	1195	1145	433	864	1217	227	1932	140	5878	2822	2009	725
Shubert03	2	878	1244	703	398	684	1678	223	1928	137	5904	2859	2010	849
Shubert04	2	634	1232	1088	414	754	1351	216	1873	131	6360	2725	1980	846
SineEnvelope	2	1918	1117	451	266	973	1199	104	1994	92	5153	370	2010	544
SixHumpCamel	2	26	1250	946	268	635	882	232	1380	90	2502	325	366	1746
Sodp	2	11	1030	1195	312	365	2019	134	594	77	2134	300	143	2001
Sphere	2	4	1296	353	360	547	2019	224	1201	78	1176	318	210	1593
Step	2	2032	111	101	282	2001	610	2001	267	87	2027	279	88	989
Stochastic	2	2013	840	2015	1810	1941	935	820	2000	97	89	518	2009	892
StretchedV	2	9	–	574	353	759	573	202	870	89	292	2503	1696	2001
StyblinskiTang	2	79	1242	787	386	658	2019	352	1641	150	2515	388	395	1073
TestTubeHolder	2	927	1286	630	352	952	1356	140	1849	159	7012	1102	1960	945
ThreeHumpCamel	2	34	1247	717	382	585	2019	404	1230	86	2400	319	240	1916
Treccani	2	12	1262	927	395	599	896	235	1275	99	951	333	327	1750
Trefethen	2	2009	1249	1696	729	1091	1288	212	1997	84	1833	321	2007	1039
Trid	6	14	14136	1335	1272	2002	2016	1342	1993	566	7767	2372	986	859
Trigonometric01	2	5	1086	205	351	1210	402	176	442	91	135	861	318	2001
Trigonometric02	2	1942	1264	1955	678	911	2019	421	1994	90	264	378	1699	991
Tripod	2	496	1257	2050	699	1081	2019	402	2000	105	2055	1682	1442	908
Ursem01	2	29	1258	635	219	536	529	236	759	169	2229	348	150	2001
Ursem03	2	1603	1240	1996	532	1008	2019	336	2000	147	5073	535	703	1843
Ursem04	2	128	1259	1267	482	875	2019	304	1984	134	543	317	388	2001
UrsemWaves	2	62	1208	539	413	1218	2019	198	852	174	3375	711	2006	2001
VenterSobiezcczanskiSobieski	2	1325	1277	920	436	936	2019	480	1999	96	4351	369	1446	941
Vincent	2	11	1193	674	371	702	2019	296	1279	163	4013	2178	1263	2001
Watson	6	78	21209	2024	1930	2001	2016	2001	2000	198	7975	4007	1452	1482
Wavy	2	1740	1280	1162	432	789	2019	174	1663	92	6633	462	1859	501
WayburnSeader01	2	40	1292	2019	673	770	1774	566	1848	102	884	1780	649	2001
WayburnSeader02	2	50	1315	2015	703	938	1653	984	1995	–	2632	3027	853	1414
Weierstrass	2	2050	1271	2083	630	1529	2019	250	2000	101	267	296	1195	1082
Whitley	2	1958	1269	1814	679	1147	843	496	2000	80	297	435	2006	1300
Wolfe	3	6	4586	449	940	2001	2018	676	107	148	2837	488	804	2001
XinSheYang01	2	2012	931	2014	1842	1557	1146	1616	1965	96	88	399	2007	1146
XinSheYang02	2	1299	1208	2041	441	909	2019	418	2000	92	4949	359	687	1556
XinSheYang03	2	1909	130	142	147	35	2019	16	1980	90	3034	799	2010	2001
XinSheYang04	2	1609	1249	566	324	1093	2019	170	2000	92	4148	688	1031	1984
Xor	9	51	48195	1700	952	1512	1823	1104	1363	209	8438	1334	1921	1658
YaoLiu04	2	60	1202	2033	593	567	1109	349	1913	80	299	593	553	1632
YaoLiu09	2	545	1252	1268	444	868	1227	278	1910	107	2606	396	1790	908
Zacharov	2	11	1314	716	400	586	2019	290	1335	106	2354	339	221	1721
ZeroSum	2	2169	1212	2042	947	784	704	2001	2000	85	282	835	695	879
Zettl	2	22	1272	1511	429	501	2019	212	1085	93	3940	390	256	1991
Zimmerman	2	1987	1244	2047	1184	1570	2019	672	2000	103	927	459	1424	1152
Zirilli	2	35	1238	763	225	589	660	257	815	79	2325	336	241	1668