SciPy Test Functions Index

This page contains the general index of the benchmark problems used to test different Global Optimization algorithms. It also shows some statistics on the “difficulty” of a multi-modal test problem, based on the average successful minimization across all the Global Optimizers tested in this benchmark exercise. The SciPy Extended test suite is based on the current SciPy Global Optimization benchmarks.

The test suite currently contains:

1. 18 one-dimensional test functions with multiple local/global minima;
2. 235 multivariate problems (where the number of independent variables ranges from 2 to 17), again with multiple local/global minima. I have added about 40 new functions to the standard SciPy benchmarks.

Some of the multivariate problems are well known in the mathematical literature; for the less well-known functions, I have taken their definitions mostly from the following references:

• A Literature Survey of Benchmark Functions For Global Optimization Problems (http://arxiv.org/pdf/1308.4008v1.pdf)
• Test functions for optimization needs (http://www.zsd.ict.pwr.wroc.pl/files/docs/functions.pdf)
• A Numerical Evaluation of Several Stochastic Algorithms on Selected Continuous Global Optimization Test Problems (http://folk.uib.no/ssu029/Pdf_file/Ali05.pdf)
• NIST standard regression dataset (http://www.itl.nist.gov/div898/strd/nls/data/eckerle4.shtml)

Test Functions “Hardness”

The test suite contains a variety of Global Optimization problems, some of them are harder to solve than others, irrespectively of the algorithm chosen to minimize the test function. The following table has been obtained by running all the Global Optimizers available against all the N-D test functions for a collection of 100 random starting points, and then averaging the successful minimizations across all the optimizers.

Table 1.4: SciPy Extended benchmark functions “hardness”

Test Function	N	Overall Success (%)
HappyCat	2	0.15%
Bukin06	2	0.31%
Cola	17	1.00%
CrownedCross	2	7.31%
CrossLegTable	2	7.54%
Meyer	3	8.23%
Paviani	10	9.77%
Thurber	7	9.85%
SineEnvelope	2	13.31%
Peaks	2	14.69%
Trefethen	2	14.77%
Whitley	2	14.92%
NewFunction02	2	15.38%
Mishra04	2	15.38%
Mishra03	2	15.38%
DeVilliersGlasser02	5	15.38%
BiggsExp06	6	15.62%
Zagros	2	18.00%
Xor	9	23.08%
EggHolder	2	23.08%
BiggsExp05	5	26.15%
Zimmerman	2	28.62%
XinSheYang03	2	30.15%
NewFunction01	2	30.77%
Rana	2	30.77%
Osborne	5	30.77%
Salomon	2	31.69%
PowerSum	4	31.92%
Griewank	2	33.62%
Damavandi	2	33.62%
Ripple01	2	34.38%
Stochastic	2	36.38%
Kowalik	4	41.46%
Hougen	5	42.23%
Schaffer01	2	43.38%
Schaffer04	2	46.00%
XinSheYang01	2	47.69%
Ackley04	2	48.85%
Schaffer03	2	51.54%
Schaffer02	2	51.69%
Watson	6	53.54%
Deceptive	2	53.69%
UrsemWaves	2	53.85%
DropWave	2	54.23%
ZeroSum	2	54.85%
Spike	2	55.23%
OddSquare	2	55.54%
RosenbrockModified	2	57.69%
MeyerRoth	3	58.85%
Corana	4	59.15%
Colville	4	59.31%
Weierstrass	2	60.69%
Ratkowsky01	4	60.92%
Easom	2	61.00%
DeVilliersGlasser01	4	61.54%
Simpleton	10	61.54%
XinSheYang04	2	63.46%
Gulf	3	63.69%
Price02	2	64.69%
Shekel10	4	65.08%
AMGM	2	65.54%
InvertedCosine	2	65.69%
TestTubeHolder	2	66.15%
Eckerle4	3	67.08%
DeJong5	2	67.46%
MultiGaussian	2	67.85%
DeflectedCorrugatedSpring	2	68.77%
Shekel07	4	69.08%
Shekel05	4	69.92%
Ackley01	2	70.00%
Price03	2	70.77%
Chichinadze	2	71.15%
Levy05	2	71.92%
Penalty02	2	72.23%
Crescent	2	72.62%
Tripod	2	73.31%
BiggsExp04	4	73.85%
GramacyLee03	2	74.38%
Shubert03	2	75.38%
Wavy	2	75.46%
Ursem03	2	75.69%
Dolan	5	75.77%
XinSheYang02	2	76.00%
Pathological	2	76.15%
Rastrigin	2	76.69%
Alpine02	2	76.85%
SchmidtVetters	3	76.92%
Mishra05	2	76.92%
CarromTable	2	76.92%
Hansen	2	78.00%
Branin02	2	79.08%
BuecheRastrigin	2	79.08%
Ripple25	2	79.08%
Mishra06	2	79.08%
YaoLiu09	2	79.54%
Schwefel26	2	79.54%
Trid	6	80.08%
LunacekBiRastrigin	2	80.77%
Powell	4	81.46%
Tsoulos	2	82.69%
Mishra10	2	82.77%
VenterSobiezcczanskiSobieski	2	82.85%
Trigonometric02	2	82.92%
Sinusoidal	2	82.92%
SphericalSinc	2	83.00%
Gear	4	83.00%
Shubert01	2	83.69%
Deb03	2	83.69%
Shubert04	2	84.23%
F2	2	84.46%
HolderTable01	2	84.46%
HolderTable02	2	84.62%
Schwefel36	2	84.62%
PenHolder	2	84.62%
GramacyLee02	2	84.62%
FreudensteinRoth	2	85.85%
CrossInTray	2	86.46%
Plateau	2	86.77%
Step01	2	86.85%
Quintic	2	87.08%
ElAttarVidyasagarDutta	2	87.08%
Step02	2	87.23%
Langermann	2	87.31%
Schwefel06	2	87.69%
Pinter	2	88.00%
Vincent	2	88.23%
EggCrate	2	88.46%
Step03	2	89.00%
Bird	2	90.00%
Penalty01	2	90.69%
Mishra09	3	90.85%
Deb01	2	90.92%
StyblinskiTang	2	91.15%
Trigonometric01	2	91.38%
Schwefel21	2	91.85%
Hartmann6	6	92.00%
Schwefel22	2	92.15%
Schwefel20	2	92.23%
MieleCantrell	4	92.31%
RosenbrockDisc	2	92.31%
LennardJones	6	92.31%
Friedman	5	92.31%
Ackley02	2	92.54%
YaoLiu04	2	92.62%
Alpine01	2	92.85%
Levy13	2	93.08%
Ursem04	2	93.08%
Ackley03	2	93.15%
SawtoothXY	2	93.31%
Ratkowsky02	3	93.31%
DeckkersAarts	2	93.54%
HelicalValley	3	93.62%
Engvall	2	93.77%
BoxBetts	3	94.15%
Bohachevsky02	2	94.23%
Michalewicz	2	94.23%
HyperGrid	2	94.69%
Bohachevsky01	2	95.00%
Decanomial	2	95.38%
BartelsConn	2	95.46%
MullerBrown	2	95.54%
Mishra08	2	95.54%
ThreeHumpCamel	2	95.54%
HimmelBlau	2	95.69%
Parsopoulos	2	95.69%
GoldsteinPrice	2	96.15%
Ursem01	2	96.23%
SixHumpCamel	2	96.38%
Levy03	2	96.46%
Cigar	2	96.54%
Branin01	2	96.62%
Beale	2	96.69%
Judge	2	96.77%
Price04	2	96.85%
Zirilli	2	96.92%
StretchedV	2	97.00%
Picheny	2	97.08%
Bohachevsky03	2	97.15%
Hartmann3	3	97.23%
PermFunction01	2	97.54%
WayburnSeader01	2	97.54%
BiggsExp03	3	97.62%
Leon	2	97.62%
Price01	2	97.77%
McCormick	2	97.85%
Cube	2	97.85%
Kearfott	2	98.00%
Qing	2	98.08%
WayburnSeader02	2	98.38%
Bukin04	2	98.38%
Giunta	2	98.46%
Rosenbrock	2	98.46%
BiggsExp02	2	98.46%
Exp2	2	98.54%
ReduxSum	2	98.62%
Mishra11	2	98.62%
Mishra10b	2	98.69%
WayburnSeader03	2	98.85%
Brent	2	98.92%
Treccani	2	98.92%
PermFunction02	2	99.00%
JennrichSampson	2	99.08%
RotatedEllipse01	2	99.08%
Schwefel04	2	99.15%
RotatedEllipse02	2	99.23%
NeedleEye	2	99.23%
DixonPrice	2	99.31%
Mishra07	2	99.46%
LunacekBiSphere	2	99.46%
Zettl	2	99.54%
TridiagonalMatrix	2	99.54%
Exponential	2	99.62%
Keane	2	99.77%
Schwefel01	2	99.77%
Sargan	2	99.77%
Matyas	2	99.77%
Brown	2	99.85%
Zacharov	2	99.85%
Quadratic	2	99.85%
Sodp	2	99.85%
Sphere	2	99.85%
Schwefel02	2	99.92%
MultiModal	2	99.92%
Csendes	2	99.92%
Bukin02	2	100.00%
Hosaki	2	100.00%
Brad	3	100.00%
Booth	2	100.00%
CosineMixture	2	100.00%
Mishra02	2	100.00%
Mishra01	2	100.00%
Adjiman	2	100.00%
Katsuura	2	100.00%
Wolfe	3	100.00%
Infinity	2	100.00%

Multidimensional Test Functions Index

• N-D Test Functions A
  1. Ackley01
  2. Ackley02
  3. Ackley03
  4. Ackley04
  5. Adjiman
  6. Alpine01
  7. Alpine02
  8. AMGM

---

• N-D Test Functions B
  1. BartelsConn
  2. Beale
  3. BiggsExp02
  4. BiggsExp03
  5. BiggsExp04
  6. BiggsExp05
  7. BiggsExp06
  8. Bird
  9. Bohachevsky01
  10. Bohachevsky02
  11. Bohachevsky03
  12. Booth
  13. BoxBetts
  14. Brad
  15. Branin01
  16. Branin02
  17. Brent
  18. Brown
  19. BuecheRastrigin
  20. Bukin02
  21. Bukin04
  22. Bukin06

---

• N-D Test Functions C
  1. CarromTable
  2. Chichinadze
  3. Cigar
  4. Cola
  5. Colville
  6. Corana
  7. CosineMixture
  8. Crescent
  9. CrossInTray
  10. CrossLegTable
  11. CrownedCross
  12. Csendes
  13. Cube

---

• N-D Test Functions D
  1. Damavandi
  2. Deb01
  3. Deb03
  4. Decanomial
  5. Deceptive
  6. DeckkersAarts
  7. DeflectedCorrugatedSpring
  8. DeJong5
  9. DeVilliersGlasser01
  10. DeVilliersGlasser02
  11. DixonPrice
  12. Dolan
  13. DropWave

---

• N-D Test Functions E
  1. Easom
  2. Eckerle4
  3. EggCrate
  4. EggHolder
  5. ElAttarVidyasagarDutta
  6. Engvall
  7. Exp2
  8. Exponential

---

• N-D Test Functions F
  1. F2
  2. FreudensteinRoth
  3. Friedman

---

• N-D Test Functions G
  1. Gear
  2. Giunta
  3. GoldsteinPrice
  4. GramacyLee02
  5. GramacyLee03
  6. Griewank
  7. Gulf

---

• N-D Test Functions H
  1. Hansen
  2. HappyCat
  3. Hartmann3
  4. Hartmann6
  5. HelicalValley
  6. HimmelBlau
  7. HolderTable01
  8. HolderTable02
  9. Hosaki
  10. Hougen
  11. HyperGrid

---

• N-D Test Functions I
  1. Infinity
  2. InvertedCosine

---

• N-D Test Functions J
  1. JennrichSampson
  2. Judge

---

• N-D Test Functions K
  1. Katsuura
  2. Keane
  3. Kearfott
  4. Kowalik

---

• N-D Test Functions L
  1. Langermann
  2. LennardJones
  3. Leon
  4. Levy03
  5. Levy05
  6. Levy13
  7. LunacekBiRastrigin
  8. LunacekBiSphere

---

• N-D Test Functions M
  1. Matyas
  2. McCormick
  3. Meyer
  4. MeyerRoth
  5. Michalewicz
  6. MieleCantrell
  7. Mishra01
  8. Mishra02
  9. Mishra03
  10. Mishra04
  11. Mishra05
  12. Mishra06
  13. Mishra07
  14. Mishra08
  15. Mishra09
  16. Mishra10
  17. Mishra10b
  18. Mishra11
  19. MullerBrown
  20. MultiGaussian
  21. MultiModal

---

• N-D Test Functions N
  1. NeedleEye
  2. NewFunction01
  3. NewFunction02

---

• N-D Test Functions O
  1. OddSquare
  2. Osborne

---

• N-D Test Functions P
  1. Parsopoulos
  2. Pathological
  3. Paviani
  4. Peaks
  5. Penalty01
  6. Penalty02
  7. PenHolder
  8. PermFunction01
  9. PermFunction02
  10. Picheny
  11. Pinter
  12. Plateau
  13. Powell
  14. PowerSum
  15. Price01
  16. Price02
  17. Price03
  18. Price04

---

• N-D Test Functions Q
  1. Qing
  2. Quadratic
  3. Quintic

---

• N-D Test Functions R
  1. Rana
  2. Rastrigin
  3. Ratkowsky01
  4. Ratkowsky02
  5. ReduxSum
  6. Ripple01
  7. Ripple25
  8. Rosenbrock
  9. RosenbrockDisc
  10. RosenbrockModified
  11. RotatedEllipse01
  12. RotatedEllipse02

---

• N-D Test Functions S
  1. Salomon
  2. Sargan
  3. SawtoothXY
  4. Schaffer01
  5. Schaffer02
  6. Schaffer03
  7. Schaffer04
  8. SchmidtVetters
  9. Schwefel01
  10. Schwefel02
  11. Schwefel04
  12. Schwefel06
  13. Schwefel20
  14. Schwefel21
  15. Schwefel22
  16. Schwefel26
  17. Schwefel36
  18. Shekel05
  19. Shekel07
  20. Shekel10
  21. Shubert01
  22. Shubert03
  23. Shubert04
  24. Simpleton
  25. SineEnvelope
  26. Sinusoidal
  27. SixHumpCamel
  28. Sodp
  29. Sphere
  30. SphericalSinc
  31. Spike
  32. Step01
  33. Step02
  34. Step03
  35. Stochastic
  36. StretchedV
  37. StyblinskiTang

---

• N-D Test Functions T
  1. TestTubeHolder
  2. ThreeHumpCamel
  3. Thurber
  4. Treccani
  5. Trefethen
  6. Trid
  7. TridiagonalMatrix
  8. Trigonometric01
  9. Trigonometric02
  10. Tripod
  11. Tsoulos

---

• N-D Test Functions U
  1. Ursem01
  2. Ursem03
  3. Ursem04
  4. UrsemWaves

---

• N-D Test Functions V
  1. VenterSobiezcczanskiSobieski
  2. Vincent

---

• N-D Test Functions W
  1. Watson
  2. Wavy
  3. WayburnSeader01
  4. WayburnSeader02
  5. WayburnSeader03
  6. Weierstrass
  7. Whitley
  8. Wolfe

---

• N-D Test Functions X
  1. XinSheYang01
  2. XinSheYang02
  3. XinSheYang03
  4. XinSheYang04
  5. Xor

---

• N-D Test Functions Y
  1. YaoLiu04
  2. YaoLiu09

---

• N-D Test Functions Z
  1. Zacharov
  2. Zagros
  3. ZeroSum
  4. Zettl
  5. Zimmerman
  6. Zirilli