ML-Flex

ML-Flex uses machine-learning algorithms to derive models from independent variables, with the purpose of predicting the values of a dependent (class) variable. For example, machine-learning algorithms have long been applied to the Iris data set, introduced by Sir Ronald Fisher in 1936, which contains four independent variables (sepal length, sepal width, petal length, petal width) and one dependent variable (species of Iris flowers = setosa, versicolor, or virginica). Deriving prediction models from the four independent variables, machine-learning algorithms can often differentiate between the species with near-perfect accuracy.

One important aspect to consider in performing a machine-learning experiment is the validation strategy. With the wrong kind of validation approach, biases can be introduced, and it may appear that an algorithm has more predictive ability than it has. Cross validation is a commonly used validation strategy that can help avoid such biases.

In cross validation, the data instances are partitioned into "k" number of groups; in turn, each group is held separate ("test" instances), the algorithm derives a model using the remaining "training" instances, and the model is applied to the test instances. The algorithm's performance is evaluated by how well the predictions for the test instances coincide with the actual values being predicted.

A common value for "k" is 10. Another variation is to use "nested" cross-validation within each training set. In this approach, some form of cross validation is used to optimize the model before it is applied to the "outer" test set. Going a step further, many studies also repeat cross validation multiple times on the same data set. This allows them to assess the robustness of their findings as data instances are assigned differently (at random) to folds.