Cross Valdiation
Methods to find the optimal number of neurons via cross validation
CausalELM.CrossValidation
— ModuleMethods to perform cross validation and find the optimum number of neurons.
To reduce computation time, the number of neurons is optimized by using cross validation to estimate the validation error on a small subset of the range of possible numbers of neurons. Then, an Extreme Learning Machine is trained to predict validation loss from the given cross validation sets. Finally, the number of neurons is selected that has the smallest predicted loss or the highest classification metric.
CausalELM.CrossValidation.recode
— Functionrecode(ŷ)
Round predicted values to their predicted class for classification tasks.
If the smallest predicted label is 0, all labels are shifted up 1; if the smallest label is -1, all labels are shifted up 2. Also labels cannot be smaller than -1.
Examples
julia> recode([-0.7, 0.2, 1.1])
3-element Vector{Float64}
1
2
3
julia> recode([0.1, 0.2, 0.3])
3-element Vector{Float64}
1
1
1
julia> recode([1.1, 1.51, 1.8])
3-element Vector{Float64}
1
2
2
CausalELM.CrossValidation.traintest
— Functiontraintest(X, Y, folds)
Create a train-test split.
If an iteration is specified, the train test split will be treated as time series/panel data.
Examples
julia> xtrain, ytrain, xtest, ytest = traintest(zeros(20, 2), zeros(20), 5)
traintest(X, Y, folds, iteration)
Create a rolling train-test split for time series/panel data.
An iteration should not be specified for non-time series/panel data.
Examples
julia> xtrain, ytrain, xtest, ytest = traintest(zeros(20, 2), zeros(20), 5, 1)
CausalELM.CrossValidation.validate
— Functionvalidate(X, Y, nodes, metric, iteration...; activation, regularized, folds)
Calculate a validation metric for a single fold in k-fold cross validation.
Examples
julia> x = rand(100, 5); y = Float64.(rand(100) .> 0.5)
julia> validate(x, y, 5, accuracy, 3)
0.0
CausalELM.CrossValidation.crossvalidate
— Functioncrossvalidate(X, Y, neurons, metric, activation, regularized, folds)
Calculate a validation metric for k folds using a single set of hyperparameters.
Examples
julia> x = rand(100, 5); y = Float64.(rand(100) .> 0.5)
julia> crossvalidate(x, y, 5, accuracy)
0.0257841765251021
CausalELM.CrossValidation.bestsize
— Functionbestsize(X, Y, metric, task, activation, min_neurons, max_neurons, regularized, folds, temporal,
iterations, approximator_neurons)
Compute the best number of neurons for an Extreme Learning Machine.
The procedure tests networks with numbers of neurons in a sequence whose length is given by iterations on the interval [minneurons, maxneurons]. Then, it uses the networks sizes and validation errors from the sequence to predict the validation error or metric for every network size between minneurons and maxneurons using the function approximation ability of an Extreme Learning Machine. Finally, it returns the network size with the best predicted validation error or metric.
Examples
julia> bestsize(rand(100, 5), rand(100), mse, "regression")
11