NESTED FOR LOOP IN R MOD

List(best_model = tune.out$best.model, best_params = tune.out$best.parameters) # model's parameters so that we can select the best one

# model object so that we can use it to predict # for each combination of gamma and degree we will retrieve Tune.out = tune(svm, label~., data = train_data, kernel = 'polynomial', Library(e1071) # we have to call the library inside each cluster We are using a polynomial SVM and we want to test out a quadratic polynomial as well as a degree 3 polynomial gamma parallel::detectCores() in your R console gamma, we will use values from 0.005 to 1Дў. For example we are going to do a grid search across two parameters:ДЎ. There are quite a few parameters in SVM that we can optimize and performing a grid search sequentially across each of them is going to take hours. ThatвЂ™s all it takes to run an efficient grid search for AdaBoost.Д®xample 2: Parallel processing for optimizing SVM in RДЇor SVM we will use the e10171 package and use the tune.out() function which performs tuning of our SVM model. StopCluster(cl) # DON'T FORGET TO STOP ALL YOUR CLUSTERS! List(model = mod, metrics = model_results) # metrics to append in to a single row of a dataframe for easy Preds = JOUSBoost::predict.adaboost(mod, X=test_data, type='response')ДЄccuracy = (conf + conf) / length(test_labels)ДЇ1_score = 2 * precision * recall / (precision + recall) # we have a dataset called test_data we will use for that # we will also, inside each of the subprocess calculate our validation metrics Tr_f1_score = 2 * tr_precision * tr_recall / (tr_precision + tr_recall) Tr_accuracy = (tr_conf + tr_conf) / length(train_labels) # we will use that to calculate training metrics # training results, the JOUSBoost model object returns a confusion matrix measured on training data # but we need to return the values we need from each sub-process so # now pretty much the parallel processing is done

NESTED FOR LOOP IN R MOD

Mod = adaboost(X = train_dat, y=train_labels, tree_depth = t, n_rounds =n) We have to call it inside our parallel cluster Library(JOUSBoost) # package for deploying AdaBoost. # then the %dopar% operator actually tells doSNOW to implement the parallel process in a cluster # the key to nested for loop in foreach package is %:% operator which tells creates # I generally return the model as well as its validation metrics so that it is easy for me to # be very sure what you want it to return. # Now the thing to note is that foreach() loop returns list items so you need

RegisterDoSNOW(cl) # register your clusters so that when you deploy them doSNOW knows what to do NumOfClusters parallel::detectCores() in your R consoleД¬l <- makeCluster(numOfClusters) # define your clusters Library(foreach) # package that converts nested for loops into parallel processes and passes it to doSNOW for parallel processing

Library(doSNOW) # package for parallel computing in R # this is going to require building 60 models (length(tree_depth) * length(n_trees)) This sounds like a compute intensive job which are going to optimize using parallel processing.ДЇor our example we are going to vary number of trees from 10 upto 200 (but this could go as much as 1000 or even higher), and we are going to vary tree depth from 1 to 3 (but this could go very high, generally not recommended to higher than 8) tree_depth = c(1,2,3) Nowadays it is not uncommon to see AdaBoost with a much greater tree depth (upto 8!) with a large number of classifiers (>500), however, which combination of tree depth and number of trees gave us the most optimal requires us to compute each combination and observe the training and validation accuracy. The original AdaBoost model, also called AdaBoost.M1 had default value for the 2nd variable (tree depth was 1) and these trees were called stumps (a literal metaphor for a short tree). depth of each individual tree or classifier.Two main parameters for an AdaBoost model are: Example 1: Parallel processing for optimizing AdaBoost in R

Nested for loop in r