Comment on page
Gradient-Boosted Trees
Note: Make sure you have your training and test data already vectorized and ready to go before you begin trying to fit the machine learning model to unprepped data.
from pyspark.ml.regression import GBTRegressor
from pyspark.ml.tuning import ParamGridBuilder, CrossValidator
from pyspark.ml.evaluation import RegressionEvaluator
gb = GBTRegressor(labelCol="label", featuresCol="features")
gbparamGrid = (ParamGridBuilder()
.addGrid(gb.maxDepth, [2, 5, 10])
.addGrid(gb.maxBins, [10, 20, 40])
.addGrid(gb.maxIter, [5, 10, 20])
.build())
gbevaluator = RegressionEvaluator(predictionCol="prediction", labelCol="label", metricName="rmse")
# Create 5-fold CrossValidator
gbcv = CrossValidator(estimator = gb,
estimatorParamMaps = gbparamGrid,
evaluator = gbevaluator,
numFolds = 5)
gbcvModel = gbcv.fit(train)
print(gbcvModel)
gbpredictions = gbcvModel.transform(test)
print('RMSE:', gbevaluator.evaluate(gbpredictions))
Note: When you use the
CrossValidator
function to set up cross-validation of your models, the resulting model object will have all the runs included, but will only use the best model when you interact with the model object using other functions like evaluate
or transform
.Last modified 4yr ago