Pacific-Design.com

    
Home Index

1. Spark

2. Resource Files

Spark / Resource Files /

Read file from resource directory and build predition model.

val ratings = spark.tFile(this.getClass.getResource("/flat-file.dat").getPath)
                .map(parseRating)
                .toDF()

val Array(training, test) = ratings.randomSplit(Array(0.8, 0.2))

// Build the recommnetation model usil ALS on the training data
val als = new ALS()
            .setMaxIter(5)
            .setRegParam(0.01)
            .setUserCol(userId)
            .setItemCol("movieId")
            .setRatingCol("rating")

// Evaluate the model by comparing the RMSE on the test data
val model = als.fit(training)
val predictions = model.transform(test)
predictions.show(100)