You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
This repository was archived by the owner on Dec 15, 2025. It is now read-only.
I wondered why the sparkbench/graph resource nweight-user-features is not loaded as a resource. The resource is packaged together with the .jar anyway, so I'd propose something like this:
NWeightDataGenerator.scala
[...]
defloadModel(modelPath: String, sc: SparkContext, partitions: Int):MatrixFactorizationModel= {
getClass.getResourceAsStream(modelPath) match {
casenull=>thrownewFileNotFoundException(modelPath)
case stream =>valin=newDataInputStream(stream)
valweights=newArray[(Int, Double)](MAX_ID)
for (i <- weights.indices) {
valw= in.readFloat()
weights(i) = (i, w)
}
in.close()
valuserFeatures= sc.parallelize(weights, math.max(400, partitions)).map { case (i, w) =>
(i, Array(w))
}
valuserFeaturesPartitioned= userFeatures.partitionBy(newHashPartitioner(partitions))
userFeaturesPartitioned.cache()
// Model matrix is symmetric, so productFeatures is the same with userFeaturesnewMatrixFactorizationModel(1, userFeaturesPartitioned, userFeaturesPartitioned)
}
}
[...]