package vw
- Alphabetic
- Public
- All
Type Members
- class ContextualBanditMetrics extends Serializable
- class ExampleStack extends AnyRef
-
trait
HasAdditionalFeatures extends Params
VW support multiple input columns which are mapped to namespaces.
VW support multiple input columns which are mapped to namespaces. Note: when one wants to create quadratic features within VW you'd specify additionalFeatures. Each feature column is treated as one namespace. Using -q 'uc' for columns 'user' and 'content' you'd get all quadratics for features in user/content (the first letter is called feature group and VW users are used to it... before somebody starts complaining ;)
-
trait
HasNumBits extends Params
Controls hashing parameters such us number of bits (numbits) and how to handle collisions.
-
trait
HasSumCollisions extends Params
Controls hashing parameters such us number of bits (numbits) and how to handle collisions.
-
final
case class
KahanSum(sum: Double = 0, c: Double = 0) extends Product with Serializable
Aggregator state.
Aggregator state.
- sum
the accumulator
- c
a running compensation for lost low-order bits
- Note
cannot use generics for double vs float due to perf: https://stackoverflow.com/questions/4753629/how-do-i-make-a-class-generic-for-all-numeric-types
-
class
KahanSumAggregator extends Aggregator[Float, KahanSum, Float] with Serializable with SynapseMLLogging
Implementation of Kahan summation (https://en.wikipedia.org/wiki/Kahan_summation_algorithm)
Implementation of Kahan summation (https://en.wikipedia.org/wiki/Kahan_summation_algorithm)
Kahan-Babushka-Neumaier sum aggregator make sure lots of small numbers are accumulated numerically stable.
- case class NamespaceInfo(hash: Int, featureGroup: Char, colIdx: Int) extends Product with Serializable
- case class PathAndData(path: String, bytes: Array[Byte]) extends Product with Serializable
-
abstract
class
PredictionBuffer extends Serializable
Collects predictions.
- class PredictionBufferDiscard extends PredictionBuffer
- class PredictionBufferKeep extends PredictionBuffer
- case class TrainContext(vw: VowpalWabbitNative, synchronizationSchedule: VowpalWabbitSyncSchedule, predictionBuffer: PredictionBuffer = new PredictionBufferDiscard, collectOneStepAheadPrediction: Boolean = false, contextualBanditMetrics: ContextualBanditMetrics = new ContextualBanditMetrics, totalTime: StopWatch = new StopWatch, nativeIngestTime: StopWatch = new StopWatch, learnTime: StopWatch = new StopWatch, multipassTime: StopWatch = new StopWatch) extends Product with Serializable
- case class TrainingResult(model: Option[Array[Byte]], stats: TrainingStats) extends Product with Serializable
- case class TrainingStats(partitionId: Int, arguments: String, learningRate: Double, powerT: Double, hashSeed: Int, numBits: Int, numberOfExamplesPerPass: Long, weightedExampleSum: Double, weightedLabelSum: Double, averageLoss: Double, bestConstant: Float, bestConstantLoss: Float, totalNumberOfFeatures: Long, timeTotalNs: Long, timeNativeIngestNs: Long, timeLearnNs: Long, timeMultipassNs: Long, ipsEstimate: Double, snipsEstimate: Double) extends Product with Serializable
-
class
VectorZipper extends Transformer with HasInputCols with HasOutputCol with Wrappable with ComplexParamsWritable with SynapseMLLogging
Combine one or more input columns into a sequence in the output column.
- trait VowpalWabbitBase extends Wrappable with ComplexParamsWritable with Logging
-
trait
VowpalWabbitBaseLearner extends VowpalWabbitBase
Base implementation of VowpalWabbit learners.
Base implementation of VowpalWabbit learners.
- Note
parameters that regularly are swept through are exposed as proper parameters.
-
trait
VowpalWabbitBaseModel extends Params
Base trait to wrap the model for prediction.
-
trait
VowpalWabbitBaseModelSpark extends VowpalWabbitBaseModel with HasFeaturesCol with HasRawPredictionCol with HasAdditionalFeatures
Base implementation for models that use Spark-based features and result in Spark vectors
-
trait
VowpalWabbitBaseProgressive extends Transformer with VowpalWabbitBase
VowpalWabbit is focused on online learning.
VowpalWabbit is focused on online learning. In these settings it is common to operate in progressive validation using a 1-step ahead approach. By default when VW learns from data, it first computes the prediction without learning and then updates the model from the new data. This is especially useful in bandit scenarios as one wants to avoid cheating - as in "knowing the future".
-
trait
VowpalWabbitBaseSpark extends VowpalWabbitBaseLearner with HasLabelCol with HasWeightCol with HasFeaturesCol with HasAdditionalFeatures
* Base implementation for estimators that use Spark-based features (e.g.
* Base implementation for estimators that use Spark-based features (e.g. SparkML vectors)
-
class
VowpalWabbitCSETransformer extends Transformer with SynapseMLLogging with Wrappable with ComplexParamsWritable
Emits continuous success experimentation metrics for contextual bandit style predictions and logs.
- class VowpalWabbitClassificationModel extends ProbabilisticClassificationModel[Row, VowpalWabbitClassificationModel] with VowpalWabbitBaseModelSpark with ComplexParamsWritable with Wrappable with SynapseMLLogging
-
class
VowpalWabbitClassifier extends ProbabilisticClassifier[Row, VowpalWabbitClassifier, VowpalWabbitClassificationModel] with VowpalWabbitBaseSpark with ComplexParamsWritable with SynapseMLLogging
VowpalWabbit exposed as SparkML classifier.
-
class
VowpalWabbitClusterUtil extends AnyRef
Helper to spin up spanning tree coordinator for AllReduce.
- class VowpalWabbitContextualBandit extends Predictor[Row, VowpalWabbitContextualBandit, VowpalWabbitContextualBanditModel] with VowpalWabbitContextualBanditBase with HasParallelismInjected with ComplexParamsWritable with SynapseMLLogging
- trait VowpalWabbitContextualBanditBase extends VowpalWabbitBaseSpark
- class VowpalWabbitContextualBanditModel extends PredictionModel[Row, VowpalWabbitContextualBanditModel] with VowpalWabbitBaseModelSpark with VowpalWabbitContextualBanditBase with ComplexParamsWritable with SynapseMLLogging
- class VowpalWabbitDSJsonTransformer extends Transformer with SynapseMLLogging with Wrappable with ComplexParamsWritable
-
class
VowpalWabbitFeaturizer extends Transformer with HasInputCols with HasOutputCol with HasNumBits with HasSumCollisions with Wrappable with ComplexParamsWritable with SynapseMLLogging
Exposes VW-style featurizer using hashing to SparkML eco system.
-
class
VowpalWabbitGeneric extends Estimator[VowpalWabbitGenericModel] with VowpalWabbitBaseLearner with HasInputCol with SynapseMLLogging
This Estimator supports driving VW using VW-style examples (e.g.
This Estimator supports driving VW using VW-style examples (e.g. 0 |a b c)
- class VowpalWabbitGenericModel extends Model[VowpalWabbitGenericModel] with VowpalWabbitBaseModel with HasInputCol with ComplexParamsWritable with Wrappable with SynapseMLLogging
-
class
VowpalWabbitGenericProgressive extends Transformer with VowpalWabbitBaseProgressive with HasInputCol with SynapseMLLogging
VW-style string based input implementation of online learning with progressive (1-step ahead) output.
-
class
VowpalWabbitInteractions extends Transformer with HasInputCols with HasOutputCol with HasNumBits with HasSumCollisions with Wrappable with ComplexParamsWritable with SynapseMLLogging
This transformer is not intended to be used with VW classifier or regressor, but rather to bring sparse interaction concept to other SparkML learners (e.g.
This transformer is not intended to be used with VW classifier or regressor, but rather to bring sparse interaction concept to other SparkML learners (e.g. LR).
-
class
VowpalWabbitMurmurWithPrefix extends Serializable
VW style murmur hash with pre-hashing of an initially specified prefix.
- class VowpalWabbitRegressionModel extends RegressionModel[Row, VowpalWabbitRegressionModel] with VowpalWabbitBaseModelSpark with ComplexParamsWritable with Wrappable with SynapseMLLogging
-
class
VowpalWabbitRegressor extends BaseRegressor[Row, VowpalWabbitRegressor, VowpalWabbitRegressionModel] with VowpalWabbitBaseSpark with ComplexParamsWritable with SynapseMLLogging
VowpalWabbit exposed as SparkML regressor.
-
trait
VowpalWabbitSyncSchedule extends Serializable
Defines when VW needs to synchronize across partitions.
- class VowpalWabbitSyncScheduleDisabled extends VowpalWabbitSyncSchedule
-
class
VowpalWabbitSyncScheduleSplits extends VowpalWabbitSyncSchedule
Row-count based splitting.
Value Members
- object KahanSum extends Serializable
- object PredictionBuffer extends Serializable
- object TrainingStats extends Serializable
- object VectorUtils
- object VectorZipper extends ComplexParamsReadable[VectorZipper] with Serializable
- object VowpalWabbitCSETransformer extends ComplexParamsReadable[VowpalWabbitCSETransformer] with Serializable
- object VowpalWabbitClassificationModel extends ComplexParamsReadable[VowpalWabbitClassificationModel] with Serializable
- object VowpalWabbitClassifier extends ComplexParamsReadable[VowpalWabbitClassifier] with Serializable
- object VowpalWabbitClusterUtil
- object VowpalWabbitContextualBandit extends ComplexParamsReadable[VowpalWabbitContextualBandit] with Serializable
- object VowpalWabbitContextualBanditModel extends ComplexParamsReadable[VowpalWabbitContextualBanditModel] with Serializable
- object VowpalWabbitDSJsonTransformer extends ComplexParamsReadable[VowpalWabbitDSJsonTransformer] with Serializable
- object VowpalWabbitFeaturizer extends ComplexParamsReadable[VowpalWabbitFeaturizer] with Serializable
- object VowpalWabbitGeneric extends ComplexParamsReadable[VowpalWabbitGeneric] with Serializable
- object VowpalWabbitGenericModel extends ComplexParamsReadable[VowpalWabbitGenericModel] with Serializable
- object VowpalWabbitGenericProgressive extends ComplexParamsReadable[VowpalWabbitGenericProgressive] with Serializable
- object VowpalWabbitInteractions extends ComplexParamsReadable[VowpalWabbitInteractions] with Serializable
-
object
VowpalWabbitPrediction
Provide schemas and accessor functions for almost all VW prediction types.
- object VowpalWabbitRegressionModel extends ComplexParamsReadable[VowpalWabbitRegressionModel] with Serializable
- object VowpalWabbitRegressor extends ComplexParamsReadable[VowpalWabbitRegressor] with Serializable
- object VowpalWabbitSchema
- object VowpalWabbitSyncSchedule extends Serializable
- object VowpalWabbitUtil