LightGBMRanker

Companion object LightGBMRanker

class LightGBMRanker extends Ranker[Vector, LightGBMRanker, LightGBMRankerModel] with LightGBMBase[LightGBMRankerModel] with SynapseMLLogging

Trains a LightGBMRanker model, a fast, distributed, high performance gradient boosting framework based on decision tree algorithms. For more information please see here: https://github.com/Microsoft/LightGBM. For parameter information see here: https://github.com/Microsoft/LightGBM/blob/master/docs/Parameters.rst

Linear Supertypes

LightGBMBase[LightGBMRankerModel], SynapseMLLogging, LightGBMPerformance, LightGBMParams, LightGBMCategoricalParams, LightGBMSeedParams, LightGBMObjectiveParams, LightGBMPredictionParams, LightGBMDartParams, LightGBMDatasetParams, LightGBMLearnerParams, LightGBMBinParams, LightGBMFractionParams, LightGBMSlotParams, LightGBMExecutionParams, HasInitScoreCol, HasValidationIndicatorCol, HasWeightCol, DefaultParamsWritable, MLWritable, Wrappable, DotnetWrappable, RWrappable, PythonWrappable, BaseWrappable, Ranker[Vector, LightGBMRanker, LightGBMRankerModel], HasGroupCol, Predictor[Vector, LightGBMRanker, LightGBMRankerModel], PredictorParams, HasPredictionCol, HasFeaturesCol, HasLabelCol, Estimator[LightGBMRankerModel], PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any

Ordering

Alphabetic
By Inheritance

Inherited

LightGBMRanker
LightGBMBase
SynapseMLLogging
LightGBMPerformance
LightGBMParams
LightGBMCategoricalParams
LightGBMSeedParams
LightGBMObjectiveParams
LightGBMPredictionParams
LightGBMDartParams
LightGBMDatasetParams
LightGBMLearnerParams
LightGBMBinParams
LightGBMFractionParams
LightGBMSlotParams
LightGBMExecutionParams
HasInitScoreCol
HasValidationIndicatorCol
HasWeightCol
DefaultParamsWritable
MLWritable
Wrappable
DotnetWrappable
RWrappable
PythonWrappable
BaseWrappable
Ranker
HasGroupCol
Predictor
PredictorParams
HasPredictionCol
HasFeaturesCol
HasLabelCol
Estimator
PipelineStage
Logging
Params
Serializable
Serializable
Identifiable
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Instance Constructors

new LightGBMRanker()
new LightGBMRanker(uid: String)
uid
The unique ID.

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def $[T](param: Param[T]): T

Attributes
protected
Definition Classes
Params
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def addCustomTrainParams(params: BaseTrainParams, dataset: Dataset[_]): BaseTrainParams

Attributes
protected
Definition Classes
LightGBMBase
def afterTrainBatch(batchIndex: Int, dataset: Dataset[_], model: LightGBMRankerModel): Unit

Definition Classes
LightGBMBase
final def asInstanceOf[T0]: T0

Definition Classes
Any
val baggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val baggingFreq: IntParam

Definition Classes
LightGBMParams
val baggingSeed: IntParam

Definition Classes
LightGBMSeedParams
def beforeTrainBatch(batchIndex: Int, dataset: Dataset[_], model: Option[LightGBMRankerModel]): Unit

Definition Classes
LightGBMBase
val binSampleCount: IntParam

Definition Classes
LightGBMBinParams
val boostFromAverage: BooleanParam

Definition Classes
LightGBMParams
val boostingType: Param[String]

Definition Classes
LightGBMParams
def calculateColumnStatistics(dataframe: DataFrame, measures: InstrumentationMeasures): (Int, Int)
Extract column counts from the dataset.
Extract column counts from the dataset.
dataframe
The dataset to train on.
returns
The number of feature columns and initial score classes

Attributes
protected
Definition Classes
LightGBMBase
def calculateRowStatistics(dataframe: DataFrame, trainingParams: BaseTrainParams, numCols: Int, measures: InstrumentationMeasures): (Array[Row], Array[Long])
Inner train method for LightGBM learners.
Inner train method for LightGBM learners. Calculates the number of workers, creates a driver thread, and runs mapPartitions on the dataset.
dataframe
The dataset to train on.
trainingParams
The training parameters.
numCols
The number of feature columns.
returns
The serialized Dataset reference and an array of partition counts.

Attributes
protected
Definition Classes
LightGBMBase
def castColumns(dataset: Dataset[_], trainingCols: Array[(String, Seq[DataType])]): DataFrame

Attributes
protected
Definition Classes
LightGBMBase
val catSmooth: DoubleParam

Definition Classes
LightGBMCategoricalParams
val categoricalSlotIndexes: IntArrayParam

Definition Classes
LightGBMSlotParams
val categoricalSlotNames: StringArrayParam

Definition Classes
LightGBMSlotParams
val catl2: DoubleParam

Definition Classes
LightGBMCategoricalParams
val chunkSize: IntParam

Definition Classes
LightGBMExecutionParams
lazy val classNameHelper: String

Attributes
protected
Definition Classes
BaseWrappable
final def clear(param: Param[_]): LightGBMRanker.this.type

Definition Classes
Params
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
def companionModelClassName: String

Attributes
protected
Definition Classes
BaseWrappable
def copy(extra: ParamMap): LightGBMRanker

Definition Classes
LightGBMRanker → Predictor → Estimator → PipelineStage → Params
def copyValues[T <: Params](to: T, extra: ParamMap): T

Attributes
protected
Definition Classes
Params
lazy val copyrightLines: String

Attributes
protected
Definition Classes
BaseWrappable
val dataRandomSeed: IntParam

Definition Classes
LightGBMSeedParams
final def defaultCopy[T <: Params](extra: ParamMap): T

Attributes
protected
Definition Classes
Params
val defaultListenPort: IntParam

Definition Classes
LightGBMExecutionParams
val delegate: Option[LightGBMDelegate]

Definition Classes
LightGBMParams
val deterministic: BooleanParam

Definition Classes
LightGBMSeedParams
def dotnetAdditionalMethods: String

Definition Classes
DotnetWrappable
def dotnetClass(): String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassName: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassNameString: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassWrapperName: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetCopyrightLines: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetExtraEstimatorImports: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetExtraMethods: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetInternalWrapper: Boolean

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetMLReadWriteMethods: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetNamespace: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetObjectBaseClass: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamGetter(p: Param[_]): String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamGetters: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamSetter(p: Param[_]): String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamSetters: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetWrapAsTypeMethod: String

Attributes
protected
Definition Classes
DotnetWrappable
val driverListenPort: IntParam

Definition Classes
LightGBMExecutionParams
val dropRate: DoubleParam

Definition Classes
LightGBMDartParams
val dropSeed: IntParam

Definition Classes
LightGBMSeedParams
val earlyStoppingRound: IntParam

Definition Classes
LightGBMLearnerParams
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
val evalAt: IntArrayParam
def executePartitionTasks(ctx: TrainingContext, dataframe: DataFrame, measures: InstrumentationMeasures): LightGBMBooster

Attributes
protected
Definition Classes
LightGBMBase
def executeTraining(dataframe: DataFrame, validationData: Option[Broadcast[Array[Row]]], broadcastedSampleData: Option[Broadcast[Array[Row]]], partitionCounts: Option[Array[Long]], trainParams: BaseTrainParams, numCols: Int, numInitValueClasses: Int, batchIndex: Int, numTasks: Int, numTasksPerExecutor: Int, measures: InstrumentationMeasures): LightGBMRankerModel
Run a parallel job via map partitions to initialize the native library and network, translate the data to the LightGBM in-memory representation and train the models.
Run a parallel job via map partitions to initialize the native library and network, translate the data to the LightGBM in-memory representation and train the models.
dataframe
The dataset to train on.
validationData
The dataset to use as validation. (optional)
broadcastedSampleData
Sample data to use for streaming mode Dataset creation (optional).
partitionCounts
The count per partition for streaming mode (optional).
trainParams
Training parameters.
numCols
Number of columns.
numInitValueClasses
Number of classes for initial values (used only for multiclass).
batchIndex
In running in batch training mode, gets the batch number.
numTasks
Number of tasks/partitions.
numTasksPerExecutor
Number of tasks per executor.
measures
Instrumentation measures to populate.
returns
The LightGBM Model from the trained LightGBM Booster.

Attributes
protected
Definition Classes
LightGBMBase
val executionMode: Param[String]

Definition Classes
LightGBMExecutionParams
def explainParam(param: Param[_]): String

Definition Classes
Params
def explainParams(): String

Definition Classes
Params
val extraSeed: IntParam

Definition Classes
LightGBMSeedParams
def extractInstances(dataset: Dataset[_], validateInstance: (Instance) ⇒ Unit): RDD[Instance]

Attributes
protected
Definition Classes
PredictorParams
def extractInstances(dataset: Dataset[_]): RDD[Instance]

Attributes
protected
Definition Classes
PredictorParams
def extractLabeledPoints(dataset: Dataset[_]): RDD[LabeledPoint]

Attributes
protected
Definition Classes
Predictor
final def extractParamMap(): ParamMap

Definition Classes
Params
final def extractParamMap(extra: ParamMap): ParamMap

Definition Classes
Params
val featureFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val featureFractionByNode: DoubleParam

Definition Classes
LightGBMFractionParams
val featureFractionSeed: IntParam

Definition Classes
LightGBMSeedParams
final val featuresCol: Param[String]

Definition Classes
HasFeaturesCol
val featuresShapCol: Param[String]

Definition Classes
LightGBMPredictionParams
def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
def fit(dataset: Dataset[_]): LightGBMRankerModel

Definition Classes
Predictor → Estimator
def fit(dataset: Dataset[_], paramMaps: Seq[ParamMap]): Seq[LightGBMRankerModel]

Definition Classes
Estimator
Annotations
@Since( "2.0.0" )
def fit(dataset: Dataset[_], paramMap: ParamMap): LightGBMRankerModel

Definition Classes
Estimator
Annotations
@Since( "2.0.0" )
def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): LightGBMRankerModel

Definition Classes
Estimator
Annotations
@Since( "2.0.0" ) @varargs()
val fobj: FObjParam

Definition Classes
LightGBMObjectiveParams
final def get[T](param: Param[T]): Option[T]

Definition Classes
Params
def getAllPerformanceMeasures: Option[Array[InstrumentationMeasures]]

Definition Classes
LightGBMPerformance
def getBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getBaggingFreq: Int

Definition Classes
LightGBMParams
def getBaggingSeed: Int

Definition Classes
LightGBMSeedParams
def getBinSampleCount: Int

Definition Classes
LightGBMBinParams
def getBoostFromAverage: Boolean

Definition Classes
LightGBMParams
def getBoostingType: String

Definition Classes
LightGBMParams
def getCatSmooth: Double

Definition Classes
LightGBMCategoricalParams
def getCategoricalIndexes(featuresSchema: StructField): Array[Int]
Retrieves the categorical indexes in the features column.
Retrieves the categorical indexes in the features column.
featuresSchema
The schema of the features column
returns
the categorical indexes in the features column.

Attributes
protected
Definition Classes
LightGBMBase
def getCategoricalParams: CategoricalParams
Constructs the CategoricalParams.
Constructs the CategoricalParams.
returns
CategoricalParams object containing the parameters related to LightGBM categorical features.

Attributes
protected
Definition Classes
LightGBMBase
def getCategoricalSlotIndexes: Array[Int]

Definition Classes
LightGBMSlotParams
def getCategoricalSlotNames: Array[String]

Definition Classes
LightGBMSlotParams
def getCatl2: Double

Definition Classes
LightGBMCategoricalParams
def getChunkSize: Int

Definition Classes
LightGBMExecutionParams
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
def getColumnParams: ColumnParams
Constructs the ColumnParams.
Constructs the ColumnParams.
returns
ColumnParams object containing the parameters related to LightGBM columns.

Attributes
protected
Definition Classes
LightGBMBase
def getDartParams: DartModeParams
Constructs the DartModeParams
Constructs the DartModeParams
returns
DartModeParams object containing parameters related to dart mode.

Attributes
protected
Definition Classes
LightGBMBase
def getDataRandomSeed: Int

Definition Classes
LightGBMSeedParams
def getDatasetCreationParams(categoricalIndexes: Array[Int], numThreads: Int): String

Definition Classes
LightGBMBase
def getDatasetParams: DatasetParams
Constructs the DatasetParams.
Constructs the DatasetParams.
returns
DatasetParams object containing parameters related to LightGBM Dataset parameters.

Attributes
protected
Definition Classes
LightGBMBase
final def getDefault[T](param: Param[T]): Option[T]

Definition Classes
Params
def getDefaultListenPort: Int

Definition Classes
LightGBMExecutionParams
def getDelegate: Option[LightGBMDelegate]

Definition Classes
LightGBMParams
def getDeterministic: Boolean

Definition Classes
LightGBMSeedParams
def getDriverListenPort: Int

Definition Classes
LightGBMExecutionParams
def getDropRate: Double

Definition Classes
LightGBMDartParams
def getDropSeed: Int

Definition Classes
LightGBMSeedParams
def getEarlyStoppingRound: Int

Definition Classes
LightGBMLearnerParams
def getEvalAt: Array[Int]
def getExecutionMode: String

Definition Classes
LightGBMExecutionParams
def getExecutionParams(numTasksPerExec: Int): ExecutionParams
Constructs the ExecutionParams.
Constructs the ExecutionParams.
returns
ExecutionParams object containing parameters related to LightGBM execution.

Attributes
protected
Definition Classes
LightGBMBase
def getExtraSeed: Int

Definition Classes
LightGBMSeedParams
def getFObj: FObjTrait

Definition Classes
LightGBMObjectiveParams
def getFeatureFraction: Double

Definition Classes
LightGBMFractionParams
def getFeatureFractionByNode: Double

Definition Classes
LightGBMFractionParams
def getFeatureFractionSeed: Int

Definition Classes
LightGBMSeedParams
final def getFeaturesCol: String

Definition Classes
HasFeaturesCol
def getFeaturesShapCol: String

Definition Classes
LightGBMPredictionParams
def getGeneralParams(numTasks: Int, featuresSchema: StructField): GeneralParams
Constructs the GeneralParams.
Constructs the GeneralParams.
returns
GeneralParams object containing parameters related to general LightGBM parameters.

Attributes
protected
Definition Classes
LightGBMBase
def getGroupCol: String

Definition Classes
HasGroupCol
def getImprovementTolerance: Double

Definition Classes
LightGBMLearnerParams
def getInitScoreCol: String

Definition Classes
HasInitScoreCol
def getIsEnableSparse: Boolean

Definition Classes
LightGBMDatasetParams
def getIsProvideTrainingMetric: Boolean

Definition Classes
LightGBMParams
final def getLabelCol: String

Definition Classes
HasLabelCol
def getLabelGain: Array[Double]
def getLambdaL1: Double

Definition Classes
LightGBMParams
def getLambdaL2: Double

Definition Classes
LightGBMParams
def getLeafPredictionCol: String

Definition Classes
LightGBMPredictionParams
def getLearningRate: Double

Definition Classes
LightGBMParams
def getMatrixType: String

Definition Classes
LightGBMExecutionParams
def getMaxBin: Int

Definition Classes
LightGBMBinParams
def getMaxBinByFeature: Array[Int]

Definition Classes
LightGBMParams
def getMaxCatThreshold: Int

Definition Classes
LightGBMCategoricalParams
def getMaxCatToOnehot: Int

Definition Classes
LightGBMCategoricalParams
def getMaxDeltaStep: Double

Definition Classes
LightGBMParams
def getMaxDepth: Int

Definition Classes
LightGBMParams
def getMaxDrop: Int

Definition Classes
LightGBMDartParams
def getMaxNumClasses: Int

Definition Classes
LightGBMParams
def getMaxPosition: Int
def getMaxStreamingOMPThreads: Int

Definition Classes
LightGBMExecutionParams
def getMetric: String

Definition Classes
LightGBMParams
def getMicroBatchSize: Int

Definition Classes
LightGBMExecutionParams
def getMinDataInLeaf: Int

Definition Classes
LightGBMParams
def getMinDataPerBin: Int

Definition Classes
LightGBMParams
def getMinDataPerGroup: Int

Definition Classes
LightGBMCategoricalParams
def getMinGainToSplit: Double

Definition Classes
LightGBMParams
def getMinSumHessianInLeaf: Double

Definition Classes
LightGBMParams
def getModel(trainParams: BaseTrainParams, lightGBMBooster: LightGBMBooster): LightGBMRankerModel
Gets the trained model given the train parameters and booster.
Gets the trained model given the train parameters and booster.
returns
trained model.

Definition Classes
LightGBMRanker → LightGBMBase
def getModelString: String

Definition Classes
LightGBMParams
def getMonotoneConstraints: Array[Int]

Definition Classes
LightGBMLearnerParams
def getMonotoneConstraintsMethod: String

Definition Classes
LightGBMLearnerParams
def getMonotonePenalty: Double

Definition Classes
LightGBMLearnerParams
def getNegBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getNumBatches: Int

Definition Classes
LightGBMExecutionParams
def getNumIterations: Int

Definition Classes
LightGBMParams
def getNumLeaves: Int

Definition Classes
LightGBMParams
def getNumTasks: Int

Definition Classes
LightGBMExecutionParams
def getNumThreads: Int

Definition Classes
LightGBMExecutionParams
def getObjective: String

Definition Classes
LightGBMObjectiveParams
def getObjectiveParams: ObjectiveParams
Constructs the ObjectiveParams.
Constructs the ObjectiveParams.
returns
ObjectiveParams object containing parameters related to the objective function.

Attributes
protected
Definition Classes
LightGBMBase
def getObjectiveSeed: Int

Definition Classes
LightGBMSeedParams
def getOptGroupCol: Option[String]
Optional group column for Ranking, set to None by default.
Optional group column for Ranking, set to None by default.
returns
None

Definition Classes
LightGBMRanker → LightGBMBase
final def getOrDefault[T](param: Param[T]): T

Definition Classes
Params
def getOtherRate: Double

Definition Classes
LightGBMLearnerParams
def getParallelism: String

Definition Classes
LightGBMExecutionParams
def getParam(paramName: String): Param[Any]

Definition Classes
Params
def getParamInfo(p: Param[_]): ParamInfo[_]

Definition Classes
BaseWrappable
def getPassThroughArgs: String

Definition Classes
LightGBMExecutionParams
def getPerformanceMeasures: Option[InstrumentationMeasures]
In the common case of 1 batch, there is only 1 measure, so this is a convenience method.
In the common case of 1 batch, there is only 1 measure, so this is a convenience method.

Definition Classes
LightGBMPerformance
def getPosBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getPredictDisableShapeCheck: Boolean

Definition Classes
LightGBMPredictionParams
final def getPredictionCol: String

Definition Classes
HasPredictionCol
def getRepartitionByGroupingColumn: Boolean

Definition Classes
LightGBMExecutionParams
def getSeed: Int

Definition Classes
LightGBMSeedParams
def getSeedParams: SeedParams
Constructs the SeedParams.
Constructs the SeedParams.
returns
SeedParams object containing the parameters related to LightGBM seeds and determinism.

Attributes
protected
Definition Classes
LightGBMBase
def getSkipDrop: Double

Definition Classes
LightGBMDartParams
def getSlotNames: Array[String]

Definition Classes
LightGBMSlotParams
def getSlotNamesWithMetadata(featuresSchema: StructField): Option[Array[String]]

Definition Classes
LightGBMBase
def getTimeout: Double

Definition Classes
LightGBMExecutionParams
def getTopK: Int

Definition Classes
LightGBMExecutionParams
def getTopRate: Double

Definition Classes
LightGBMLearnerParams
def getTrainParams(numTasks: Int, featuresSchema: StructField, numTasksPerExec: Int): BaseTrainParams
Gets the training parameters.
Gets the training parameters.
numTasks
The total number of tasks.
featuresSchema
The features column schema.
numTasksPerExec
The number of tasks per executor.
returns
train parameters.

Definition Classes
LightGBMRanker → LightGBMBase
def getTrainingCols: Array[(String, Seq[DataType])]

Attributes
protected
Definition Classes
LightGBMBase
def getTrainingContext(dataframe: DataFrame, validationData: Option[Broadcast[Array[Row]]], broadcastedSampleData: Option[Broadcast[Array[Row]]], partitionCounts: Option[Array[Long]], trainParams: BaseTrainParams, numCols: Int, numInitValueClasses: Int, batchIndex: Int, numTasksPerExecutor: Int, networkManager: NetworkManager): TrainingContext
Get the object that holds all relevant context information for the training session.
Get the object that holds all relevant context information for the training session.
dataframe
The dataset to train on.
validationData
The dataset to use as validation. (optional)
broadcastedSampleData
Sample data to use for streaming mode Dataset creation (optional).
partitionCounts
The count per partition for streaming mode (optional).
trainParams
Training parameters.
numCols
Number of columns.
numInitValueClasses
Number of classes for initial values (used only for multiclass).
batchIndex
In running in batch training mode, gets the batch number.
numTasksPerExecutor
Number of tasks per executor.
networkManager
The network manager.
returns
The context of the training session.

Attributes
protected
Definition Classes
LightGBMBase
def getUniformDrop: Boolean

Definition Classes
LightGBMDartParams
def getUseBarrierExecutionMode: Boolean

Definition Classes
LightGBMExecutionParams
def getUseMissing: Boolean

Definition Classes
LightGBMDatasetParams
def getUseSingleDatasetMode: Boolean

Definition Classes
LightGBMExecutionParams
def getValidationIndicatorCol: String

Definition Classes
HasValidationIndicatorCol
def getVerbosity: Int

Definition Classes
LightGBMParams
def getWeightCol: String

Definition Classes
HasWeightCol
def getXGBoostDartMode: Boolean

Definition Classes
LightGBMDartParams
def getZeroAsMissing: Boolean

Definition Classes
LightGBMDatasetParams
val groupCol: Param[String]
The name of the group column
The name of the group column

Definition Classes
HasGroupCol
final def hasDefault[T](param: Param[T]): Boolean

Definition Classes
Params
def hasParam(paramName: String): Boolean

Definition Classes
Params
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
val improvementTolerance: DoubleParam

Definition Classes
LightGBMLearnerParams
def initPerformanceMeasures(batchCount: Int): Unit

Attributes
protected
Definition Classes
LightGBMPerformance
val initScoreCol: Param[String]
The name of the initial score column
The name of the initial score column

Definition Classes
HasInitScoreCol
def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

Attributes
protected
Definition Classes
Logging
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
final def isDefined(param: Param[_]): Boolean

Definition Classes
Params
val isEnableSparse: BooleanParam

Definition Classes
LightGBMDatasetParams
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val isProvideTrainingMetric: BooleanParam

Definition Classes
LightGBMParams
final def isSet(param: Param[_]): Boolean

Definition Classes
Params
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
final val labelCol: Param[String]

Definition Classes
HasLabelCol
val labelGain: DoubleArrayParam
val lambdaL1: DoubleParam

Definition Classes
LightGBMParams
val lambdaL2: DoubleParam

Definition Classes
LightGBMParams
val leafPredictionCol: Param[String]

Definition Classes
LightGBMPredictionParams
val learningRate: DoubleParam

Definition Classes
LightGBMParams
def log: Logger

Attributes
protected
Definition Classes
Logging
def logBase(info: SynapseMLLogInfo): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logBase(methodName: String, columns: Option[Int]): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logClass(): Unit

Definition Classes
SynapseMLLogging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logErrorBase(methodName: String, e: Exception): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logFit[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logTrain[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logTransform[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logVerb[T](verb: String, f: ⇒ T, columns: Int = -1): T

Definition Classes
SynapseMLLogging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def makeDotnetFile(conf: CodegenConfig): Unit

Definition Classes
DotnetWrappable
def makePyFile(conf: CodegenConfig): Unit

Definition Classes
PythonWrappable
def makeRFile(conf: CodegenConfig): Unit

Definition Classes
RWrappable
val matrixType: Param[String]

Definition Classes
LightGBMExecutionParams
val maxBin: IntParam

Definition Classes
LightGBMBinParams
val maxBinByFeature: IntArrayParam

Definition Classes
LightGBMParams
val maxCatThreshold: IntParam

Definition Classes
LightGBMCategoricalParams
val maxCatToOnehot: IntParam

Definition Classes
LightGBMCategoricalParams
val maxDeltaStep: DoubleParam

Definition Classes
LightGBMParams
val maxDepth: IntParam

Definition Classes
LightGBMParams
val maxDrop: IntParam

Definition Classes
LightGBMDartParams
val maxNumClasses: IntParam

Definition Classes
LightGBMParams
val maxPosition: IntParam
val maxStreamingOMPThreads: IntParam

Definition Classes
LightGBMExecutionParams
val metric: Param[String]

Definition Classes
LightGBMParams
val microBatchSize: IntParam

Definition Classes
LightGBMExecutionParams
val minDataInLeaf: IntParam

Definition Classes
LightGBMParams
val minDataPerBin: IntParam

Definition Classes
LightGBMParams
val minDataPerGroup: IntParam

Definition Classes
LightGBMCategoricalParams
val minGainToSplit: DoubleParam

Definition Classes
LightGBMParams
val minSumHessianInLeaf: DoubleParam

Definition Classes
LightGBMParams
val modelString: Param[String]

Definition Classes
LightGBMParams
val monotoneConstraints: IntArrayParam

Definition Classes
LightGBMLearnerParams
val monotoneConstraintsMethod: Param[String]

Definition Classes
LightGBMLearnerParams
val monotonePenalty: DoubleParam

Definition Classes
LightGBMLearnerParams
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
val negBaggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
val numBatches: IntParam

Definition Classes
LightGBMExecutionParams
val numIterations: IntParam

Definition Classes
LightGBMParams
val numLeaves: IntParam

Definition Classes
LightGBMParams
val numTasks: IntParam

Definition Classes
LightGBMExecutionParams
val numThreads: IntParam

Definition Classes
LightGBMExecutionParams
val objective: Param[String]

Definition Classes
LightGBMObjectiveParams
val objectiveSeed: IntParam

Definition Classes
LightGBMSeedParams
val otherRate: DoubleParam

Definition Classes
LightGBMLearnerParams
val parallelism: Param[String]

Definition Classes
LightGBMExecutionParams
lazy val params: Array[Param[_]]

Definition Classes
Params
val passThroughArgs: Param[String]

Definition Classes
LightGBMExecutionParams
val posBaggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val predictDisableShapeCheck: BooleanParam

Definition Classes
LightGBMPredictionParams
final val predictionCol: Param[String]

Definition Classes
HasPredictionCol
def prepareDataframe(dataset: Dataset[_], numTasks: Int): DataFrame

Definition Classes
LightGBMRanker → LightGBMBase
def preprocessData(df: DataFrame): DataFrame
For Ranking, we need to sort the data within partitions by group prior to training to ensure training succeeds.
For Ranking, we need to sort the data within partitions by group prior to training to ensure training succeeds.
df
The data frame to preprocess prior to training.
returns
The preprocessed data, sorted within partition by group.

Definition Classes
LightGBMRanker → LightGBMBase
def pyAdditionalMethods: String

Definition Classes
PythonWrappable
lazy val pyClassDoc: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyClassName: String

Attributes
protected
Definition Classes
PythonWrappable
def pyExtraEstimatorImports: String

Attributes
protected
Definition Classes
PythonWrappable
def pyExtraEstimatorMethods: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyInheritedClasses: Seq[String]

Attributes
protected
Definition Classes
PythonWrappable
def pyInitFunc(): String

Definition Classes
PythonWrappable
lazy val pyInternalWrapper: Boolean

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyObjectBaseClass: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamArg[T](p: Param[T]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamDefault[T](p: Param[T]): Option[String]

Attributes
protected
Definition Classes
PythonWrappable
def pyParamGetter(p: Param[_]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamSetter(p: Param[_]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsArgs: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsDefaults: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyParamsDefinitions: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsGetters: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsSetters: String

Attributes
protected
Definition Classes
PythonWrappable
def pythonClass(): String

Attributes
protected
Definition Classes
PythonWrappable
def rClass(): String

Attributes
protected
Definition Classes
RWrappable
def rDocString: String

Attributes
protected
Definition Classes
RWrappable
def rExtraBodyLines: String

Attributes
protected
Definition Classes
RWrappable
def rExtraInitLines: String

Attributes
protected
Definition Classes
RWrappable
lazy val rFuncName: String

Attributes
protected
Definition Classes
RWrappable
lazy val rInternalWrapper: Boolean

Attributes
protected
Definition Classes
RWrappable
def rParamArg[T](p: Param[T]): String

Attributes
protected
Definition Classes
RWrappable
def rParamsArgs: String

Attributes
protected
Definition Classes
RWrappable
def rSetterLines: String

Attributes
protected
Definition Classes
RWrappable
val repartitionByGroupingColumn: BooleanParam

Definition Classes
LightGBMExecutionParams
def save(path: String): Unit

Definition Classes
MLWritable
Annotations
@Since( "1.6.0" ) @throws( ... )
val seed: IntParam

Definition Classes
LightGBMSeedParams
final def set(paramPair: ParamPair[_]): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def set(param: String, value: Any): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def set[T](param: Param[T], value: T): LightGBMRanker.this.type

Definition Classes
Params
def setBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setBaggingFreq(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setBaggingSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setBatchPerformanceMeasure(index: Int, measures: InstrumentationMeasures): Unit

Attributes
protected
Definition Classes
LightGBMPerformance
def setBatchPerformanceMeasures(measures: Array[Option[InstrumentationMeasures]]): LightGBMRanker.this.type

Attributes
protected
Definition Classes
LightGBMPerformance
def setBinSampleCount(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMBinParams
def setBoostFromAverage(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setBoostingType(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setCatSmooth(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setCategoricalSlotIndexes(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setCategoricalSlotNames(value: Array[String]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setCatl2(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setChunkSize(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDataRandomSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
final def setDefault(paramPairs: ParamPair[_]*): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def setDefault[T](param: Param[T], value: T): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
def setDefaultListenPort(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDelegate(delegate: LightGBMDelegate): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setDeterministic(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setDriverListenPort(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDropRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setDropSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setEarlyStoppingRound(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setEvalAt(value: Array[Int]): LightGBMRanker.this.type
def setExecutionMode(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setExtraSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setFObj(value: FObjTrait): LightGBMRanker.this.type

Definition Classes
LightGBMObjectiveParams
def setFeatureFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setFeatureFractionByNode(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setFeatureFractionSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setFeaturesCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setFeaturesShapCol(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setGroupCol(value: String): LightGBMRanker.this.type

Definition Classes
HasGroupCol
def setImprovementTolerance(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setInitScoreCol(value: String): LightGBMRanker.this.type

Definition Classes
HasInitScoreCol
def setIsEnableSparse(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
def setIsProvideTrainingMetric(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLabelCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setLabelGain(value: Array[Double]): LightGBMRanker.this.type
def setLambdaL1(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLambdaL2(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLeafPredictionCol(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setLearningRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMatrixType(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMaxBin(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMBinParams
def setMaxBinByFeature(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxCatThreshold(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMaxCatToOnehot(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMaxDeltaStep(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxDepth(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxDrop(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setMaxNumClasses(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxPosition(value: Int): LightGBMRanker.this.type
def setMaxStreamingOMPThreads(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMetric(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMicroBatchSize(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMinDataInLeaf(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinDataPerBin(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinDataPerGroup(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMinGainToSplit(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinSumHessianInLeaf(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setModelString(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMonotoneConstraints(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setMonotoneConstraintsMethod(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setMonotonePenalty(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setNegBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setNumBatches(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setNumIterations(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setNumLeaves(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setNumTasks(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setNumThreads(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setObjective(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMObjectiveParams
def setObjectiveSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setOtherRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setParallelism(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setPassThroughArgs(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setPosBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setPredictDisableShapeCheck(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setPredictionCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setRepartitionByGroupingColumn(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setSkipDrop(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setSlotNames(value: Array[String]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setTimeout(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setTopK(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setTopRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setUniformDrop(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setUseBarrierExecutionMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setUseMissing(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
def setUseSingleDatasetMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setValidationIndicatorCol(value: String): LightGBMRanker.this.type

Definition Classes
HasValidationIndicatorCol
def setVerbosity(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setWeightCol(value: String): LightGBMRanker.this.type

Definition Classes
HasWeightCol
def setXGBoostDartMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setZeroAsMissing(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
val skipDrop: DoubleParam

Definition Classes
LightGBMDartParams
val slotNames: StringArrayParam

Definition Classes
LightGBMSlotParams
def stringFromTrainedModel(model: LightGBMRankerModel): String

Definition Classes
LightGBMRanker → LightGBMBase
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
val thisStage: Params

Attributes
protected
Definition Classes
BaseWrappable
val timeout: DoubleParam

Definition Classes
LightGBMExecutionParams
def toString(): String

Definition Classes
Identifiable → AnyRef → Any
val topK: IntParam

Definition Classes
LightGBMExecutionParams
val topRate: DoubleParam

Definition Classes
LightGBMLearnerParams
def train(dataset: Dataset[_]): LightGBMRankerModel
Trains the LightGBM model.
Trains the LightGBM model. If batches are specified, breaks training dataset into batches for training.
dataset
The input dataset to train.
returns
The trained model.

Attributes
protected
Definition Classes
LightGBMBase
def trainOneDataBatch(dataset: Dataset[_], batchIndex: Int, batchCount: Int): LightGBMRankerModel
Inner train method for LightGBM learners.
Inner train method for LightGBM learners. Calculates the number of workers, creates a driver thread, and runs mapPartitions on the dataset.
dataset
The dataset to train on.
batchIndex
In running in batch training mode, gets the batch number.
returns
The LightGBM Model from the trained LightGBM Booster.

Attributes
protected
Definition Classes
LightGBMBase
def transformSchema(schema: StructType): StructType

Definition Classes
Predictor → PipelineStage
def transformSchema(schema: StructType, logging: Boolean): StructType

Attributes
protected
Definition Classes
PipelineStage
Annotations
@DeveloperApi()
val uid: String

Definition Classes
LightGBMRanker → SynapseMLLogging → Identifiable
val uniformDrop: BooleanParam

Definition Classes
LightGBMDartParams
val useBarrierExecutionMode: BooleanParam

Definition Classes
LightGBMExecutionParams
val useMissing: BooleanParam

Definition Classes
LightGBMDatasetParams
val useSingleDatasetMode: BooleanParam

Definition Classes
LightGBMExecutionParams
def validateAndTransformSchema(schema: StructType, fitting: Boolean, featuresDataType: DataType): StructType

Attributes
protected
Definition Classes
PredictorParams
val validationIndicatorCol: Param[String]
The name of the validation indicator column
The name of the validation indicator column

Definition Classes
HasValidationIndicatorCol
val verbosity: IntParam

Definition Classes
LightGBMParams
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
val weightCol: Param[String]
The name of the weight column
The name of the weight column

Definition Classes
HasWeightCol
def write: MLWriter

Definition Classes
DefaultParamsWritable → MLWritable
val xGBoostDartMode: BooleanParam

Definition Classes
LightGBMDartParams
val zeroAsMissing: BooleanParam

Definition Classes
LightGBMDatasetParams

Inherited from LightGBMBase[LightGBMRankerModel]

Value Members

def addCustomTrainParams(params: BaseTrainParams, dataset: Dataset[_]): BaseTrainParams

Attributes
protected
Definition Classes
LightGBMBase
def afterTrainBatch(batchIndex: Int, dataset: Dataset[_], model: LightGBMRankerModel): Unit

Definition Classes
LightGBMBase
def beforeTrainBatch(batchIndex: Int, dataset: Dataset[_], model: Option[LightGBMRankerModel]): Unit

Definition Classes
LightGBMBase
def calculateColumnStatistics(dataframe: DataFrame, measures: InstrumentationMeasures): (Int, Int)
Extract column counts from the dataset.
Extract column counts from the dataset.
dataframe
The dataset to train on.
returns
The number of feature columns and initial score classes

Attributes
protected
Definition Classes
LightGBMBase
def calculateRowStatistics(dataframe: DataFrame, trainingParams: BaseTrainParams, numCols: Int, measures: InstrumentationMeasures): (Array[Row], Array[Long])
Inner train method for LightGBM learners.
Inner train method for LightGBM learners. Calculates the number of workers, creates a driver thread, and runs mapPartitions on the dataset.
dataframe
The dataset to train on.
trainingParams
The training parameters.
numCols
The number of feature columns.
returns
The serialized Dataset reference and an array of partition counts.

Attributes
protected
Definition Classes
LightGBMBase
def castColumns(dataset: Dataset[_], trainingCols: Array[(String, Seq[DataType])]): DataFrame

Attributes
protected
Definition Classes
LightGBMBase
def executePartitionTasks(ctx: TrainingContext, dataframe: DataFrame, measures: InstrumentationMeasures): LightGBMBooster

Attributes
protected
Definition Classes
LightGBMBase
def executeTraining(dataframe: DataFrame, validationData: Option[Broadcast[Array[Row]]], broadcastedSampleData: Option[Broadcast[Array[Row]]], partitionCounts: Option[Array[Long]], trainParams: BaseTrainParams, numCols: Int, numInitValueClasses: Int, batchIndex: Int, numTasks: Int, numTasksPerExecutor: Int, measures: InstrumentationMeasures): LightGBMRankerModel
Run a parallel job via map partitions to initialize the native library and network, translate the data to the LightGBM in-memory representation and train the models.
Run a parallel job via map partitions to initialize the native library and network, translate the data to the LightGBM in-memory representation and train the models.
dataframe
The dataset to train on.
validationData
The dataset to use as validation. (optional)
broadcastedSampleData
Sample data to use for streaming mode Dataset creation (optional).
partitionCounts
The count per partition for streaming mode (optional).
trainParams
Training parameters.
numCols
Number of columns.
numInitValueClasses
Number of classes for initial values (used only for multiclass).
batchIndex
In running in batch training mode, gets the batch number.
numTasks
Number of tasks/partitions.
numTasksPerExecutor
Number of tasks per executor.
measures
Instrumentation measures to populate.
returns
The LightGBM Model from the trained LightGBM Booster.

Attributes
protected
Definition Classes
LightGBMBase
def getCategoricalIndexes(featuresSchema: StructField): Array[Int]
Retrieves the categorical indexes in the features column.
Retrieves the categorical indexes in the features column.
featuresSchema
The schema of the features column
returns
the categorical indexes in the features column.

Attributes
protected
Definition Classes
LightGBMBase
def getCategoricalParams: CategoricalParams
Constructs the CategoricalParams.
Constructs the CategoricalParams.
returns
CategoricalParams object containing the parameters related to LightGBM categorical features.

Attributes
protected
Definition Classes
LightGBMBase
def getColumnParams: ColumnParams
Constructs the ColumnParams.
Constructs the ColumnParams.
returns
ColumnParams object containing the parameters related to LightGBM columns.

Attributes
protected
Definition Classes
LightGBMBase
def getDartParams: DartModeParams
Constructs the DartModeParams
Constructs the DartModeParams
returns
DartModeParams object containing parameters related to dart mode.

Attributes
protected
Definition Classes
LightGBMBase
def getDatasetCreationParams(categoricalIndexes: Array[Int], numThreads: Int): String

Definition Classes
LightGBMBase
def getDatasetParams: DatasetParams
Constructs the DatasetParams.
Constructs the DatasetParams.
returns
DatasetParams object containing parameters related to LightGBM Dataset parameters.

Attributes
protected
Definition Classes
LightGBMBase
def getExecutionParams(numTasksPerExec: Int): ExecutionParams
Constructs the ExecutionParams.
Constructs the ExecutionParams.
returns
ExecutionParams object containing parameters related to LightGBM execution.

Attributes
protected
Definition Classes
LightGBMBase
def getGeneralParams(numTasks: Int, featuresSchema: StructField): GeneralParams
Constructs the GeneralParams.
Constructs the GeneralParams.
returns
GeneralParams object containing parameters related to general LightGBM parameters.

Attributes
protected
Definition Classes
LightGBMBase
def getObjectiveParams: ObjectiveParams
Constructs the ObjectiveParams.
Constructs the ObjectiveParams.
returns
ObjectiveParams object containing parameters related to the objective function.

Attributes
protected
Definition Classes
LightGBMBase
def getSeedParams: SeedParams
Constructs the SeedParams.
Constructs the SeedParams.
returns
SeedParams object containing the parameters related to LightGBM seeds and determinism.

Attributes
protected
Definition Classes
LightGBMBase
def getSlotNamesWithMetadata(featuresSchema: StructField): Option[Array[String]]

Definition Classes
LightGBMBase
def getTrainingCols: Array[(String, Seq[DataType])]

Attributes
protected
Definition Classes
LightGBMBase
def getTrainingContext(dataframe: DataFrame, validationData: Option[Broadcast[Array[Row]]], broadcastedSampleData: Option[Broadcast[Array[Row]]], partitionCounts: Option[Array[Long]], trainParams: BaseTrainParams, numCols: Int, numInitValueClasses: Int, batchIndex: Int, numTasksPerExecutor: Int, networkManager: NetworkManager): TrainingContext
Get the object that holds all relevant context information for the training session.
Get the object that holds all relevant context information for the training session.
dataframe
The dataset to train on.
validationData
The dataset to use as validation. (optional)
broadcastedSampleData
Sample data to use for streaming mode Dataset creation (optional).
partitionCounts
The count per partition for streaming mode (optional).
trainParams
Training parameters.
numCols
Number of columns.
numInitValueClasses
Number of classes for initial values (used only for multiclass).
batchIndex
In running in batch training mode, gets the batch number.
numTasksPerExecutor
Number of tasks per executor.
networkManager
The network manager.
returns
The context of the training session.

Attributes
protected
Definition Classes
LightGBMBase
def train(dataset: Dataset[_]): LightGBMRankerModel
Trains the LightGBM model.
Trains the LightGBM model. If batches are specified, breaks training dataset into batches for training.
dataset
The input dataset to train.
returns
The trained model.

Attributes
protected
Definition Classes
LightGBMBase
def trainOneDataBatch(dataset: Dataset[_], batchIndex: Int, batchCount: Int): LightGBMRankerModel
Inner train method for LightGBM learners.
Inner train method for LightGBM learners. Calculates the number of workers, creates a driver thread, and runs mapPartitions on the dataset.
dataset
The dataset to train on.
batchIndex
In running in batch training mode, gets the batch number.
returns
The LightGBM Model from the trained LightGBM Booster.

Attributes
protected
Definition Classes
LightGBMBase

Inherited from SynapseMLLogging

Value Members

def logBase(info: SynapseMLLogInfo): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logBase(methodName: String, columns: Option[Int]): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logClass(): Unit

Definition Classes
SynapseMLLogging
def logErrorBase(methodName: String, e: Exception): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logFit[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logTrain[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logTransform[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logVerb[T](verb: String, f: ⇒ T, columns: Int = -1): T

Definition Classes
SynapseMLLogging

Inherited from LightGBMPerformance

Value Members

def getAllPerformanceMeasures: Option[Array[InstrumentationMeasures]]

Definition Classes
LightGBMPerformance
def getPerformanceMeasures: Option[InstrumentationMeasures]
In the common case of 1 batch, there is only 1 measure, so this is a convenience method.
In the common case of 1 batch, there is only 1 measure, so this is a convenience method.

Definition Classes
LightGBMPerformance
def initPerformanceMeasures(batchCount: Int): Unit

Attributes
protected
Definition Classes
LightGBMPerformance
def setBatchPerformanceMeasure(index: Int, measures: InstrumentationMeasures): Unit

Attributes
protected
Definition Classes
LightGBMPerformance
def setBatchPerformanceMeasures(measures: Array[Option[InstrumentationMeasures]]): LightGBMRanker.this.type

Attributes
protected
Definition Classes
LightGBMPerformance

Inherited from LightGBMParams

Value Members

val baggingFreq: IntParam

Definition Classes
LightGBMParams
val boostFromAverage: BooleanParam

Definition Classes
LightGBMParams
val boostingType: Param[String]

Definition Classes
LightGBMParams
val delegate: Option[LightGBMDelegate]

Definition Classes
LightGBMParams
def getBaggingFreq: Int

Definition Classes
LightGBMParams
def getBoostFromAverage: Boolean

Definition Classes
LightGBMParams
def getBoostingType: String

Definition Classes
LightGBMParams
def getDelegate: Option[LightGBMDelegate]

Definition Classes
LightGBMParams
def getIsProvideTrainingMetric: Boolean

Definition Classes
LightGBMParams
def getLambdaL1: Double

Definition Classes
LightGBMParams
def getLambdaL2: Double

Definition Classes
LightGBMParams
def getLearningRate: Double

Definition Classes
LightGBMParams
def getMaxBinByFeature: Array[Int]

Definition Classes
LightGBMParams
def getMaxDeltaStep: Double

Definition Classes
LightGBMParams
def getMaxDepth: Int

Definition Classes
LightGBMParams
def getMaxNumClasses: Int

Definition Classes
LightGBMParams
def getMetric: String

Definition Classes
LightGBMParams
def getMinDataInLeaf: Int

Definition Classes
LightGBMParams
def getMinDataPerBin: Int

Definition Classes
LightGBMParams
def getMinGainToSplit: Double

Definition Classes
LightGBMParams
def getMinSumHessianInLeaf: Double

Definition Classes
LightGBMParams
def getModelString: String

Definition Classes
LightGBMParams
def getNumIterations: Int

Definition Classes
LightGBMParams
def getNumLeaves: Int

Definition Classes
LightGBMParams
def getVerbosity: Int

Definition Classes
LightGBMParams
val isProvideTrainingMetric: BooleanParam

Definition Classes
LightGBMParams
val lambdaL1: DoubleParam

Definition Classes
LightGBMParams
val lambdaL2: DoubleParam

Definition Classes
LightGBMParams
val learningRate: DoubleParam

Definition Classes
LightGBMParams
val maxBinByFeature: IntArrayParam

Definition Classes
LightGBMParams
val maxDeltaStep: DoubleParam

Definition Classes
LightGBMParams
val maxDepth: IntParam

Definition Classes
LightGBMParams
val maxNumClasses: IntParam

Definition Classes
LightGBMParams
val metric: Param[String]

Definition Classes
LightGBMParams
val minDataInLeaf: IntParam

Definition Classes
LightGBMParams
val minDataPerBin: IntParam

Definition Classes
LightGBMParams
val minGainToSplit: DoubleParam

Definition Classes
LightGBMParams
val minSumHessianInLeaf: DoubleParam

Definition Classes
LightGBMParams
val modelString: Param[String]

Definition Classes
LightGBMParams
val numIterations: IntParam

Definition Classes
LightGBMParams
val numLeaves: IntParam

Definition Classes
LightGBMParams
def setBaggingFreq(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setBoostFromAverage(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setBoostingType(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setDelegate(delegate: LightGBMDelegate): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setIsProvideTrainingMetric(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLambdaL1(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLambdaL2(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLearningRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxBinByFeature(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxDeltaStep(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxDepth(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxNumClasses(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMetric(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinDataInLeaf(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinDataPerBin(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinGainToSplit(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinSumHessianInLeaf(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setModelString(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setNumIterations(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setNumLeaves(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setVerbosity(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
val verbosity: IntParam

Definition Classes
LightGBMParams

Inherited from LightGBMCategoricalParams

Value Members

val catSmooth: DoubleParam

Definition Classes
LightGBMCategoricalParams
val catl2: DoubleParam

Definition Classes
LightGBMCategoricalParams
def getCatSmooth: Double

Definition Classes
LightGBMCategoricalParams
def getCatl2: Double

Definition Classes
LightGBMCategoricalParams
def getMaxCatThreshold: Int

Definition Classes
LightGBMCategoricalParams
def getMaxCatToOnehot: Int

Definition Classes
LightGBMCategoricalParams
def getMinDataPerGroup: Int

Definition Classes
LightGBMCategoricalParams
val maxCatThreshold: IntParam

Definition Classes
LightGBMCategoricalParams
val maxCatToOnehot: IntParam

Definition Classes
LightGBMCategoricalParams
val minDataPerGroup: IntParam

Definition Classes
LightGBMCategoricalParams
def setCatSmooth(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setCatl2(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMaxCatThreshold(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMaxCatToOnehot(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMinDataPerGroup(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams

Inherited from LightGBMSeedParams

Value Members

val baggingSeed: IntParam

Definition Classes
LightGBMSeedParams
val dataRandomSeed: IntParam

Definition Classes
LightGBMSeedParams
val deterministic: BooleanParam

Definition Classes
LightGBMSeedParams
val dropSeed: IntParam

Definition Classes
LightGBMSeedParams
val extraSeed: IntParam

Definition Classes
LightGBMSeedParams
val featureFractionSeed: IntParam

Definition Classes
LightGBMSeedParams
def getBaggingSeed: Int

Definition Classes
LightGBMSeedParams
def getDataRandomSeed: Int

Definition Classes
LightGBMSeedParams
def getDeterministic: Boolean

Definition Classes
LightGBMSeedParams
def getDropSeed: Int

Definition Classes
LightGBMSeedParams
def getExtraSeed: Int

Definition Classes
LightGBMSeedParams
def getFeatureFractionSeed: Int

Definition Classes
LightGBMSeedParams
def getObjectiveSeed: Int

Definition Classes
LightGBMSeedParams
def getSeed: Int

Definition Classes
LightGBMSeedParams
val objectiveSeed: IntParam

Definition Classes
LightGBMSeedParams
val seed: IntParam

Definition Classes
LightGBMSeedParams
def setBaggingSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setDataRandomSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setDeterministic(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setDropSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setExtraSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setFeatureFractionSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setObjectiveSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams

Inherited from LightGBMObjectiveParams

Value Members

val fobj: FObjParam

Definition Classes
LightGBMObjectiveParams
def getFObj: FObjTrait

Definition Classes
LightGBMObjectiveParams
def getObjective: String

Definition Classes
LightGBMObjectiveParams
val objective: Param[String]

Definition Classes
LightGBMObjectiveParams
def setFObj(value: FObjTrait): LightGBMRanker.this.type

Definition Classes
LightGBMObjectiveParams
def setObjective(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMObjectiveParams

Inherited from LightGBMPredictionParams

Value Members

val featuresShapCol: Param[String]

Definition Classes
LightGBMPredictionParams
def getFeaturesShapCol: String

Definition Classes
LightGBMPredictionParams
def getLeafPredictionCol: String

Definition Classes
LightGBMPredictionParams
def getPredictDisableShapeCheck: Boolean

Definition Classes
LightGBMPredictionParams
val leafPredictionCol: Param[String]

Definition Classes
LightGBMPredictionParams
val predictDisableShapeCheck: BooleanParam

Definition Classes
LightGBMPredictionParams
def setFeaturesShapCol(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setLeafPredictionCol(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setPredictDisableShapeCheck(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams

Inherited from LightGBMDartParams

Value Members

val dropRate: DoubleParam

Definition Classes
LightGBMDartParams
def getDropRate: Double

Definition Classes
LightGBMDartParams
def getMaxDrop: Int

Definition Classes
LightGBMDartParams
def getSkipDrop: Double

Definition Classes
LightGBMDartParams
def getUniformDrop: Boolean

Definition Classes
LightGBMDartParams
def getXGBoostDartMode: Boolean

Definition Classes
LightGBMDartParams
val maxDrop: IntParam

Definition Classes
LightGBMDartParams
def setDropRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setMaxDrop(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setSkipDrop(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setUniformDrop(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setXGBoostDartMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
val skipDrop: DoubleParam

Definition Classes
LightGBMDartParams
val uniformDrop: BooleanParam

Definition Classes
LightGBMDartParams
val xGBoostDartMode: BooleanParam

Definition Classes
LightGBMDartParams

Inherited from LightGBMDatasetParams

Value Members

def getIsEnableSparse: Boolean

Definition Classes
LightGBMDatasetParams
def getUseMissing: Boolean

Definition Classes
LightGBMDatasetParams
def getZeroAsMissing: Boolean

Definition Classes
LightGBMDatasetParams
val isEnableSparse: BooleanParam

Definition Classes
LightGBMDatasetParams
def setIsEnableSparse(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
def setUseMissing(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
def setZeroAsMissing(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
val useMissing: BooleanParam

Definition Classes
LightGBMDatasetParams
val zeroAsMissing: BooleanParam

Definition Classes
LightGBMDatasetParams

Inherited from LightGBMLearnerParams

Value Members

val earlyStoppingRound: IntParam

Definition Classes
LightGBMLearnerParams
def getEarlyStoppingRound: Int

Definition Classes
LightGBMLearnerParams
def getImprovementTolerance: Double

Definition Classes
LightGBMLearnerParams
def getMonotoneConstraints: Array[Int]

Definition Classes
LightGBMLearnerParams
def getMonotoneConstraintsMethod: String

Definition Classes
LightGBMLearnerParams
def getMonotonePenalty: Double

Definition Classes
LightGBMLearnerParams
def getOtherRate: Double

Definition Classes
LightGBMLearnerParams
def getTopRate: Double

Definition Classes
LightGBMLearnerParams
val improvementTolerance: DoubleParam

Definition Classes
LightGBMLearnerParams
val monotoneConstraints: IntArrayParam

Definition Classes
LightGBMLearnerParams
val monotoneConstraintsMethod: Param[String]

Definition Classes
LightGBMLearnerParams
val monotonePenalty: DoubleParam

Definition Classes
LightGBMLearnerParams
val otherRate: DoubleParam

Definition Classes
LightGBMLearnerParams
def setEarlyStoppingRound(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setImprovementTolerance(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setMonotoneConstraints(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setMonotoneConstraintsMethod(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setMonotonePenalty(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setOtherRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setTopRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
val topRate: DoubleParam

Definition Classes
LightGBMLearnerParams

Inherited from LightGBMBinParams

Value Members

val binSampleCount: IntParam

Definition Classes
LightGBMBinParams
def getBinSampleCount: Int

Definition Classes
LightGBMBinParams
def getMaxBin: Int

Definition Classes
LightGBMBinParams
val maxBin: IntParam

Definition Classes
LightGBMBinParams
def setBinSampleCount(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMBinParams
def setMaxBin(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMBinParams

Inherited from LightGBMFractionParams

Value Members

val baggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val featureFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val featureFractionByNode: DoubleParam

Definition Classes
LightGBMFractionParams
def getBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getFeatureFraction: Double

Definition Classes
LightGBMFractionParams
def getFeatureFractionByNode: Double

Definition Classes
LightGBMFractionParams
def getNegBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getPosBaggingFraction: Double

Definition Classes
LightGBMFractionParams
val negBaggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val posBaggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
def setBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setFeatureFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setFeatureFractionByNode(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setNegBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setPosBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams

Inherited from LightGBMSlotParams

Value Members

val categoricalSlotIndexes: IntArrayParam

Definition Classes
LightGBMSlotParams
val categoricalSlotNames: StringArrayParam

Definition Classes
LightGBMSlotParams
def getCategoricalSlotIndexes: Array[Int]

Definition Classes
LightGBMSlotParams
def getCategoricalSlotNames: Array[String]

Definition Classes
LightGBMSlotParams
def getSlotNames: Array[String]

Definition Classes
LightGBMSlotParams
def setCategoricalSlotIndexes(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setCategoricalSlotNames(value: Array[String]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setSlotNames(value: Array[String]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
val slotNames: StringArrayParam

Definition Classes
LightGBMSlotParams

Inherited from LightGBMExecutionParams

Value Members

val chunkSize: IntParam

Definition Classes
LightGBMExecutionParams
val defaultListenPort: IntParam

Definition Classes
LightGBMExecutionParams
val driverListenPort: IntParam

Definition Classes
LightGBMExecutionParams
val executionMode: Param[String]

Definition Classes
LightGBMExecutionParams
def getChunkSize: Int

Definition Classes
LightGBMExecutionParams
def getDefaultListenPort: Int

Definition Classes
LightGBMExecutionParams
def getDriverListenPort: Int

Definition Classes
LightGBMExecutionParams
def getExecutionMode: String

Definition Classes
LightGBMExecutionParams
def getMatrixType: String

Definition Classes
LightGBMExecutionParams
def getMaxStreamingOMPThreads: Int

Definition Classes
LightGBMExecutionParams
def getMicroBatchSize: Int

Definition Classes
LightGBMExecutionParams
def getNumBatches: Int

Definition Classes
LightGBMExecutionParams
def getNumTasks: Int

Definition Classes
LightGBMExecutionParams
def getNumThreads: Int

Definition Classes
LightGBMExecutionParams
def getParallelism: String

Definition Classes
LightGBMExecutionParams
def getPassThroughArgs: String

Definition Classes
LightGBMExecutionParams
def getRepartitionByGroupingColumn: Boolean

Definition Classes
LightGBMExecutionParams
def getTimeout: Double

Definition Classes
LightGBMExecutionParams
def getTopK: Int

Definition Classes
LightGBMExecutionParams
def getUseBarrierExecutionMode: Boolean

Definition Classes
LightGBMExecutionParams
def getUseSingleDatasetMode: Boolean

Definition Classes
LightGBMExecutionParams
val matrixType: Param[String]

Definition Classes
LightGBMExecutionParams
val maxStreamingOMPThreads: IntParam

Definition Classes
LightGBMExecutionParams
val microBatchSize: IntParam

Definition Classes
LightGBMExecutionParams
val numBatches: IntParam

Definition Classes
LightGBMExecutionParams
val numTasks: IntParam

Definition Classes
LightGBMExecutionParams
val numThreads: IntParam

Definition Classes
LightGBMExecutionParams
val parallelism: Param[String]

Definition Classes
LightGBMExecutionParams
val passThroughArgs: Param[String]

Definition Classes
LightGBMExecutionParams
val repartitionByGroupingColumn: BooleanParam

Definition Classes
LightGBMExecutionParams
def setChunkSize(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDefaultListenPort(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDriverListenPort(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setExecutionMode(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMatrixType(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMaxStreamingOMPThreads(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMicroBatchSize(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setNumBatches(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setNumTasks(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setNumThreads(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setParallelism(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setPassThroughArgs(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setRepartitionByGroupingColumn(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setTimeout(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setTopK(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setUseBarrierExecutionMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setUseSingleDatasetMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
val timeout: DoubleParam

Definition Classes
LightGBMExecutionParams
val topK: IntParam

Definition Classes
LightGBMExecutionParams
val useBarrierExecutionMode: BooleanParam

Definition Classes
LightGBMExecutionParams
val useSingleDatasetMode: BooleanParam

Definition Classes
LightGBMExecutionParams

Inherited from HasInitScoreCol

Value Members

def getInitScoreCol: String

Definition Classes
HasInitScoreCol
val initScoreCol: Param[String]
The name of the initial score column
The name of the initial score column

Definition Classes
HasInitScoreCol
def setInitScoreCol(value: String): LightGBMRanker.this.type

Definition Classes
HasInitScoreCol

Inherited from HasValidationIndicatorCol

Value Members

def getValidationIndicatorCol: String

Definition Classes
HasValidationIndicatorCol
def setValidationIndicatorCol(value: String): LightGBMRanker.this.type

Definition Classes
HasValidationIndicatorCol
val validationIndicatorCol: Param[String]
The name of the validation indicator column
The name of the validation indicator column

Definition Classes
HasValidationIndicatorCol

Inherited from HasWeightCol

Value Members

def getWeightCol: String

Definition Classes
HasWeightCol
def setWeightCol(value: String): LightGBMRanker.this.type

Definition Classes
HasWeightCol
val weightCol: Param[String]
The name of the weight column
The name of the weight column

Definition Classes
HasWeightCol

Inherited from DefaultParamsWritable

Value Members

def write: MLWriter

Definition Classes
DefaultParamsWritable → MLWritable

Inherited from MLWritable

Value Members

def save(path: String): Unit

Definition Classes
MLWritable
Annotations
@Since( "1.6.0" ) @throws( ... )

Inherited from DotnetWrappable

Value Members

def dotnetAdditionalMethods: String

Definition Classes
DotnetWrappable
def dotnetClass(): String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassName: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassNameString: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassWrapperName: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetCopyrightLines: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetExtraEstimatorImports: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetExtraMethods: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetInternalWrapper: Boolean

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetMLReadWriteMethods: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetNamespace: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetObjectBaseClass: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamGetter(p: Param[_]): String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamGetters: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamSetter(p: Param[_]): String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamSetters: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetWrapAsTypeMethod: String

Attributes
protected
Definition Classes
DotnetWrappable
def makeDotnetFile(conf: CodegenConfig): Unit

Definition Classes
DotnetWrappable

Inherited from RWrappable

Value Members

def makeRFile(conf: CodegenConfig): Unit

Definition Classes
RWrappable
def rClass(): String

Attributes
protected
Definition Classes
RWrappable
def rDocString: String

Attributes
protected
Definition Classes
RWrappable
def rExtraBodyLines: String

Attributes
protected
Definition Classes
RWrappable
def rExtraInitLines: String

Attributes
protected
Definition Classes
RWrappable
lazy val rFuncName: String

Attributes
protected
Definition Classes
RWrappable
lazy val rInternalWrapper: Boolean

Attributes
protected
Definition Classes
RWrappable
def rParamArg[T](p: Param[T]): String

Attributes
protected
Definition Classes
RWrappable
def rParamsArgs: String

Attributes
protected
Definition Classes
RWrappable
def rSetterLines: String

Attributes
protected
Definition Classes
RWrappable

Inherited from PythonWrappable

Value Members

def makePyFile(conf: CodegenConfig): Unit

Definition Classes
PythonWrappable
def pyAdditionalMethods: String

Definition Classes
PythonWrappable
lazy val pyClassDoc: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyClassName: String

Attributes
protected
Definition Classes
PythonWrappable
def pyExtraEstimatorImports: String

Attributes
protected
Definition Classes
PythonWrappable
def pyExtraEstimatorMethods: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyInheritedClasses: Seq[String]

Attributes
protected
Definition Classes
PythonWrappable
def pyInitFunc(): String

Definition Classes
PythonWrappable
lazy val pyInternalWrapper: Boolean

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyObjectBaseClass: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamArg[T](p: Param[T]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamDefault[T](p: Param[T]): Option[String]

Attributes
protected
Definition Classes
PythonWrappable
def pyParamGetter(p: Param[_]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamSetter(p: Param[_]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsArgs: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsDefaults: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyParamsDefinitions: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsGetters: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsSetters: String

Attributes
protected
Definition Classes
PythonWrappable
def pythonClass(): String

Attributes
protected
Definition Classes
PythonWrappable

Inherited from BaseWrappable

Value Members

lazy val classNameHelper: String

Attributes
protected
Definition Classes
BaseWrappable
def companionModelClassName: String

Attributes
protected
Definition Classes
BaseWrappable
lazy val copyrightLines: String

Attributes
protected
Definition Classes
BaseWrappable
def getParamInfo(p: Param[_]): ParamInfo[_]

Definition Classes
BaseWrappable
val thisStage: Params

Attributes
protected
Definition Classes
BaseWrappable

Inherited from HasGroupCol

Value Members

def getGroupCol: String

Definition Classes
HasGroupCol
val groupCol: Param[String]
The name of the group column
The name of the group column

Definition Classes
HasGroupCol
def setGroupCol(value: String): LightGBMRanker.this.type

Definition Classes
HasGroupCol

Inherited from Predictor[Vector, LightGBMRanker, LightGBMRankerModel]

Value Members

def extractLabeledPoints(dataset: Dataset[_]): RDD[LabeledPoint]

Attributes
protected
Definition Classes
Predictor
def fit(dataset: Dataset[_]): LightGBMRankerModel

Definition Classes
Predictor → Estimator
def setFeaturesCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setLabelCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setPredictionCol(value: String): LightGBMRanker

Definition Classes
Predictor
def transformSchema(schema: StructType): StructType

Definition Classes
Predictor → PipelineStage

Inherited from PredictorParams

Value Members

def extractInstances(dataset: Dataset[_], validateInstance: (Instance) ⇒ Unit): RDD[Instance]

Attributes
protected
Definition Classes
PredictorParams
def extractInstances(dataset: Dataset[_]): RDD[Instance]

Attributes
protected
Definition Classes
PredictorParams
def validateAndTransformSchema(schema: StructType, fitting: Boolean, featuresDataType: DataType): StructType

Attributes
protected
Definition Classes
PredictorParams

Inherited from HasPredictionCol

Value Members

final def getPredictionCol: String

Definition Classes
HasPredictionCol
final val predictionCol: Param[String]

Definition Classes
HasPredictionCol

Inherited from HasFeaturesCol

Value Members

final val featuresCol: Param[String]

Definition Classes
HasFeaturesCol
final def getFeaturesCol: String

Definition Classes
HasFeaturesCol

Inherited from HasLabelCol

Value Members

final def getLabelCol: String

Definition Classes
HasLabelCol
final val labelCol: Param[String]

Definition Classes
HasLabelCol

Inherited from Estimator[LightGBMRankerModel]

Value Members

def fit(dataset: Dataset[_], paramMaps: Seq[ParamMap]): Seq[LightGBMRankerModel]

Definition Classes
Estimator
Annotations
@Since( "2.0.0" )
def fit(dataset: Dataset[_], paramMap: ParamMap): LightGBMRankerModel

Definition Classes
Estimator
Annotations
@Since( "2.0.0" )
def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): LightGBMRankerModel

Definition Classes
Estimator
Annotations
@Since( "2.0.0" ) @varargs()

Inherited from PipelineStage

Value Members

def transformSchema(schema: StructType, logging: Boolean): StructType

Attributes
protected
Definition Classes
PipelineStage
Annotations
@DeveloperApi()

Inherited from Logging

Value Members

def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

Attributes
protected
Definition Classes
Logging
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
def log: Logger

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging

Inherited from Params

Value Members

final def $[T](param: Param[T]): T

Attributes
protected
Definition Classes
Params
final def clear(param: Param[_]): LightGBMRanker.this.type

Definition Classes
Params
def copyValues[T <: Params](to: T, extra: ParamMap): T

Attributes
protected
Definition Classes
Params
final def defaultCopy[T <: Params](extra: ParamMap): T

Attributes
protected
Definition Classes
Params
def explainParam(param: Param[_]): String

Definition Classes
Params
def explainParams(): String

Definition Classes
Params
final def extractParamMap(): ParamMap

Definition Classes
Params
final def extractParamMap(extra: ParamMap): ParamMap

Definition Classes
Params
final def get[T](param: Param[T]): Option[T]

Definition Classes
Params
final def getDefault[T](param: Param[T]): Option[T]

Definition Classes
Params
final def getOrDefault[T](param: Param[T]): T

Definition Classes
Params
def getParam(paramName: String): Param[Any]

Definition Classes
Params
final def hasDefault[T](param: Param[T]): Boolean

Definition Classes
Params
def hasParam(paramName: String): Boolean

Definition Classes
Params
final def isDefined(param: Param[_]): Boolean

Definition Classes
Params
final def isSet(param: Param[_]): Boolean

Definition Classes
Params
lazy val params: Array[Param[_]]

Definition Classes
Params
final def set(paramPair: ParamPair[_]): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def set(param: String, value: Any): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def set[T](param: Param[T], value: T): LightGBMRanker.this.type

Definition Classes
Params
final def setDefault(paramPairs: ParamPair[_]*): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def setDefault[T](param: Param[T], value: T): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params

Inherited from Identifiable

Value Members

def toString(): String

Definition Classes
Identifiable → AnyRef → Any

Inherited from AnyRef

Value Members

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()

Inherited from Any

Value Members

final def asInstanceOf[T0]: T0

Definition Classes
Any
final def isInstanceOf[T0]: Boolean

Definition Classes
Any

getParam

def getGroupCol: String

Definition Classes
HasGroupCol
def getInitScoreCol: String

Definition Classes
HasInitScoreCol
def getValidationIndicatorCol: String

Definition Classes
HasValidationIndicatorCol
def getWeightCol: String

Definition Classes
HasWeightCol

param

val groupCol: Param[String]
The name of the group column
The name of the group column

Definition Classes
HasGroupCol
val initScoreCol: Param[String]
The name of the initial score column
The name of the initial score column

Definition Classes
HasInitScoreCol
val validationIndicatorCol: Param[String]
The name of the validation indicator column
The name of the validation indicator column

Definition Classes
HasValidationIndicatorCol
val weightCol: Param[String]
The name of the weight column
The name of the weight column

Definition Classes
HasWeightCol

setParam

def setGroupCol(value: String): LightGBMRanker.this.type

Definition Classes
HasGroupCol
def setInitScoreCol(value: String): LightGBMRanker.this.type

Definition Classes
HasInitScoreCol
def setValidationIndicatorCol(value: String): LightGBMRanker.this.type

Definition Classes
HasValidationIndicatorCol
def setWeightCol(value: String): LightGBMRanker.this.type

Definition Classes
HasWeightCol

Ungrouped

final def !=(arg0: Any): Boolean

Definition Classes
AnyRef → Any
final def ##(): Int

Definition Classes
AnyRef → Any
final def $[T](param: Param[T]): T

Attributes
protected
Definition Classes
Params
final def ==(arg0: Any): Boolean

Definition Classes
AnyRef → Any
def addCustomTrainParams(params: BaseTrainParams, dataset: Dataset[_]): BaseTrainParams

Attributes
protected
Definition Classes
LightGBMBase
def afterTrainBatch(batchIndex: Int, dataset: Dataset[_], model: LightGBMRankerModel): Unit

Definition Classes
LightGBMBase
final def asInstanceOf[T0]: T0

Definition Classes
Any
val baggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val baggingFreq: IntParam

Definition Classes
LightGBMParams
val baggingSeed: IntParam

Definition Classes
LightGBMSeedParams
def beforeTrainBatch(batchIndex: Int, dataset: Dataset[_], model: Option[LightGBMRankerModel]): Unit

Definition Classes
LightGBMBase
val binSampleCount: IntParam

Definition Classes
LightGBMBinParams
val boostFromAverage: BooleanParam

Definition Classes
LightGBMParams
val boostingType: Param[String]

Definition Classes
LightGBMParams
def calculateColumnStatistics(dataframe: DataFrame, measures: InstrumentationMeasures): (Int, Int)
Extract column counts from the dataset.
Extract column counts from the dataset.
dataframe
The dataset to train on.
returns
The number of feature columns and initial score classes

Attributes
protected
Definition Classes
LightGBMBase
def calculateRowStatistics(dataframe: DataFrame, trainingParams: BaseTrainParams, numCols: Int, measures: InstrumentationMeasures): (Array[Row], Array[Long])
Inner train method for LightGBM learners.
Inner train method for LightGBM learners. Calculates the number of workers, creates a driver thread, and runs mapPartitions on the dataset.
dataframe
The dataset to train on.
trainingParams
The training parameters.
numCols
The number of feature columns.
returns
The serialized Dataset reference and an array of partition counts.

Attributes
protected
Definition Classes
LightGBMBase
def castColumns(dataset: Dataset[_], trainingCols: Array[(String, Seq[DataType])]): DataFrame

Attributes
protected
Definition Classes
LightGBMBase
val catSmooth: DoubleParam

Definition Classes
LightGBMCategoricalParams
val categoricalSlotIndexes: IntArrayParam

Definition Classes
LightGBMSlotParams
val categoricalSlotNames: StringArrayParam

Definition Classes
LightGBMSlotParams
val catl2: DoubleParam

Definition Classes
LightGBMCategoricalParams
val chunkSize: IntParam

Definition Classes
LightGBMExecutionParams
lazy val classNameHelper: String

Attributes
protected
Definition Classes
BaseWrappable
final def clear(param: Param[_]): LightGBMRanker.this.type

Definition Classes
Params
def clone(): AnyRef

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
def companionModelClassName: String

Attributes
protected
Definition Classes
BaseWrappable
def copy(extra: ParamMap): LightGBMRanker

Definition Classes
LightGBMRanker → Predictor → Estimator → PipelineStage → Params
def copyValues[T <: Params](to: T, extra: ParamMap): T

Attributes
protected
Definition Classes
Params
lazy val copyrightLines: String

Attributes
protected
Definition Classes
BaseWrappable
val dataRandomSeed: IntParam

Definition Classes
LightGBMSeedParams
final def defaultCopy[T <: Params](extra: ParamMap): T

Attributes
protected
Definition Classes
Params
val defaultListenPort: IntParam

Definition Classes
LightGBMExecutionParams
val delegate: Option[LightGBMDelegate]

Definition Classes
LightGBMParams
val deterministic: BooleanParam

Definition Classes
LightGBMSeedParams
def dotnetAdditionalMethods: String

Definition Classes
DotnetWrappable
def dotnetClass(): String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassName: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassNameString: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetClassWrapperName: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetCopyrightLines: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetExtraEstimatorImports: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetExtraMethods: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetInternalWrapper: Boolean

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetMLReadWriteMethods: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetNamespace: String

Attributes
protected
Definition Classes
DotnetWrappable
lazy val dotnetObjectBaseClass: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamGetter(p: Param[_]): String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamGetters: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamSetter(p: Param[_]): String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetParamSetters: String

Attributes
protected
Definition Classes
DotnetWrappable
def dotnetWrapAsTypeMethod: String

Attributes
protected
Definition Classes
DotnetWrappable
val driverListenPort: IntParam

Definition Classes
LightGBMExecutionParams
val dropRate: DoubleParam

Definition Classes
LightGBMDartParams
val dropSeed: IntParam

Definition Classes
LightGBMSeedParams
val earlyStoppingRound: IntParam

Definition Classes
LightGBMLearnerParams
final def eq(arg0: AnyRef): Boolean

Definition Classes
AnyRef
def equals(arg0: Any): Boolean

Definition Classes
AnyRef → Any
val evalAt: IntArrayParam
def executePartitionTasks(ctx: TrainingContext, dataframe: DataFrame, measures: InstrumentationMeasures): LightGBMBooster

Attributes
protected
Definition Classes
LightGBMBase
def executeTraining(dataframe: DataFrame, validationData: Option[Broadcast[Array[Row]]], broadcastedSampleData: Option[Broadcast[Array[Row]]], partitionCounts: Option[Array[Long]], trainParams: BaseTrainParams, numCols: Int, numInitValueClasses: Int, batchIndex: Int, numTasks: Int, numTasksPerExecutor: Int, measures: InstrumentationMeasures): LightGBMRankerModel
Run a parallel job via map partitions to initialize the native library and network, translate the data to the LightGBM in-memory representation and train the models.
Run a parallel job via map partitions to initialize the native library and network, translate the data to the LightGBM in-memory representation and train the models.
dataframe
The dataset to train on.
validationData
The dataset to use as validation. (optional)
broadcastedSampleData
Sample data to use for streaming mode Dataset creation (optional).
partitionCounts
The count per partition for streaming mode (optional).
trainParams
Training parameters.
numCols
Number of columns.
numInitValueClasses
Number of classes for initial values (used only for multiclass).
batchIndex
In running in batch training mode, gets the batch number.
numTasks
Number of tasks/partitions.
numTasksPerExecutor
Number of tasks per executor.
measures
Instrumentation measures to populate.
returns
The LightGBM Model from the trained LightGBM Booster.

Attributes
protected
Definition Classes
LightGBMBase
val executionMode: Param[String]

Definition Classes
LightGBMExecutionParams
def explainParam(param: Param[_]): String

Definition Classes
Params
def explainParams(): String

Definition Classes
Params
val extraSeed: IntParam

Definition Classes
LightGBMSeedParams
def extractInstances(dataset: Dataset[_], validateInstance: (Instance) ⇒ Unit): RDD[Instance]

Attributes
protected
Definition Classes
PredictorParams
def extractInstances(dataset: Dataset[_]): RDD[Instance]

Attributes
protected
Definition Classes
PredictorParams
def extractLabeledPoints(dataset: Dataset[_]): RDD[LabeledPoint]

Attributes
protected
Definition Classes
Predictor
final def extractParamMap(): ParamMap

Definition Classes
Params
final def extractParamMap(extra: ParamMap): ParamMap

Definition Classes
Params
val featureFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val featureFractionByNode: DoubleParam

Definition Classes
LightGBMFractionParams
val featureFractionSeed: IntParam

Definition Classes
LightGBMSeedParams
final val featuresCol: Param[String]

Definition Classes
HasFeaturesCol
val featuresShapCol: Param[String]

Definition Classes
LightGBMPredictionParams
def finalize(): Unit

Attributes
protected[lang]
Definition Classes
AnyRef
Annotations
@throws( classOf[java.lang.Throwable] )
def fit(dataset: Dataset[_]): LightGBMRankerModel

Definition Classes
Predictor → Estimator
def fit(dataset: Dataset[_], paramMaps: Seq[ParamMap]): Seq[LightGBMRankerModel]

Definition Classes
Estimator
Annotations
@Since( "2.0.0" )
def fit(dataset: Dataset[_], paramMap: ParamMap): LightGBMRankerModel

Definition Classes
Estimator
Annotations
@Since( "2.0.0" )
def fit(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): LightGBMRankerModel

Definition Classes
Estimator
Annotations
@Since( "2.0.0" ) @varargs()
val fobj: FObjParam

Definition Classes
LightGBMObjectiveParams
final def get[T](param: Param[T]): Option[T]

Definition Classes
Params
def getAllPerformanceMeasures: Option[Array[InstrumentationMeasures]]

Definition Classes
LightGBMPerformance
def getBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getBaggingFreq: Int

Definition Classes
LightGBMParams
def getBaggingSeed: Int

Definition Classes
LightGBMSeedParams
def getBinSampleCount: Int

Definition Classes
LightGBMBinParams
def getBoostFromAverage: Boolean

Definition Classes
LightGBMParams
def getBoostingType: String

Definition Classes
LightGBMParams
def getCatSmooth: Double

Definition Classes
LightGBMCategoricalParams
def getCategoricalIndexes(featuresSchema: StructField): Array[Int]
Retrieves the categorical indexes in the features column.
Retrieves the categorical indexes in the features column.
featuresSchema
The schema of the features column
returns
the categorical indexes in the features column.

Attributes
protected
Definition Classes
LightGBMBase
def getCategoricalParams: CategoricalParams
Constructs the CategoricalParams.
Constructs the CategoricalParams.
returns
CategoricalParams object containing the parameters related to LightGBM categorical features.

Attributes
protected
Definition Classes
LightGBMBase
def getCategoricalSlotIndexes: Array[Int]

Definition Classes
LightGBMSlotParams
def getCategoricalSlotNames: Array[String]

Definition Classes
LightGBMSlotParams
def getCatl2: Double

Definition Classes
LightGBMCategoricalParams
def getChunkSize: Int

Definition Classes
LightGBMExecutionParams
final def getClass(): Class[_]

Definition Classes
AnyRef → Any
Annotations
@native()
def getColumnParams: ColumnParams
Constructs the ColumnParams.
Constructs the ColumnParams.
returns
ColumnParams object containing the parameters related to LightGBM columns.

Attributes
protected
Definition Classes
LightGBMBase
def getDartParams: DartModeParams
Constructs the DartModeParams
Constructs the DartModeParams
returns
DartModeParams object containing parameters related to dart mode.

Attributes
protected
Definition Classes
LightGBMBase
def getDataRandomSeed: Int

Definition Classes
LightGBMSeedParams
def getDatasetCreationParams(categoricalIndexes: Array[Int], numThreads: Int): String

Definition Classes
LightGBMBase
def getDatasetParams: DatasetParams
Constructs the DatasetParams.
Constructs the DatasetParams.
returns
DatasetParams object containing parameters related to LightGBM Dataset parameters.

Attributes
protected
Definition Classes
LightGBMBase
final def getDefault[T](param: Param[T]): Option[T]

Definition Classes
Params
def getDefaultListenPort: Int

Definition Classes
LightGBMExecutionParams
def getDelegate: Option[LightGBMDelegate]

Definition Classes
LightGBMParams
def getDeterministic: Boolean

Definition Classes
LightGBMSeedParams
def getDriverListenPort: Int

Definition Classes
LightGBMExecutionParams
def getDropRate: Double

Definition Classes
LightGBMDartParams
def getDropSeed: Int

Definition Classes
LightGBMSeedParams
def getEarlyStoppingRound: Int

Definition Classes
LightGBMLearnerParams
def getEvalAt: Array[Int]
def getExecutionMode: String

Definition Classes
LightGBMExecutionParams
def getExecutionParams(numTasksPerExec: Int): ExecutionParams
Constructs the ExecutionParams.
Constructs the ExecutionParams.
returns
ExecutionParams object containing parameters related to LightGBM execution.

Attributes
protected
Definition Classes
LightGBMBase
def getExtraSeed: Int

Definition Classes
LightGBMSeedParams
def getFObj: FObjTrait

Definition Classes
LightGBMObjectiveParams
def getFeatureFraction: Double

Definition Classes
LightGBMFractionParams
def getFeatureFractionByNode: Double

Definition Classes
LightGBMFractionParams
def getFeatureFractionSeed: Int

Definition Classes
LightGBMSeedParams
final def getFeaturesCol: String

Definition Classes
HasFeaturesCol
def getFeaturesShapCol: String

Definition Classes
LightGBMPredictionParams
def getGeneralParams(numTasks: Int, featuresSchema: StructField): GeneralParams
Constructs the GeneralParams.
Constructs the GeneralParams.
returns
GeneralParams object containing parameters related to general LightGBM parameters.

Attributes
protected
Definition Classes
LightGBMBase
def getImprovementTolerance: Double

Definition Classes
LightGBMLearnerParams
def getIsEnableSparse: Boolean

Definition Classes
LightGBMDatasetParams
def getIsProvideTrainingMetric: Boolean

Definition Classes
LightGBMParams
final def getLabelCol: String

Definition Classes
HasLabelCol
def getLabelGain: Array[Double]
def getLambdaL1: Double

Definition Classes
LightGBMParams
def getLambdaL2: Double

Definition Classes
LightGBMParams
def getLeafPredictionCol: String

Definition Classes
LightGBMPredictionParams
def getLearningRate: Double

Definition Classes
LightGBMParams
def getMatrixType: String

Definition Classes
LightGBMExecutionParams
def getMaxBin: Int

Definition Classes
LightGBMBinParams
def getMaxBinByFeature: Array[Int]

Definition Classes
LightGBMParams
def getMaxCatThreshold: Int

Definition Classes
LightGBMCategoricalParams
def getMaxCatToOnehot: Int

Definition Classes
LightGBMCategoricalParams
def getMaxDeltaStep: Double

Definition Classes
LightGBMParams
def getMaxDepth: Int

Definition Classes
LightGBMParams
def getMaxDrop: Int

Definition Classes
LightGBMDartParams
def getMaxNumClasses: Int

Definition Classes
LightGBMParams
def getMaxPosition: Int
def getMaxStreamingOMPThreads: Int

Definition Classes
LightGBMExecutionParams
def getMetric: String

Definition Classes
LightGBMParams
def getMicroBatchSize: Int

Definition Classes
LightGBMExecutionParams
def getMinDataInLeaf: Int

Definition Classes
LightGBMParams
def getMinDataPerBin: Int

Definition Classes
LightGBMParams
def getMinDataPerGroup: Int

Definition Classes
LightGBMCategoricalParams
def getMinGainToSplit: Double

Definition Classes
LightGBMParams
def getMinSumHessianInLeaf: Double

Definition Classes
LightGBMParams
def getModel(trainParams: BaseTrainParams, lightGBMBooster: LightGBMBooster): LightGBMRankerModel
Gets the trained model given the train parameters and booster.
Gets the trained model given the train parameters and booster.
returns
trained model.

Definition Classes
LightGBMRanker → LightGBMBase
def getModelString: String

Definition Classes
LightGBMParams
def getMonotoneConstraints: Array[Int]

Definition Classes
LightGBMLearnerParams
def getMonotoneConstraintsMethod: String

Definition Classes
LightGBMLearnerParams
def getMonotonePenalty: Double

Definition Classes
LightGBMLearnerParams
def getNegBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getNumBatches: Int

Definition Classes
LightGBMExecutionParams
def getNumIterations: Int

Definition Classes
LightGBMParams
def getNumLeaves: Int

Definition Classes
LightGBMParams
def getNumTasks: Int

Definition Classes
LightGBMExecutionParams
def getNumThreads: Int

Definition Classes
LightGBMExecutionParams
def getObjective: String

Definition Classes
LightGBMObjectiveParams
def getObjectiveParams: ObjectiveParams
Constructs the ObjectiveParams.
Constructs the ObjectiveParams.
returns
ObjectiveParams object containing parameters related to the objective function.

Attributes
protected
Definition Classes
LightGBMBase
def getObjectiveSeed: Int

Definition Classes
LightGBMSeedParams
def getOptGroupCol: Option[String]
Optional group column for Ranking, set to None by default.
Optional group column for Ranking, set to None by default.
returns
None

Definition Classes
LightGBMRanker → LightGBMBase
final def getOrDefault[T](param: Param[T]): T

Definition Classes
Params
def getOtherRate: Double

Definition Classes
LightGBMLearnerParams
def getParallelism: String

Definition Classes
LightGBMExecutionParams
def getParam(paramName: String): Param[Any]

Definition Classes
Params
def getParamInfo(p: Param[_]): ParamInfo[_]

Definition Classes
BaseWrappable
def getPassThroughArgs: String

Definition Classes
LightGBMExecutionParams
def getPerformanceMeasures: Option[InstrumentationMeasures]
In the common case of 1 batch, there is only 1 measure, so this is a convenience method.
In the common case of 1 batch, there is only 1 measure, so this is a convenience method.

Definition Classes
LightGBMPerformance
def getPosBaggingFraction: Double

Definition Classes
LightGBMFractionParams
def getPredictDisableShapeCheck: Boolean

Definition Classes
LightGBMPredictionParams
final def getPredictionCol: String

Definition Classes
HasPredictionCol
def getRepartitionByGroupingColumn: Boolean

Definition Classes
LightGBMExecutionParams
def getSeed: Int

Definition Classes
LightGBMSeedParams
def getSeedParams: SeedParams
Constructs the SeedParams.
Constructs the SeedParams.
returns
SeedParams object containing the parameters related to LightGBM seeds and determinism.

Attributes
protected
Definition Classes
LightGBMBase
def getSkipDrop: Double

Definition Classes
LightGBMDartParams
def getSlotNames: Array[String]

Definition Classes
LightGBMSlotParams
def getSlotNamesWithMetadata(featuresSchema: StructField): Option[Array[String]]

Definition Classes
LightGBMBase
def getTimeout: Double

Definition Classes
LightGBMExecutionParams
def getTopK: Int

Definition Classes
LightGBMExecutionParams
def getTopRate: Double

Definition Classes
LightGBMLearnerParams
def getTrainParams(numTasks: Int, featuresSchema: StructField, numTasksPerExec: Int): BaseTrainParams
Gets the training parameters.
Gets the training parameters.
numTasks
The total number of tasks.
featuresSchema
The features column schema.
numTasksPerExec
The number of tasks per executor.
returns
train parameters.

Definition Classes
LightGBMRanker → LightGBMBase
def getTrainingCols: Array[(String, Seq[DataType])]

Attributes
protected
Definition Classes
LightGBMBase
def getTrainingContext(dataframe: DataFrame, validationData: Option[Broadcast[Array[Row]]], broadcastedSampleData: Option[Broadcast[Array[Row]]], partitionCounts: Option[Array[Long]], trainParams: BaseTrainParams, numCols: Int, numInitValueClasses: Int, batchIndex: Int, numTasksPerExecutor: Int, networkManager: NetworkManager): TrainingContext
Get the object that holds all relevant context information for the training session.
Get the object that holds all relevant context information for the training session.
dataframe
The dataset to train on.
validationData
The dataset to use as validation. (optional)
broadcastedSampleData
Sample data to use for streaming mode Dataset creation (optional).
partitionCounts
The count per partition for streaming mode (optional).
trainParams
Training parameters.
numCols
Number of columns.
numInitValueClasses
Number of classes for initial values (used only for multiclass).
batchIndex
In running in batch training mode, gets the batch number.
numTasksPerExecutor
Number of tasks per executor.
networkManager
The network manager.
returns
The context of the training session.

Attributes
protected
Definition Classes
LightGBMBase
def getUniformDrop: Boolean

Definition Classes
LightGBMDartParams
def getUseBarrierExecutionMode: Boolean

Definition Classes
LightGBMExecutionParams
def getUseMissing: Boolean

Definition Classes
LightGBMDatasetParams
def getUseSingleDatasetMode: Boolean

Definition Classes
LightGBMExecutionParams
def getVerbosity: Int

Definition Classes
LightGBMParams
def getXGBoostDartMode: Boolean

Definition Classes
LightGBMDartParams
def getZeroAsMissing: Boolean

Definition Classes
LightGBMDatasetParams
final def hasDefault[T](param: Param[T]): Boolean

Definition Classes
Params
def hasParam(paramName: String): Boolean

Definition Classes
Params
def hashCode(): Int

Definition Classes
AnyRef → Any
Annotations
@native()
val improvementTolerance: DoubleParam

Definition Classes
LightGBMLearnerParams
def initPerformanceMeasures(batchCount: Int): Unit

Attributes
protected
Definition Classes
LightGBMPerformance
def initializeLogIfNecessary(isInterpreter: Boolean, silent: Boolean): Boolean

Attributes
protected
Definition Classes
Logging
def initializeLogIfNecessary(isInterpreter: Boolean): Unit

Attributes
protected
Definition Classes
Logging
final def isDefined(param: Param[_]): Boolean

Definition Classes
Params
val isEnableSparse: BooleanParam

Definition Classes
LightGBMDatasetParams
final def isInstanceOf[T0]: Boolean

Definition Classes
Any
val isProvideTrainingMetric: BooleanParam

Definition Classes
LightGBMParams
final def isSet(param: Param[_]): Boolean

Definition Classes
Params
def isTraceEnabled(): Boolean

Attributes
protected
Definition Classes
Logging
final val labelCol: Param[String]

Definition Classes
HasLabelCol
val labelGain: DoubleArrayParam
val lambdaL1: DoubleParam

Definition Classes
LightGBMParams
val lambdaL2: DoubleParam

Definition Classes
LightGBMParams
val leafPredictionCol: Param[String]

Definition Classes
LightGBMPredictionParams
val learningRate: DoubleParam

Definition Classes
LightGBMParams
def log: Logger

Attributes
protected
Definition Classes
Logging
def logBase(info: SynapseMLLogInfo): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logBase(methodName: String, columns: Option[Int]): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logClass(): Unit

Definition Classes
SynapseMLLogging
def logDebug(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logDebug(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logError(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logErrorBase(methodName: String, e: Exception): Unit

Attributes
protected
Definition Classes
SynapseMLLogging
def logFit[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logInfo(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logInfo(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logName: String

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logTrace(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def logTrain[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logTransform[T](f: ⇒ T, columns: Int): T

Definition Classes
SynapseMLLogging
def logVerb[T](verb: String, f: ⇒ T, columns: Int = -1): T

Definition Classes
SynapseMLLogging
def logWarning(msg: ⇒ String, throwable: Throwable): Unit

Attributes
protected
Definition Classes
Logging
def logWarning(msg: ⇒ String): Unit

Attributes
protected
Definition Classes
Logging
def makeDotnetFile(conf: CodegenConfig): Unit

Definition Classes
DotnetWrappable
def makePyFile(conf: CodegenConfig): Unit

Definition Classes
PythonWrappable
def makeRFile(conf: CodegenConfig): Unit

Definition Classes
RWrappable
val matrixType: Param[String]

Definition Classes
LightGBMExecutionParams
val maxBin: IntParam

Definition Classes
LightGBMBinParams
val maxBinByFeature: IntArrayParam

Definition Classes
LightGBMParams
val maxCatThreshold: IntParam

Definition Classes
LightGBMCategoricalParams
val maxCatToOnehot: IntParam

Definition Classes
LightGBMCategoricalParams
val maxDeltaStep: DoubleParam

Definition Classes
LightGBMParams
val maxDepth: IntParam

Definition Classes
LightGBMParams
val maxDrop: IntParam

Definition Classes
LightGBMDartParams
val maxNumClasses: IntParam

Definition Classes
LightGBMParams
val maxPosition: IntParam
val maxStreamingOMPThreads: IntParam

Definition Classes
LightGBMExecutionParams
val metric: Param[String]

Definition Classes
LightGBMParams
val microBatchSize: IntParam

Definition Classes
LightGBMExecutionParams
val minDataInLeaf: IntParam

Definition Classes
LightGBMParams
val minDataPerBin: IntParam

Definition Classes
LightGBMParams
val minDataPerGroup: IntParam

Definition Classes
LightGBMCategoricalParams
val minGainToSplit: DoubleParam

Definition Classes
LightGBMParams
val minSumHessianInLeaf: DoubleParam

Definition Classes
LightGBMParams
val modelString: Param[String]

Definition Classes
LightGBMParams
val monotoneConstraints: IntArrayParam

Definition Classes
LightGBMLearnerParams
val monotoneConstraintsMethod: Param[String]

Definition Classes
LightGBMLearnerParams
val monotonePenalty: DoubleParam

Definition Classes
LightGBMLearnerParams
final def ne(arg0: AnyRef): Boolean

Definition Classes
AnyRef
val negBaggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
final def notify(): Unit

Definition Classes
AnyRef
Annotations
@native()
final def notifyAll(): Unit

Definition Classes
AnyRef
Annotations
@native()
val numBatches: IntParam

Definition Classes
LightGBMExecutionParams
val numIterations: IntParam

Definition Classes
LightGBMParams
val numLeaves: IntParam

Definition Classes
LightGBMParams
val numTasks: IntParam

Definition Classes
LightGBMExecutionParams
val numThreads: IntParam

Definition Classes
LightGBMExecutionParams
val objective: Param[String]

Definition Classes
LightGBMObjectiveParams
val objectiveSeed: IntParam

Definition Classes
LightGBMSeedParams
val otherRate: DoubleParam

Definition Classes
LightGBMLearnerParams
val parallelism: Param[String]

Definition Classes
LightGBMExecutionParams
lazy val params: Array[Param[_]]

Definition Classes
Params
val passThroughArgs: Param[String]

Definition Classes
LightGBMExecutionParams
val posBaggingFraction: DoubleParam

Definition Classes
LightGBMFractionParams
val predictDisableShapeCheck: BooleanParam

Definition Classes
LightGBMPredictionParams
final val predictionCol: Param[String]

Definition Classes
HasPredictionCol
def prepareDataframe(dataset: Dataset[_], numTasks: Int): DataFrame

Definition Classes
LightGBMRanker → LightGBMBase
def preprocessData(df: DataFrame): DataFrame
For Ranking, we need to sort the data within partitions by group prior to training to ensure training succeeds.
For Ranking, we need to sort the data within partitions by group prior to training to ensure training succeeds.
df
The data frame to preprocess prior to training.
returns
The preprocessed data, sorted within partition by group.

Definition Classes
LightGBMRanker → LightGBMBase
def pyAdditionalMethods: String

Definition Classes
PythonWrappable
lazy val pyClassDoc: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyClassName: String

Attributes
protected
Definition Classes
PythonWrappable
def pyExtraEstimatorImports: String

Attributes
protected
Definition Classes
PythonWrappable
def pyExtraEstimatorMethods: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyInheritedClasses: Seq[String]

Attributes
protected
Definition Classes
PythonWrappable
def pyInitFunc(): String

Definition Classes
PythonWrappable
lazy val pyInternalWrapper: Boolean

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyObjectBaseClass: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamArg[T](p: Param[T]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamDefault[T](p: Param[T]): Option[String]

Attributes
protected
Definition Classes
PythonWrappable
def pyParamGetter(p: Param[_]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamSetter(p: Param[_]): String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsArgs: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsDefaults: String

Attributes
protected
Definition Classes
PythonWrappable
lazy val pyParamsDefinitions: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsGetters: String

Attributes
protected
Definition Classes
PythonWrappable
def pyParamsSetters: String

Attributes
protected
Definition Classes
PythonWrappable
def pythonClass(): String

Attributes
protected
Definition Classes
PythonWrappable
def rClass(): String

Attributes
protected
Definition Classes
RWrappable
def rDocString: String

Attributes
protected
Definition Classes
RWrappable
def rExtraBodyLines: String

Attributes
protected
Definition Classes
RWrappable
def rExtraInitLines: String

Attributes
protected
Definition Classes
RWrappable
lazy val rFuncName: String

Attributes
protected
Definition Classes
RWrappable
lazy val rInternalWrapper: Boolean

Attributes
protected
Definition Classes
RWrappable
def rParamArg[T](p: Param[T]): String

Attributes
protected
Definition Classes
RWrappable
def rParamsArgs: String

Attributes
protected
Definition Classes
RWrappable
def rSetterLines: String

Attributes
protected
Definition Classes
RWrappable
val repartitionByGroupingColumn: BooleanParam

Definition Classes
LightGBMExecutionParams
def save(path: String): Unit

Definition Classes
MLWritable
Annotations
@Since( "1.6.0" ) @throws( ... )
val seed: IntParam

Definition Classes
LightGBMSeedParams
final def set(paramPair: ParamPair[_]): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def set(param: String, value: Any): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def set[T](param: Param[T], value: T): LightGBMRanker.this.type

Definition Classes
Params
def setBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setBaggingFreq(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setBaggingSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setBatchPerformanceMeasure(index: Int, measures: InstrumentationMeasures): Unit

Attributes
protected
Definition Classes
LightGBMPerformance
def setBatchPerformanceMeasures(measures: Array[Option[InstrumentationMeasures]]): LightGBMRanker.this.type

Attributes
protected
Definition Classes
LightGBMPerformance
def setBinSampleCount(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMBinParams
def setBoostFromAverage(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setBoostingType(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setCatSmooth(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setCategoricalSlotIndexes(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setCategoricalSlotNames(value: Array[String]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setCatl2(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setChunkSize(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDataRandomSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
final def setDefault(paramPairs: ParamPair[_]*): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
final def setDefault[T](param: Param[T], value: T): LightGBMRanker.this.type

Attributes
protected
Definition Classes
Params
def setDefaultListenPort(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDelegate(delegate: LightGBMDelegate): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setDeterministic(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setDriverListenPort(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setDropRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setDropSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setEarlyStoppingRound(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setEvalAt(value: Array[Int]): LightGBMRanker.this.type
def setExecutionMode(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setExtraSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setFObj(value: FObjTrait): LightGBMRanker.this.type

Definition Classes
LightGBMObjectiveParams
def setFeatureFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setFeatureFractionByNode(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setFeatureFractionSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setFeaturesCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setFeaturesShapCol(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setImprovementTolerance(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setIsEnableSparse(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
def setIsProvideTrainingMetric(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLabelCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setLabelGain(value: Array[Double]): LightGBMRanker.this.type
def setLambdaL1(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLambdaL2(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setLeafPredictionCol(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setLearningRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMatrixType(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMaxBin(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMBinParams
def setMaxBinByFeature(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxCatThreshold(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMaxCatToOnehot(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMaxDeltaStep(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxDepth(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxDrop(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setMaxNumClasses(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMaxPosition(value: Int): LightGBMRanker.this.type
def setMaxStreamingOMPThreads(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMetric(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMicroBatchSize(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setMinDataInLeaf(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinDataPerBin(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinDataPerGroup(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMCategoricalParams
def setMinGainToSplit(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMinSumHessianInLeaf(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setModelString(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setMonotoneConstraints(value: Array[Int]): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setMonotoneConstraintsMethod(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setMonotonePenalty(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setNegBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setNumBatches(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setNumIterations(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setNumLeaves(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setNumTasks(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setNumThreads(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setObjective(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMObjectiveParams
def setObjectiveSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setOtherRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setParallelism(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setPassThroughArgs(value: String): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setPosBaggingFraction(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMFractionParams
def setPredictDisableShapeCheck(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMPredictionParams
def setPredictionCol(value: String): LightGBMRanker

Definition Classes
Predictor
def setRepartitionByGroupingColumn(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setSeed(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMSeedParams
def setSkipDrop(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setSlotNames(value: Array[String]): LightGBMRanker.this.type

Definition Classes
LightGBMSlotParams
def setTimeout(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setTopK(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setTopRate(value: Double): LightGBMRanker.this.type

Definition Classes
LightGBMLearnerParams
def setUniformDrop(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setUseBarrierExecutionMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setUseMissing(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
def setUseSingleDatasetMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMExecutionParams
def setVerbosity(value: Int): LightGBMRanker.this.type

Definition Classes
LightGBMParams
def setXGBoostDartMode(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDartParams
def setZeroAsMissing(value: Boolean): LightGBMRanker.this.type

Definition Classes
LightGBMDatasetParams
val skipDrop: DoubleParam

Definition Classes
LightGBMDartParams
val slotNames: StringArrayParam

Definition Classes
LightGBMSlotParams
def stringFromTrainedModel(model: LightGBMRankerModel): String

Definition Classes
LightGBMRanker → LightGBMBase
final def synchronized[T0](arg0: ⇒ T0): T0

Definition Classes
AnyRef
val thisStage: Params

Attributes
protected
Definition Classes
BaseWrappable
val timeout: DoubleParam

Definition Classes
LightGBMExecutionParams
def toString(): String

Definition Classes
Identifiable → AnyRef → Any
val topK: IntParam

Definition Classes
LightGBMExecutionParams
val topRate: DoubleParam

Definition Classes
LightGBMLearnerParams
def train(dataset: Dataset[_]): LightGBMRankerModel
Trains the LightGBM model.
Trains the LightGBM model. If batches are specified, breaks training dataset into batches for training.
dataset
The input dataset to train.
returns
The trained model.

Attributes
protected
Definition Classes
LightGBMBase
def trainOneDataBatch(dataset: Dataset[_], batchIndex: Int, batchCount: Int): LightGBMRankerModel
Inner train method for LightGBM learners.
Inner train method for LightGBM learners. Calculates the number of workers, creates a driver thread, and runs mapPartitions on the dataset.
dataset
The dataset to train on.
batchIndex
In running in batch training mode, gets the batch number.
returns
The LightGBM Model from the trained LightGBM Booster.

Attributes
protected
Definition Classes
LightGBMBase
def transformSchema(schema: StructType): StructType

Definition Classes
Predictor → PipelineStage
def transformSchema(schema: StructType, logging: Boolean): StructType

Attributes
protected
Definition Classes
PipelineStage
Annotations
@DeveloperApi()
val uid: String

Definition Classes
LightGBMRanker → SynapseMLLogging → Identifiable
val uniformDrop: BooleanParam

Definition Classes
LightGBMDartParams
val useBarrierExecutionMode: BooleanParam

Definition Classes
LightGBMExecutionParams
val useMissing: BooleanParam

Definition Classes
LightGBMDatasetParams
val useSingleDatasetMode: BooleanParam

Definition Classes
LightGBMExecutionParams
def validateAndTransformSchema(schema: StructType, fitting: Boolean, featuresDataType: DataType): StructType

Attributes
protected
Definition Classes
PredictorParams
val verbosity: IntParam

Definition Classes
LightGBMParams
final def wait(): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long, arg1: Int): Unit

Definition Classes
AnyRef
Annotations
@throws( ... )
final def wait(arg0: Long): Unit

Definition Classes
AnyRef
Annotations
@throws( ... ) @native()
def write: MLWriter

Definition Classes
DefaultParamsWritable → MLWritable
val xGBoostDartMode: BooleanParam

Definition Classes
LightGBMDartParams
val zeroAsMissing: BooleanParam

Definition Classes
LightGBMDatasetParams

Packages

LightGBMRanker 

Companion object LightGBMRanker

class LightGBMRanker extends Ranker[Vector, LightGBMRanker, LightGBMRankerModel] with LightGBMBase[LightGBMRankerModel] with SynapseMLLogging

Instance Constructors

Value Members

Inherited from LightGBMBase[LightGBMRankerModel]

Value Members

Inherited from SynapseMLLogging

Value Members

Inherited from LightGBMPerformance

Value Members

Inherited from LightGBMParams

Value Members

Inherited from LightGBMCategoricalParams

Value Members

Inherited from LightGBMSeedParams

Value Members

Inherited from LightGBMObjectiveParams

Value Members

Inherited from LightGBMPredictionParams

Value Members

Inherited from LightGBMDartParams

Value Members

Inherited from LightGBMDatasetParams

Value Members

Inherited from LightGBMLearnerParams

Value Members

Inherited from LightGBMBinParams

Value Members

Inherited from LightGBMFractionParams

Value Members

Inherited from LightGBMSlotParams

Value Members

Inherited from LightGBMExecutionParams

Value Members

Inherited from HasInitScoreCol

Value Members

Inherited from HasValidationIndicatorCol

Value Members

Inherited from HasWeightCol

Value Members

Inherited from DefaultParamsWritable

Value Members

Inherited from MLWritable

Value Members

Inherited from DotnetWrappable

Value Members

Inherited from RWrappable

Value Members

Inherited from PythonWrappable

Value Members

Inherited from BaseWrappable

Value Members

Inherited from HasGroupCol

Value Members

Inherited from Predictor[Vector, LightGBMRanker, LightGBMRankerModel]

Value Members

Inherited from PredictorParams

Value Members

Inherited from HasPredictionCol

Value Members

Inherited from HasFeaturesCol

Value Members

Inherited from HasLabelCol

Value Members

Inherited from Estimator[LightGBMRankerModel]

Value Members

Inherited from PipelineStage

Value Members

Inherited from Logging

Value Members

Inherited from Params

Value Members

Inherited from Identifiable

Value Members

Inherited from AnyRef

Value Members

Inherited from Any

Value Members

LightGBMRanker