c

com.microsoft.azure.synapse.ml.lightgbm

BasePartitionTask

abstract class BasePartitionTask extends Serializable with Logging

Class for handling the execution of Tasks on workers for each partition. Only runs on worker Tasks.

Linear Supertypes
Logging, Serializable, Serializable, AnyRef, Any
Known Subclasses
BulkPartitionTask, StreamingPartitionTask
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. BasePartitionTask
  2. Logging
  3. Serializable
  4. Serializable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new BasePartitionTask()

Concrete Value Members

  1. def mapPartitionTask(ctx: TrainingContext)(inputRows: Iterator[Row]): Iterator[PartitionResult]

    This method will be passed to Spark's mapPartition method and handle execution of training on the workers.

    This method will be passed to Spark's mapPartition method and handle execution of training on the workers. Main stages: (and each execution mode has an "Internal" version to perform mode-specific operations) initialize() preparePartitionData() finalizeDatasetAndTrain() cleanup()

    ctx

    The training context.

    inputRows

    The Spark rows as an iterator.

    returns

    result iterator (to comply with Spark mapPartition API).