object NetworkManager extends Serializable

Linear Supertypes
Serializable, Serializable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. NetworkManager
  2. Serializable
  3. Serializable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  6. def create(numTasks: Int, spark: SparkSession, driverListenPort: Int, timeout: Double, useBarrierExecutionMode: Boolean): NetworkManager

    Create a NetworkManager, which will encapsulate all network operations.

    Create a NetworkManager, which will encapsulate all network operations. This method will opens a socket communications channel on the driver, and then initialize the network manager itself. The NetworkManager object will start a thread that waits for the host:port from the executors, and then sends back the information to the executors.

    numTasks

    The total number of training tasks to wait for.

    spark

    The Spark session.

    driverListenPort

    The port to listen for the driver on.

    timeout

    The timeout (in seconds).

    useBarrierExecutionMode

    Whether to use barrier mode.

    returns

    The NetworkTopology.

  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  10. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def getGlobalNetworkInfo(ctx: TrainingContext, log: Logger, taskId: Long, partitionId: Int, shouldExecuteTraining: Boolean, measures: TaskInstrumentationMeasures): NetworkTopologyInfo

    Retrieve the network nodes and current port information from the driver.

    Retrieve the network nodes and current port information from the driver.

    Establish local socket connection.

    Note: Ideally we would start the socket connections in the C layer, this opens us up for race conditions in case other applications open sockets on cluster, but usually this should not be a problem

    ctx

    Information about the current training session.

    log

    The Logger.

    taskId

    The task id.

    partitionId

    The partition id.

    shouldExecuteTraining

    Whether this task should be a part of the training network.

    measures

    Instrumentation for perf measurements.

    returns

    Information about the network topology.

  12. def getMainWorkerPort(nodes: String, log: Logger): Int

    Gets the main node's port that will return the LightGBM Booster.

    Gets the main node's port that will return the LightGBM Booster. Used to minimize network communication overhead in reduce step.

    returns

    The main node's port number.

  13. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  14. def initLightGBMNetwork(ctx: PartitionTaskContext, log: Logger, retry: Int = LightGBMConstants.NetworkRetries, delay: Long = LightGBMConstants.InitialDelay): Unit
  15. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  16. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  17. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  18. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  19. def parseWorkerMessage(message: String): TaskMessageInfo
  20. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  21. def toString(): String
    Definition Classes
    AnyRef → Any
  22. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  23. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  24. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from Serializable

Inherited from Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped