Packages

object LightGBMUtils

Helper utilities for LightGBM learners

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. LightGBMUtils
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. def addSocketAndComm(hostAndPorts: ListBuffer[(Socket, String)], log: Logger, comm: String, driverSocket: Socket): Unit
  2. def closeConnections(log: Logger, hostAndPorts: ListBuffer[(Socket, String)], driverServerSocket: ServerSocket): Unit
  3. def getFeaturizer(dataset: Dataset[_], labelColumn: String, featuresColumn: String, weightColumn: Option[String] = None, groupColumn: Option[String] = None, oneHotEncodeCategoricals: Boolean = true): PipelineModel
  4. def getNumRowsForChunksArray(numRows: Int, chunkSize: Int): SWIGTYPE_p_int
  5. def getTaskId: Long

    Returns a unique task Id for the current task run on the executor.

    Returns a unique task Id for the current task run on the executor.

    returns

    A unique task id.

  6. def getWorkerId: Int

    Returns an integer ID for the current worker.

    Returns an integer ID for the current worker.

    returns

    In cluster, returns the executor id. In local case, returns the partition id.

  7. def handleConnection(driverServerSocket: ServerSocket, log: Logger, hostAndPorts: ListBuffer[(Socket, String)]): ConnectionState.ConnectionState

    Handles the connection to a task from the driver.

    Handles the connection to a task from the driver.

    driverServerSocket

    The driver socket.

    log

    The log4j logger.

    hostAndPorts

    A list of host and ports of connected tasks.

    returns

    The connection status, can be finished for barrier mode, empty task or connected.

  8. def initializeNativeLibrary(): Unit

    Loads the native shared object binaries lib_lightgbm.so and lib_lightgbm_swig.so

  9. def isLocalExecution: Boolean

    Returns true if spark is run in local mode.

    Returns true if spark is run in local mode.

    returns

    True if spark is run in local mode.

  10. def sendDataToExecutors(hostAndPorts: ListBuffer[(Socket, String)], allConnections: String): Unit
  11. def validate(result: Int, component: String): Unit
  12. def validateArray(result: SWIGTYPE_p_void, component: String): Unit