Packages

object SparkSchema

Schema modification and information retrieval methods.

Linear Supertypes
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. SparkSchema
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Value Members

  1. def getLabelColumnName: (StructType, String) ⇒ String

    Gets the label column name.

    Gets the label column name.

    returns

    The label column name.

  2. def getLabelColumnName(dataset: DataFrame, modelName: String): String

    Gets the label column name.

    Gets the label column name.

    dataset

    The dataset to get the label column from.

    modelName

    The model to retrieve the label column from.

    returns

    The label column name.

  3. def getScoreValueKind(schema: StructType, modelName: String, columnName: String): String

    Gets the score value kind or null if it does not exist from the schema.

    Gets the score value kind or null if it does not exist from the schema.

    schema

    The schema to get the score column kind column name from.

    modelName

    The model to retrieve the score column kind column name from.

    columnName

    The column to retrieve the score value kind from.

  4. def getScoreValueKind(dataset: DataFrame, modelName: String, columnName: String): String

    Gets the score value kind or null if it does not exist from a dataset.

    Gets the score value kind or null if it does not exist from a dataset.

    dataset

    The dataset to get the score column kind column name from.

    modelName

    The model to retrieve the score column kind column name from.

    columnName

    The column to retrieve the score value kind from.

  5. def getSparkPredictionColumnName: (StructType, String) ⇒ String

    Gets the spark prediction column name.

    Gets the spark prediction column name.

    returns

    The spark prediction column name.

  6. def getSparkPredictionColumnName(dataset: DataFrame, modelName: String): String

    Gets the spark prediction column name.

    Gets the spark prediction column name.

    dataset

    The dataset to get the spark prediction column from.

    modelName

    The model to retrieve the spark prediction column from.

    returns

    The spark prediction column name.

  7. def getSparkProbabilityColumnName: (StructType, String) ⇒ String

    Gets the spark probability column name.

    Gets the spark probability column name.

    returns

    The spark probability column name.

  8. def getSparkProbabilityColumnName(dataset: DataFrame, modelName: String): String

    Gets the spark probability column name.

    Gets the spark probability column name.

    dataset

    The dataset to get the spark probability column from.

    modelName

    The model to retrieve the spark probability column from.

    returns

    The spark probability column name.

  9. def getSparkRawPredictionColumnName: (StructType, String) ⇒ String

    Gets the spark raw prediction column name.

    Gets the spark raw prediction column name.

    returns

    The spark raw prediction column name.

  10. def getSparkRawPredictionColumnName(dataset: DataFrame, modelName: String): String

    Gets the spark raw prediction column name.

    Gets the spark raw prediction column name.

    dataset

    The dataset to get the spark raw prediction column from.

    modelName

    The model to retrieve the spark raw prediction column from.

    returns

    The spark raw prediction column name.

  11. def isBoolean(df: DataFrame, column: String): Boolean

    Find if the given column is boolean

  12. def isCategorical(df: DataFrame, column: String): Boolean

    Find if the given column is Categorical; use CategoricalColumnInfo for more details

  13. def isNumeric(df: DataFrame, column: String): Boolean

    Find if the given column is numeric

  14. def isString(df: DataFrame, column: String): Boolean

    Find if the given column is a string

  15. def setLabelColumnName: (DataFrame, String, String, String) ⇒ DataFrame

    Sets the label column name.

    Sets the label column name.

    returns

    The modified dataset.

  16. def updateColumnMetadata(dataset: DataFrame, modelName: String, columnName: String, modelKind: String): DataFrame

    Sets the column kind without renaming column name.

    Sets the column kind without renaming column name.

    dataset

    The dataset to set the column kind on.

    modelName

    The model name.

    columnName

    The column name to set as the specified column kind.

    modelKind

    The model type.