object SparkSchema
Schema modification and information retrieval methods.
- Alphabetic
- By Inheritance
- SparkSchema
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
def
getLabelColumnName: (StructType, String) ⇒ String
Gets the label column name.
Gets the label column name.
- returns
The label column name.
-
def
getLabelColumnName(dataset: DataFrame, modelName: String): String
Gets the label column name.
Gets the label column name.
- dataset
The dataset to get the label column from.
- modelName
The model to retrieve the label column from.
- returns
The label column name.
-
def
getScoreValueKind(schema: StructType, modelName: String, columnName: String): String
Gets the score value kind or null if it does not exist from the schema.
Gets the score value kind or null if it does not exist from the schema.
- schema
The schema to get the score column kind column name from.
- modelName
The model to retrieve the score column kind column name from.
- columnName
The column to retrieve the score value kind from.
-
def
getScoreValueKind(dataset: DataFrame, modelName: String, columnName: String): String
Gets the score value kind or null if it does not exist from a dataset.
Gets the score value kind or null if it does not exist from a dataset.
- dataset
The dataset to get the score column kind column name from.
- modelName
The model to retrieve the score column kind column name from.
- columnName
The column to retrieve the score value kind from.
-
def
getSparkPredictionColumnName: (StructType, String) ⇒ String
Gets the spark prediction column name.
Gets the spark prediction column name.
- returns
The spark prediction column name.
-
def
getSparkPredictionColumnName(dataset: DataFrame, modelName: String): String
Gets the spark prediction column name.
Gets the spark prediction column name.
- dataset
The dataset to get the spark prediction column from.
- modelName
The model to retrieve the spark prediction column from.
- returns
The spark prediction column name.
-
def
getSparkProbabilityColumnName: (StructType, String) ⇒ String
Gets the spark probability column name.
Gets the spark probability column name.
- returns
The spark probability column name.
-
def
getSparkProbabilityColumnName(dataset: DataFrame, modelName: String): String
Gets the spark probability column name.
Gets the spark probability column name.
- dataset
The dataset to get the spark probability column from.
- modelName
The model to retrieve the spark probability column from.
- returns
The spark probability column name.
-
def
getSparkRawPredictionColumnName: (StructType, String) ⇒ String
Gets the spark raw prediction column name.
Gets the spark raw prediction column name.
- returns
The spark raw prediction column name.
-
def
getSparkRawPredictionColumnName(dataset: DataFrame, modelName: String): String
Gets the spark raw prediction column name.
Gets the spark raw prediction column name.
- dataset
The dataset to get the spark raw prediction column from.
- modelName
The model to retrieve the spark raw prediction column from.
- returns
The spark raw prediction column name.
-
def
isBoolean(df: DataFrame, column: String): Boolean
Find if the given column is boolean
-
def
isCategorical(df: DataFrame, column: String): Boolean
Find if the given column is Categorical; use CategoricalColumnInfo for more details
-
def
isNumeric(df: DataFrame, column: String): Boolean
Find if the given column is numeric
-
def
isString(df: DataFrame, column: String): Boolean
Find if the given column is a string
-
def
setLabelColumnName: (DataFrame, String, String, String) ⇒ DataFrame
Sets the label column name.
Sets the label column name.
- returns
The modified dataset.
-
def
updateColumnMetadata(dataset: DataFrame, modelName: String, columnName: String, modelKind: String): DataFrame
Sets the column kind without renaming column name.
Sets the column kind without renaming column name.
- dataset
The dataset to set the column kind on.
- modelName
The model name.
- columnName
The column name to set as the specified column kind.
- modelKind
The model type.