synapse.ml.train package

Submodules

synapse.ml.train.ComputeModelStatistics module

class synapse.ml.train.ComputeModelStatistics.ComputeModelStatistics(java_obj=None, evaluationMetric='all', labelCol=None, scoredLabelsCol=None, scoresCol=None)[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters
  • evaluationMetric (str) – Metric to evaluate models with

  • labelCol (str) – The name of the label column

  • scoredLabelsCol (str) – Scored labels column name, only required if using SparkML estimators

  • scoresCol (str) – Scores or raw prediction column name, only required if using SparkML estimators

evaluationMetric = Param(parent='undefined', name='evaluationMetric', doc='Metric to evaluate models with')
getEvaluationMetric()[source]
Returns

Metric to evaluate models with

Return type

evaluationMetric

static getJavaPackage()[source]

Returns package name String.

getLabelCol()[source]
Returns

The name of the label column

Return type

labelCol

getScoredLabelsCol()[source]
Returns

Scored labels column name, only required if using SparkML estimators

Return type

scoredLabelsCol

getScoresCol()[source]
Returns

Scores or raw prediction column name, only required if using SparkML estimators

Return type

scoresCol

labelCol = Param(parent='undefined', name='labelCol', doc='The name of the label column')
classmethod read()[source]

Returns an MLReader instance for this class.

scoredLabelsCol = Param(parent='undefined', name='scoredLabelsCol', doc='Scored labels column name, only required if using SparkML estimators')
scoresCol = Param(parent='undefined', name='scoresCol', doc='Scores or raw prediction column name, only required if using SparkML estimators')
setEvaluationMetric(value)[source]
Parameters

evaluationMetric – Metric to evaluate models with

setLabelCol(value)[source]
Parameters

labelCol – The name of the label column

setParams(evaluationMetric='all', labelCol=None, scoredLabelsCol=None, scoresCol=None)[source]

Set the (keyword only) parameters

setScoredLabelsCol(value)[source]
Parameters

scoredLabelsCol – Scored labels column name, only required if using SparkML estimators

setScoresCol(value)[source]
Parameters

scoresCol – Scores or raw prediction column name, only required if using SparkML estimators

synapse.ml.train.ComputePerInstanceStatistics module

class synapse.ml.train.ComputePerInstanceStatistics.ComputePerInstanceStatistics(java_obj=None, evaluationMetric='all', labelCol=None, scoredLabelsCol=None, scoredProbabilitiesCol=None, scoresCol=None)[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters
  • evaluationMetric (str) – Metric to evaluate models with

  • labelCol (str) – The name of the label column

  • scoredLabelsCol (str) – Scored labels column name, only required if using SparkML estimators

  • scoredProbabilitiesCol (str) – Scored probabilities, usually calibrated from raw scores, only required if using SparkML estimators

  • scoresCol (str) – Scores or raw prediction column name, only required if using SparkML estimators

evaluationMetric = Param(parent='undefined', name='evaluationMetric', doc='Metric to evaluate models with')
getEvaluationMetric()[source]
Returns

Metric to evaluate models with

Return type

evaluationMetric

static getJavaPackage()[source]

Returns package name String.

getLabelCol()[source]
Returns

The name of the label column

Return type

labelCol

getScoredLabelsCol()[source]
Returns

Scored labels column name, only required if using SparkML estimators

Return type

scoredLabelsCol

getScoredProbabilitiesCol()[source]
Returns

Scored probabilities, usually calibrated from raw scores, only required if using SparkML estimators

Return type

scoredProbabilitiesCol

getScoresCol()[source]
Returns

Scores or raw prediction column name, only required if using SparkML estimators

Return type

scoresCol

labelCol = Param(parent='undefined', name='labelCol', doc='The name of the label column')
classmethod read()[source]

Returns an MLReader instance for this class.

scoredLabelsCol = Param(parent='undefined', name='scoredLabelsCol', doc='Scored labels column name, only required if using SparkML estimators')
scoredProbabilitiesCol = Param(parent='undefined', name='scoredProbabilitiesCol', doc='Scored probabilities, usually calibrated from raw scores, only required if using SparkML estimators')
scoresCol = Param(parent='undefined', name='scoresCol', doc='Scores or raw prediction column name, only required if using SparkML estimators')
setEvaluationMetric(value)[source]
Parameters

evaluationMetric – Metric to evaluate models with

setLabelCol(value)[source]
Parameters

labelCol – The name of the label column

setParams(evaluationMetric='all', labelCol=None, scoredLabelsCol=None, scoredProbabilitiesCol=None, scoresCol=None)[source]

Set the (keyword only) parameters

setScoredLabelsCol(value)[source]
Parameters

scoredLabelsCol – Scored labels column name, only required if using SparkML estimators

setScoredProbabilitiesCol(value)[source]
Parameters

scoredProbabilitiesCol – Scored probabilities, usually calibrated from raw scores, only required if using SparkML estimators

setScoresCol(value)[source]
Parameters

scoresCol – Scores or raw prediction column name, only required if using SparkML estimators

synapse.ml.train.TrainClassifier module

class synapse.ml.train.TrainClassifier.TrainClassifier(java_obj=None, featuresCol='TrainClassifier_419c9d27becf_features', inputCols=None, labelCol=None, labels=None, model=None, numFeatures=0, reindexLabel=True)[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters
  • featuresCol (str) – The name of the features column

  • inputCols (list) – The names of the input columns

  • labelCol (str) – The name of the label column

  • labels (list) – Sorted label values on the labels column

  • model (object) – Classifier to run

  • numFeatures (int) – Number of features to hash to

  • reindexLabel (bool) – Re-index the label column

featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
getFeaturesCol()[source]
Returns

The name of the features column

Return type

featuresCol

getInputCols()[source]
Returns

The names of the input columns

Return type

inputCols

static getJavaPackage()[source]

Returns package name String.

getLabelCol()[source]
Returns

The name of the label column

Return type

labelCol

getLabels()[source]
Returns

Sorted label values on the labels column

Return type

labels

getModel()[source]
Returns

Classifier to run

Return type

model

getNumFeatures()[source]
Returns

Number of features to hash to

Return type

numFeatures

getReindexLabel()[source]
Returns

Re-index the label column

Return type

reindexLabel

inputCols = Param(parent='undefined', name='inputCols', doc='The names of the input columns')
labelCol = Param(parent='undefined', name='labelCol', doc='The name of the label column')
labels = Param(parent='undefined', name='labels', doc='Sorted label values on the labels column')
model = Param(parent='undefined', name='model', doc='Classifier to run')
numFeatures = Param(parent='undefined', name='numFeatures', doc='Number of features to hash to')
classmethod read()[source]

Returns an MLReader instance for this class.

reindexLabel = Param(parent='undefined', name='reindexLabel', doc='Re-index the label column')
setFeaturesCol(value)[source]
Parameters

featuresCol – The name of the features column

setInputCols(value)[source]
Parameters

inputCols – The names of the input columns

setLabelCol(value)[source]
Parameters

labelCol – The name of the label column

setLabels(value)[source]
Parameters

labels – Sorted label values on the labels column

setModel(value)[source]
Parameters

model – Classifier to run

setNumFeatures(value)[source]
Parameters

numFeatures – Number of features to hash to

setParams(featuresCol='TrainClassifier_419c9d27becf_features', inputCols=None, labelCol=None, labels=None, model=None, numFeatures=0, reindexLabel=True)[source]

Set the (keyword only) parameters

setReindexLabel(value)[source]
Parameters

reindexLabel – Re-index the label column

synapse.ml.train.TrainRegressor module

class synapse.ml.train.TrainRegressor.TrainRegressor(java_obj=None, featuresCol='TrainRegressor_b2d566bd148c_features', inputCols=None, labelCol=None, model=None, numFeatures=0)[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters
  • featuresCol (str) – The name of the features column

  • inputCols (list) – The names of the input columns

  • labelCol (str) – The name of the label column

  • model (object) – Regressor to run

  • numFeatures (int) – Number of features to hash to

featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
getFeaturesCol()[source]
Returns

The name of the features column

Return type

featuresCol

getInputCols()[source]
Returns

The names of the input columns

Return type

inputCols

static getJavaPackage()[source]

Returns package name String.

getLabelCol()[source]
Returns

The name of the label column

Return type

labelCol

getModel()[source]
Returns

Regressor to run

Return type

model

getNumFeatures()[source]
Returns

Number of features to hash to

Return type

numFeatures

inputCols = Param(parent='undefined', name='inputCols', doc='The names of the input columns')
labelCol = Param(parent='undefined', name='labelCol', doc='The name of the label column')
model = Param(parent='undefined', name='model', doc='Regressor to run')
numFeatures = Param(parent='undefined', name='numFeatures', doc='Number of features to hash to')
classmethod read()[source]

Returns an MLReader instance for this class.

setFeaturesCol(value)[source]
Parameters

featuresCol – The name of the features column

setInputCols(value)[source]
Parameters

inputCols – The names of the input columns

setLabelCol(value)[source]
Parameters

labelCol – The name of the label column

setModel(value)[source]
Parameters

model – Regressor to run

setNumFeatures(value)[source]
Parameters

numFeatures – Number of features to hash to

setParams(featuresCol='TrainRegressor_b2d566bd148c_features', inputCols=None, labelCol=None, model=None, numFeatures=0)[source]

Set the (keyword only) parameters

synapse.ml.train.TrainedClassifierModel module

class synapse.ml.train.TrainedClassifierModel.TrainedClassifierModel(java_obj=None, featuresCol=None, labelCol=None, levels=None, model=None)[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters
  • featuresCol (str) – The name of the features column

  • labelCol (str) – The name of the label column

  • levels (object) – the levels

  • model (object) – model produced by training

featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
getFeaturesCol()[source]
Returns

The name of the features column

Return type

featuresCol

static getJavaPackage()[source]

Returns package name String.

getLabelCol()[source]
Returns

The name of the label column

Return type

labelCol

getLevels()[source]
Returns

the levels

Return type

levels

getModel()[source]
Returns

model produced by training

Return type

model

labelCol = Param(parent='undefined', name='labelCol', doc='The name of the label column')
levels = Param(parent='undefined', name='levels', doc='the levels')
model = Param(parent='undefined', name='model', doc='model produced by training')
classmethod read()[source]

Returns an MLReader instance for this class.

setFeaturesCol(value)[source]
Parameters

featuresCol – The name of the features column

setLabelCol(value)[source]
Parameters

labelCol – The name of the label column

setLevels(value)[source]
Parameters

levels – the levels

setModel(value)[source]
Parameters

model – model produced by training

setParams(featuresCol=None, labelCol=None, levels=None, model=None)[source]

Set the (keyword only) parameters

synapse.ml.train.TrainedRegressorModel module

class synapse.ml.train.TrainedRegressorModel.TrainedRegressorModel(java_obj=None, featuresCol=None, labelCol=None, model=None)[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters
  • featuresCol (str) – The name of the features column

  • labelCol (str) – The name of the label column

  • model (object) – model produced by training

featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
getFeaturesCol()[source]
Returns

The name of the features column

Return type

featuresCol

static getJavaPackage()[source]

Returns package name String.

getLabelCol()[source]
Returns

The name of the label column

Return type

labelCol

getModel()[source]
Returns

model produced by training

Return type

model

labelCol = Param(parent='undefined', name='labelCol', doc='The name of the label column')
model = Param(parent='undefined', name='model', doc='model produced by training')
classmethod read()[source]

Returns an MLReader instance for this class.

setFeaturesCol(value)[source]
Parameters

featuresCol – The name of the features column

setLabelCol(value)[source]
Parameters

labelCol – The name of the label column

setModel(value)[source]
Parameters

model – model produced by training

setParams(featuresCol=None, labelCol=None, model=None)[source]

Set the (keyword only) parameters

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.