synapse.ml.automl package

Submodules

synapse.ml.automl.BestModel module

class synapse.ml.automl.BestModel.BestModel(java_obj=None, allModelMetrics=None, bestModel=None, bestModelMetrics=None, rocCurve=None, scoredDataset=None)[source]

Bases: synapse.ml.automl._BestModel._BestModel

getAllModelMetrics()[source]

Returns a table of metrics from all models compared from the evaluation comparison.

getBestModel()[source]

Returns the best model.

getBestModelMetrics()[source]

Returns all of the best model metrics results from the evaluator.

getEvaluationResults()[source]

Returns the ROC curve with TPR, FPR.

getScoredDataset()[source]

Returns scored dataset for the best model.

synapse.ml.automl.FindBestModel module

class synapse.ml.automl.FindBestModel.FindBestModel(java_obj=None, evaluationMetric='accuracy', models=None)[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaEstimator

Parameters
  • evaluationMetric (object) – Metric to evaluate models with

  • models (object) – List of models to be evaluated

evaluationMetric = Param(parent='undefined', name='evaluationMetric', doc='Metric to evaluate models with')
getEvaluationMetric()[source]
Returns

Metric to evaluate models with

Return type

evaluationMetric

static getJavaPackage()[source]

Returns package name String.

getModels()[source]
Returns

List of models to be evaluated

Return type

models

models = Param(parent='undefined', name='models', doc='List of models to be evaluated')
classmethod read()[source]

Returns an MLReader instance for this class.

setEvaluationMetric(value)[source]
Parameters

evaluationMetric – Metric to evaluate models with

setModels(value)[source]
Parameters

models – List of models to be evaluated

setParams(evaluationMetric='accuracy', models=None)[source]

Set the (keyword only) parameters

synapse.ml.automl.HyperparamBuilder module

class synapse.ml.automl.HyperparamBuilder.DiscreteHyperParam(values, seed=0)[source]

Bases: object

Specifies a discrete list of values.

get()[source]
class synapse.ml.automl.HyperparamBuilder.GridSpace(paramValues)[source]

Bases: object

Specifies a predetermined grid of values to search through.

space()[source]
class synapse.ml.automl.HyperparamBuilder.HyperparamBuilder[source]

Bases: object

Specifies the search space for hyperparameters.

addHyperparam(est, param, hyperParam)[source]

Add a hyperparam to the builder

Parameters
  • param (Param) – The param to tune

  • dist (Dist) – Distribution of values

build()[source]

Builds the search space of hyperparameters, returns the map of hyperparameters to search through.

class synapse.ml.automl.HyperparamBuilder.RandomSpace(paramDistributions)[source]

Bases: object

Specifies a random streaming range of values to search through.

space()[source]
class synapse.ml.automl.HyperparamBuilder.RangeHyperParam(min, max, seed=0)[source]

Bases: object

Specifies a range of values.

get()[source]

synapse.ml.automl.TuneHyperparameters module

class synapse.ml.automl.TuneHyperparameters.TuneHyperparameters(java_obj=None, evaluationMetric=None, models=None, numFolds=None, numRuns=None, parallelism=None, paramSpace=None, seed=0)[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaEstimator

Parameters
  • evaluationMetric (object) – Metric to evaluate models with

  • models (object) – Estimators to run

  • numFolds (int) – Number of folds

  • numRuns (int) – Termination criteria for randomized search

  • parallelism (int) – The number of models to run in parallel

  • paramSpace (object) – Parameter space for generating hyperparameters

  • seed (long) – Random number generator seed

evaluationMetric = Param(parent='undefined', name='evaluationMetric', doc='Metric to evaluate models with')
getEvaluationMetric()[source]
Returns

Metric to evaluate models with

Return type

evaluationMetric

static getJavaPackage()[source]

Returns package name String.

getModels()[source]
Returns

Estimators to run

Return type

models

getNumFolds()[source]
Returns

Number of folds

Return type

numFolds

getNumRuns()[source]
Returns

Termination criteria for randomized search

Return type

numRuns

getParallelism()[source]
Returns

The number of models to run in parallel

Return type

parallelism

getParamSpace()[source]
Returns

Parameter space for generating hyperparameters

Return type

paramSpace

getSeed()[source]
Returns

Random number generator seed

Return type

seed

models = Param(parent='undefined', name='models', doc='Estimators to run')
numFolds = Param(parent='undefined', name='numFolds', doc='Number of folds')
numRuns = Param(parent='undefined', name='numRuns', doc='Termination criteria for randomized search')
parallelism = Param(parent='undefined', name='parallelism', doc='The number of models to run in parallel')
paramSpace = Param(parent='undefined', name='paramSpace', doc='Parameter space for generating hyperparameters')
classmethod read()[source]

Returns an MLReader instance for this class.

seed = Param(parent='undefined', name='seed', doc='Random number generator seed')
setEvaluationMetric(value)[source]
Parameters

evaluationMetric – Metric to evaluate models with

setModels(value)[source]
Parameters

models – Estimators to run

setNumFolds(value)[source]
Parameters

numFolds – Number of folds

setNumRuns(value)[source]
Parameters

numRuns – Termination criteria for randomized search

setParallelism(value)[source]
Parameters

parallelism – The number of models to run in parallel

setParamSpace(value)[source]
Parameters

paramSpace – Parameter space for generating hyperparameters

setParams(evaluationMetric=None, models=None, numFolds=None, numRuns=None, parallelism=None, paramSpace=None, seed=0)[source]

Set the (keyword only) parameters

setSeed(value)[source]
Parameters

seed – Random number generator seed

synapse.ml.automl.TuneHyperparametersModel module

class synapse.ml.automl.TuneHyperparametersModel.TuneHyperparametersModel(java_obj=None, bestMetric=None, bestModel=None)[source]

Bases: synapse.ml.automl._TuneHyperparametersModel._TuneHyperparametersModel

getBestModel()[source]

Returns the best model.

getBestModelInfo()[source]

Returns the best model parameter info.

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.