synapse.ml.lime package

Submodules

synapse.ml.lime.ImageLIME module

class synapse.ml.lime.ImageLIME.ImageLIME(java_obj=None, cellSize=16.0, inputCol=None, model=None, modifier=130.0, nSamples=900, outputCol=None, predictionCol='prediction', regularization=0.0, samplingFraction=0.3, superpixelCol='superpixels')[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaTransformer

Parameters

cellSize (float) – Number that controls the size of the superpixels
inputCol (object) – The name of the input column
model (object) – Model to try to locally approximate
modifier (float) – Controls the trade-off spatial and color distance
nSamples (int) – The number of samples to generate
outputCol (object) – The name of the output column
predictionCol (object) – prediction column name
regularization (float) – regularization param for the lasso
samplingFraction (float) – The fraction of superpixels to keep on
superpixelCol (object) – The column holding the superpixel decompositions

cellSize = Param(parent='undefined', name='cellSize', doc='Number that controls the size of the superpixels')

getCellSize()[source]

Returns: Number that controls the size of the superpixels
Return type: cellSize

getInputCol()[source]

Returns: The name of the input column
Return type: inputCol

static getJavaPackage()[source]: Returns package name String.

getModel()[source]

Returns: Model to try to locally approximate
Return type: model

getModifier()[source]

Returns: Controls the trade-off spatial and color distance
Return type: modifier

getNSamples()[source]

Returns: The number of samples to generate
Return type: nSamples

getOutputCol()[source]

Returns: The name of the output column
Return type: outputCol

getPredictionCol()[source]

Returns: prediction column name
Return type: predictionCol

getRegularization()[source]

Returns: regularization param for the lasso
Return type: regularization

getSamplingFraction()[source]

Returns: The fraction of superpixels to keep on
Return type: samplingFraction

getSuperpixelCol()[source]

Returns: The column holding the superpixel decompositions
Return type: superpixelCol

inputCol = Param(parent='undefined', name='inputCol', doc='The name of the input column')

model = Param(parent='undefined', name='model', doc='Model to try to locally approximate')

modifier = Param(parent='undefined', name='modifier', doc='Controls the trade-off spatial and color distance')

nSamples = Param(parent='undefined', name='nSamples', doc='The number of samples to generate')

outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')

predictionCol = Param(parent='undefined', name='predictionCol', doc='prediction column name')

classmethod read()[source]: Returns an MLReader instance for this class.

regularization = Param(parent='undefined', name='regularization', doc='regularization param for the lasso')

samplingFraction = Param(parent='undefined', name='samplingFraction', doc='The fraction of superpixels to keep on')

setCellSize(value)[source]

Parameters: cellSize – Number that controls the size of the superpixels

setInputCol(value)[source]

Parameters: inputCol – The name of the input column

setModel(value)[source]

Parameters: model – Model to try to locally approximate

setModifier(value)[source]

Parameters: modifier – Controls the trade-off spatial and color distance

setNSamples(value)[source]

Parameters: nSamples – The number of samples to generate

setOutputCol(value)[source]

Parameters: outputCol – The name of the output column

setParams(cellSize=16.0, inputCol=None, model=None, modifier=130.0, nSamples=900, outputCol=None, predictionCol='prediction', regularization=0.0, samplingFraction=0.3, superpixelCol='superpixels')[source]: Set the (keyword only) parameters

setPredictionCol(value)[source]

Parameters: predictionCol – prediction column name

setRegularization(value)[source]

Parameters: regularization – regularization param for the lasso

setSamplingFraction(value)[source]

Parameters: samplingFraction – The fraction of superpixels to keep on

setSuperpixelCol(value)[source]

Parameters: superpixelCol – The column holding the superpixel decompositions

superpixelCol = Param(parent='undefined', name='superpixelCol', doc='The column holding the superpixel decompositions')

synapse.ml.lime.SuperpixelTransformer module

class synapse.ml.lime.SuperpixelTransformer.SuperpixelTransformer(java_obj=None, cellSize=16.0, inputCol=None, modifier=130.0, outputCol='SuperpixelTransformer_32767c336269_output')[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaTransformer

Parameters

cellSize (float) – Number that controls the size of the superpixels
inputCol (object) – The name of the input column
modifier (float) – Controls the trade-off spatial and color distance
outputCol (object) – The name of the output column

cellSize = Param(parent='undefined', name='cellSize', doc='Number that controls the size of the superpixels')

getCellSize()[source]

Returns: Number that controls the size of the superpixels
Return type: cellSize

getInputCol()[source]

Returns: The name of the input column
Return type: inputCol

static getJavaPackage()[source]: Returns package name String.

getModifier()[source]

Returns: Controls the trade-off spatial and color distance
Return type: modifier

getOutputCol()[source]

Returns: The name of the output column
Return type: outputCol

inputCol = Param(parent='undefined', name='inputCol', doc='The name of the input column')

modifier = Param(parent='undefined', name='modifier', doc='Controls the trade-off spatial and color distance')

outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')

classmethod read()[source]: Returns an MLReader instance for this class.

setCellSize(value)[source]

Parameters: cellSize – Number that controls the size of the superpixels

setInputCol(value)[source]

Parameters: inputCol – The name of the input column

setModifier(value)[source]

Parameters: modifier – Controls the trade-off spatial and color distance

setOutputCol(value)[source]

Parameters: outputCol – The name of the output column

setParams(cellSize=16.0, inputCol=None, modifier=130.0, outputCol='SuperpixelTransformer_32767c336269_output')[source]: Set the (keyword only) parameters

synapse.ml.lime.TabularLIME module

class synapse.ml.lime.TabularLIME.TabularLIME(java_obj=None, inputCol=None, model=None, nSamples=1000, outputCol=None, predictionCol='prediction', regularization=0.0, samplingFraction=0.3)[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaEstimator

Parameters

inputCol (object) – The name of the input column
model (object) – Model to try to locally approximate
nSamples (int) – The number of samples to generate
outputCol (object) – The name of the output column
predictionCol (object) – prediction column name
regularization (float) – regularization param for the lasso
samplingFraction (float) – The fraction of superpixels to keep on

getInputCol()[source]

Returns: The name of the input column
Return type: inputCol

static getJavaPackage()[source]: Returns package name String.

getModel()[source]

Returns: Model to try to locally approximate
Return type: model

getNSamples()[source]

Returns: The number of samples to generate
Return type: nSamples

getOutputCol()[source]

Returns: The name of the output column
Return type: outputCol

getPredictionCol()[source]

Returns: prediction column name
Return type: predictionCol

getRegularization()[source]

Returns: regularization param for the lasso
Return type: regularization

getSamplingFraction()[source]

Returns: The fraction of superpixels to keep on
Return type: samplingFraction

inputCol = Param(parent='undefined', name='inputCol', doc='The name of the input column')

model = Param(parent='undefined', name='model', doc='Model to try to locally approximate')

nSamples = Param(parent='undefined', name='nSamples', doc='The number of samples to generate')

outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')

predictionCol = Param(parent='undefined', name='predictionCol', doc='prediction column name')

classmethod read()[source]: Returns an MLReader instance for this class.

regularization = Param(parent='undefined', name='regularization', doc='regularization param for the lasso')

samplingFraction = Param(parent='undefined', name='samplingFraction', doc='The fraction of superpixels to keep on')

setInputCol(value)[source]

Parameters: inputCol – The name of the input column

setModel(value)[source]

Parameters: model – Model to try to locally approximate

setNSamples(value)[source]

Parameters: nSamples – The number of samples to generate

setOutputCol(value)[source]

Parameters: outputCol – The name of the output column

setParams(inputCol=None, model=None, nSamples=1000, outputCol=None, predictionCol='prediction', regularization=0.0, samplingFraction=0.3)[source]: Set the (keyword only) parameters

setPredictionCol(value)[source]

Parameters: predictionCol – prediction column name

setRegularization(value)[source]

Parameters: regularization – regularization param for the lasso

setSamplingFraction(value)[source]

Parameters: samplingFraction – The fraction of superpixels to keep on

synapse.ml.lime.TabularLIMEModel module

class synapse.ml.lime.TabularLIMEModel.TabularLIMEModel(java_obj=None, columnSTDs=None, inputCol=None, model=None, nSamples=None, outputCol=None, predictionCol='prediction', regularization=None, samplingFraction=None)[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaModel

Parameters

columnSTDs (list) – the standard deviations of each of the columns for perturbation
inputCol (object) – The name of the input column
model (object) – Model to try to locally approximate
nSamples (int) – The number of samples to generate
outputCol (object) – The name of the output column
predictionCol (object) – prediction column name
regularization (float) – regularization param for the lasso
samplingFraction (float) – The fraction of superpixels to keep on

columnSTDs = Param(parent='undefined', name='columnSTDs', doc='the standard deviations of each of the columns for perturbation')

getColumnSTDs()[source]

Returns: the standard deviations of each of the columns for perturbation
Return type: columnSTDs

getInputCol()[source]

Returns: The name of the input column
Return type: inputCol

static getJavaPackage()[source]: Returns package name String.

getModel()[source]

Returns: Model to try to locally approximate
Return type: model

getNSamples()[source]

Returns: The number of samples to generate
Return type: nSamples

getOutputCol()[source]

Returns: The name of the output column
Return type: outputCol

getPredictionCol()[source]

Returns: prediction column name
Return type: predictionCol

getRegularization()[source]

Returns: regularization param for the lasso
Return type: regularization

getSamplingFraction()[source]

Returns: The fraction of superpixels to keep on
Return type: samplingFraction

inputCol = Param(parent='undefined', name='inputCol', doc='The name of the input column')

model = Param(parent='undefined', name='model', doc='Model to try to locally approximate')

nSamples = Param(parent='undefined', name='nSamples', doc='The number of samples to generate')

outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')

predictionCol = Param(parent='undefined', name='predictionCol', doc='prediction column name')

classmethod read()[source]: Returns an MLReader instance for this class.

regularization = Param(parent='undefined', name='regularization', doc='regularization param for the lasso')

samplingFraction = Param(parent='undefined', name='samplingFraction', doc='The fraction of superpixels to keep on')

setColumnSTDs(value)[source]

Parameters: columnSTDs – the standard deviations of each of the columns for perturbation

setInputCol(value)[source]

Parameters: inputCol – The name of the input column

setModel(value)[source]

Parameters: model – Model to try to locally approximate

setNSamples(value)[source]

Parameters: nSamples – The number of samples to generate

setOutputCol(value)[source]

Parameters: outputCol – The name of the output column

setParams(columnSTDs=None, inputCol=None, model=None, nSamples=None, outputCol=None, predictionCol='prediction', regularization=None, samplingFraction=None)[source]: Set the (keyword only) parameters

setPredictionCol(value)[source]

Parameters: predictionCol – prediction column name

setRegularization(value)[source]

Parameters: regularization – regularization param for the lasso

setSamplingFraction(value)[source]

Parameters: samplingFraction – The fraction of superpixels to keep on

synapse.ml.lime.TextLIME module

class synapse.ml.lime.TextLIME.TextLIME(java_obj=None, inputCol=None, model=None, nSamples=1000, outputCol=None, predictionCol='prediction', regularization=0.0, samplingFraction=0.3, tokenCol=None)[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaModel

Parameters

inputCol (object) – The name of the input column
model (object) – Model to try to locally approximate
nSamples (int) – The number of samples to generate
outputCol (object) – The name of the output column
predictionCol (object) – prediction column name
regularization (float) – regularization param for the lasso
samplingFraction (float) – The fraction of superpixels to keep on
tokenCol (object) – The column holding the token

getInputCol()[source]

Returns: The name of the input column
Return type: inputCol

static getJavaPackage()[source]: Returns package name String.

getModel()[source]

Returns: Model to try to locally approximate
Return type: model

getNSamples()[source]

Returns: The number of samples to generate
Return type: nSamples

getOutputCol()[source]

Returns: The name of the output column
Return type: outputCol

getPredictionCol()[source]

Returns: prediction column name
Return type: predictionCol

getRegularization()[source]

Returns: regularization param for the lasso
Return type: regularization

getSamplingFraction()[source]

Returns: The fraction of superpixels to keep on
Return type: samplingFraction

getTokenCol()[source]

Returns: The column holding the token
Return type: tokenCol

inputCol = Param(parent='undefined', name='inputCol', doc='The name of the input column')

model = Param(parent='undefined', name='model', doc='Model to try to locally approximate')

nSamples = Param(parent='undefined', name='nSamples', doc='The number of samples to generate')

outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')

predictionCol = Param(parent='undefined', name='predictionCol', doc='prediction column name')

classmethod read()[source]: Returns an MLReader instance for this class.

regularization = Param(parent='undefined', name='regularization', doc='regularization param for the lasso')

samplingFraction = Param(parent='undefined', name='samplingFraction', doc='The fraction of superpixels to keep on')

setInputCol(value)[source]

Parameters: inputCol – The name of the input column

setModel(value)[source]

Parameters: model – Model to try to locally approximate

setNSamples(value)[source]

Parameters: nSamples – The number of samples to generate

setOutputCol(value)[source]

Parameters: outputCol – The name of the output column

setParams(inputCol=None, model=None, nSamples=1000, outputCol=None, predictionCol='prediction', regularization=0.0, samplingFraction=0.3, tokenCol=None)[source]: Set the (keyword only) parameters

setPredictionCol(value)[source]

Parameters: predictionCol – prediction column name

setRegularization(value)[source]

Parameters: regularization – regularization param for the lasso

setSamplingFraction(value)[source]

Parameters: samplingFraction – The fraction of superpixels to keep on

setTokenCol(value)[source]

Parameters: tokenCol – The column holding the token

tokenCol = Param(parent='undefined', name='tokenCol', doc='The column holding the token')

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.