synapse.ml.causal package

Submodules

synapse.ml.causal.DoubleMLEstimator module

class synapse.ml.causal.DoubleMLEstimator.DoubleMLEstimator(java_obj=None, confidenceLevel=0.975, featuresCol=None, maxIter=1, outcomeCol=None, outcomeModel=None, parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, weightCol=None)[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaEstimator

Parameters
  • confidenceLevel (float) – confidence level, default value is 0.975

  • featuresCol (str) – The name of the features column

  • maxIter (int) – maximum number of iterations (>= 0)

  • outcomeCol (str) – outcome column

  • outcomeModel (object) – outcome model to run

  • parallelism (int) – the number of threads to use when running parallel algorithms

  • sampleSplitRatio (list) – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

  • treatmentCol (str) – treatment column

  • treatmentModel (object) – treatment model to run

  • weightCol (str) – The name of the weight column

confidenceLevel = Param(parent='undefined', name='confidenceLevel', doc='confidence level, default value is 0.975')
featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
getConfidenceLevel()[source]
Returns

confidence level, default value is 0.975

Return type

confidenceLevel

getFeaturesCol()[source]
Returns

The name of the features column

Return type

featuresCol

static getJavaPackage()[source]

Returns package name String.

getMaxIter()[source]
Returns

maximum number of iterations (>= 0)

Return type

maxIter

getOutcomeCol()[source]
Returns

outcome column

Return type

outcomeCol

getOutcomeModel()[source]
Returns

outcome model to run

Return type

outcomeModel

getParallelism()[source]
Returns

the number of threads to use when running parallel algorithms

Return type

parallelism

getSampleSplitRatio()[source]
Returns

Sample split ratio for cross-fitting. Default: [0.5, 0.5].

Return type

sampleSplitRatio

getTreatmentCol()[source]
Returns

treatment column

Return type

treatmentCol

getTreatmentModel()[source]
Returns

treatment model to run

Return type

treatmentModel

getWeightCol()[source]
Returns

The name of the weight column

Return type

weightCol

maxIter = Param(parent='undefined', name='maxIter', doc='maximum number of iterations (>= 0)')
outcomeCol = Param(parent='undefined', name='outcomeCol', doc='outcome column')
outcomeModel = Param(parent='undefined', name='outcomeModel', doc='outcome model to run')
parallelism = Param(parent='undefined', name='parallelism', doc='the number of threads to use when running parallel algorithms')
classmethod read()[source]

Returns an MLReader instance for this class.

sampleSplitRatio = Param(parent='undefined', name='sampleSplitRatio', doc='Sample split ratio for cross-fitting. Default: [0.5, 0.5].')
setConfidenceLevel(value)[source]
Parameters

confidenceLevel – confidence level, default value is 0.975

setFeaturesCol(value)[source]
Parameters

featuresCol – The name of the features column

setMaxIter(value)[source]
Parameters

maxIter – maximum number of iterations (>= 0)

setOutcomeCol(value)[source]
Parameters

outcomeCol – outcome column

setOutcomeModel(value)[source]
Parameters

outcomeModel – outcome model to run

setParallelism(value)[source]
Parameters

parallelism – the number of threads to use when running parallel algorithms

setParams(confidenceLevel=0.975, featuresCol=None, maxIter=1, outcomeCol=None, outcomeModel=None, parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, weightCol=None)[source]

Set the (keyword only) parameters

setSampleSplitRatio(value)[source]
Parameters

sampleSplitRatio – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

setTreatmentCol(value)[source]
Parameters

treatmentCol – treatment column

setTreatmentModel(value)[source]
Parameters

treatmentModel – treatment model to run

setWeightCol(value)[source]
Parameters

weightCol – The name of the weight column

treatmentCol = Param(parent='undefined', name='treatmentCol', doc='treatment column')
treatmentModel = Param(parent='undefined', name='treatmentModel', doc='treatment model to run')
weightCol = Param(parent='undefined', name='weightCol', doc='The name of the weight column')

synapse.ml.causal.DoubleMLModel module

class synapse.ml.causal.DoubleMLModel.DoubleMLModel(java_obj=None, confidenceLevel=0.975, featuresCol=None, maxIter=1, outcomeCol=None, outcomeModel=None, parallelism=10, rawTreatmentEffects=None, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, weightCol=None)[source]

Bases: synapse.ml.causal._DoubleMLModel._DoubleMLModel

getAvgTreatmentEffect()[source]
getConfidenceInterval()[source]
getPValue()[source]

synapse.ml.causal.ResidualTransformer module

class synapse.ml.causal.ResidualTransformer.ResidualTransformer(java_obj=None, classIndex=1, observedCol='label', outputCol='residual', predictedCol='prediction')[source]

Bases: synapse.ml.core.schema.Utils.ComplexParamsMixin, pyspark.ml.util.JavaMLReadable, pyspark.ml.util.JavaMLWritable, pyspark.ml.wrapper.JavaTransformer

Parameters
  • classIndex (int) – The index of the class to compute residual for classification outputs. Default value is 1.

  • observedCol (str) – observed data (label column)

  • outputCol (str) – The name of the output column

  • predictedCol (str) – predicted data (prediction or probability columns

classIndex = Param(parent='undefined', name='classIndex', doc='The index of the class to compute residual for classification outputs. Default value is 1.')
getClassIndex()[source]
Returns

The index of the class to compute residual for classification outputs. Default value is 1.

Return type

classIndex

static getJavaPackage()[source]

Returns package name String.

getObservedCol()[source]
Returns

observed data (label column)

Return type

observedCol

getOutputCol()[source]
Returns

The name of the output column

Return type

outputCol

getPredictedCol()[source]
Returns

predicted data (prediction or probability columns

Return type

predictedCol

observedCol = Param(parent='undefined', name='observedCol', doc='observed data (label column)')
outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')
predictedCol = Param(parent='undefined', name='predictedCol', doc='predicted data (prediction or probability columns')
classmethod read()[source]

Returns an MLReader instance for this class.

setClassIndex(value)[source]
Parameters

classIndex – The index of the class to compute residual for classification outputs. Default value is 1.

setObservedCol(value)[source]
Parameters

observedCol – observed data (label column)

setOutputCol(value)[source]
Parameters

outputCol – The name of the output column

setParams(classIndex=1, observedCol='label', outputCol='residual', predictedCol='prediction')[source]

Set the (keyword only) parameters

setPredictedCol(value)[source]
Parameters

predictedCol – predicted data (prediction or probability columns

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.