synapse.ml.causal package

Submodules

synapse.ml.causal.DiffInDiffEstimator module

class synapse.ml.causal.DiffInDiffEstimator.DiffInDiffEstimator(java_obj=None, outcomeCol=None, postTreatmentCol=None, treatmentCol=None)[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaEstimator

Parameters:
  • outcomeCol (str) – outcome column

  • postTreatmentCol (str) – post treatment indicator column

  • treatmentCol (str) – treatment column

static getJavaPackage()[source]

Returns package name String.

getOutcomeCol()[source]
Returns:

outcome column

Return type:

outcomeCol

getPostTreatmentCol()[source]
Returns:

post treatment indicator column

Return type:

postTreatmentCol

getTreatmentCol()[source]
Returns:

treatment column

Return type:

treatmentCol

outcomeCol = Param(parent='undefined', name='outcomeCol', doc='outcome column')
postTreatmentCol = Param(parent='undefined', name='postTreatmentCol', doc='post treatment indicator column')
classmethod read()[source]

Returns an MLReader instance for this class.

setOutcomeCol(value)[source]
Parameters:

outcomeCol – outcome column

setParams(outcomeCol=None, postTreatmentCol=None, treatmentCol=None)[source]

Set the (keyword only) parameters

setPostTreatmentCol(value)[source]
Parameters:

postTreatmentCol – post treatment indicator column

setTreatmentCol(value)[source]
Parameters:

treatmentCol – treatment column

treatmentCol = Param(parent='undefined', name='treatmentCol', doc='treatment column')

synapse.ml.causal.DiffInDiffModel module

class synapse.ml.causal.DiffInDiffModel.DiffInDiffModel(java_obj=None)[source]

Bases: _DiffInDiffModel

synapse.ml.causal.DoubleMLEstimator module

class synapse.ml.causal.DoubleMLEstimator.DoubleMLEstimator(java_obj=None, confidenceLevel=0.975, featuresCol=None, maxIter=1, outcomeCol=None, outcomeModel=None, parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, weightCol=None)[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaEstimator

Parameters:
  • confidenceLevel (float) – confidence level, default value is 0.975

  • featuresCol (str) – The name of the features column

  • maxIter (int) – maximum number of iterations (>= 0)

  • outcomeCol (str) – outcome column

  • outcomeModel (object) – outcome model to run

  • parallelism (int) – the number of threads to use when running parallel algorithms

  • sampleSplitRatio (list) – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

  • treatmentCol (str) – treatment column

  • treatmentModel (object) – treatment model to run

  • weightCol (str) – The name of the weight column

confidenceLevel = Param(parent='undefined', name='confidenceLevel', doc='confidence level, default value is 0.975')
featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
getConfidenceLevel()[source]
Returns:

confidence level, default value is 0.975

Return type:

confidenceLevel

getFeaturesCol()[source]
Returns:

The name of the features column

Return type:

featuresCol

static getJavaPackage()[source]

Returns package name String.

getMaxIter()[source]
Returns:

maximum number of iterations (>= 0)

Return type:

maxIter

getOutcomeCol()[source]
Returns:

outcome column

Return type:

outcomeCol

getOutcomeModel()[source]
Returns:

outcome model to run

Return type:

outcomeModel

getParallelism()[source]
Returns:

the number of threads to use when running parallel algorithms

Return type:

parallelism

getSampleSplitRatio()[source]
Returns:

Sample split ratio for cross-fitting. Default: [0.5, 0.5].

Return type:

sampleSplitRatio

getTreatmentCol()[source]
Returns:

treatment column

Return type:

treatmentCol

getTreatmentModel()[source]
Returns:

treatment model to run

Return type:

treatmentModel

getWeightCol()[source]
Returns:

The name of the weight column

Return type:

weightCol

maxIter = Param(parent='undefined', name='maxIter', doc='maximum number of iterations (>= 0)')
outcomeCol = Param(parent='undefined', name='outcomeCol', doc='outcome column')
outcomeModel = Param(parent='undefined', name='outcomeModel', doc='outcome model to run')
parallelism = Param(parent='undefined', name='parallelism', doc='the number of threads to use when running parallel algorithms')
classmethod read()[source]

Returns an MLReader instance for this class.

sampleSplitRatio = Param(parent='undefined', name='sampleSplitRatio', doc='Sample split ratio for cross-fitting. Default: [0.5, 0.5].')
setConfidenceLevel(value)[source]
Parameters:

confidenceLevel – confidence level, default value is 0.975

setFeaturesCol(value)[source]
Parameters:

featuresCol – The name of the features column

setMaxIter(value)[source]
Parameters:

maxIter – maximum number of iterations (>= 0)

setOutcomeCol(value)[source]
Parameters:

outcomeCol – outcome column

setOutcomeModel(value)[source]
Parameters:

outcomeModel – outcome model to run

setParallelism(value)[source]
Parameters:

parallelism – the number of threads to use when running parallel algorithms

setParams(confidenceLevel=0.975, featuresCol=None, maxIter=1, outcomeCol=None, outcomeModel=None, parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, weightCol=None)[source]

Set the (keyword only) parameters

setSampleSplitRatio(value)[source]
Parameters:

sampleSplitRatio – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

setTreatmentCol(value)[source]
Parameters:

treatmentCol – treatment column

setTreatmentModel(value)[source]
Parameters:

treatmentModel – treatment model to run

setWeightCol(value)[source]
Parameters:

weightCol – The name of the weight column

treatmentCol = Param(parent='undefined', name='treatmentCol', doc='treatment column')
treatmentModel = Param(parent='undefined', name='treatmentModel', doc='treatment model to run')
weightCol = Param(parent='undefined', name='weightCol', doc='The name of the weight column')

synapse.ml.causal.DoubleMLModel module

class synapse.ml.causal.DoubleMLModel.DoubleMLModel(java_obj=None, confidenceLevel=0.975, featuresCol=None, maxIter=1, outcomeCol=None, outcomeModel=None, parallelism=10, rawTreatmentEffects=None, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, weightCol=None)[source]

Bases: _DoubleMLModel

getAvgTreatmentEffect()[source]
getConfidenceInterval()[source]
getPValue()[source]

synapse.ml.causal.OrthoForestDMLEstimator module

class synapse.ml.causal.OrthoForestDMLEstimator.OrthoForestDMLEstimator(java_obj=None, confidenceLevel=0.975, confounderVecCol='XW', featuresCol=None, heterogeneityVecCol='X', maxDepth=5, maxIter=1, minSamplesLeaf=10, numTrees=20, outcomeCol=None, outcomeModel=None, outcomeResidualCol='OutcomeResidual', outputCol='EffectAverage', outputHighCol='EffectUpperBound', outputLowCol='EffectLowerBound', parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, treatmentResidualCol='TreatmentResidual', weightCol=None)[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaEstimator

Parameters:
  • confidenceLevel (float) – confidence level, default value is 0.975

  • confounderVecCol (str) – Confounders to control for

  • featuresCol (str) – The name of the features column

  • heterogeneityVecCol (str) – Vector to divide the treatment by

  • maxDepth (int) – Max Depth of Tree

  • maxIter (int) – maximum number of iterations (>= 0)

  • minSamplesLeaf (int) – Max Depth of Tree

  • numTrees (int) – Number of trees

  • outcomeCol (str) – outcome column

  • outcomeModel (object) – outcome model to run

  • outcomeResidualCol (str) – Outcome Residual Column

  • outputCol (str) – The name of the output column

  • outputHighCol (str) – Output Confidence Interval Low

  • outputLowCol (str) – Output Confidence Interval Low

  • parallelism (int) – the number of threads to use when running parallel algorithms

  • sampleSplitRatio (list) – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

  • treatmentCol (str) – treatment column

  • treatmentModel (object) – treatment model to run

  • treatmentResidualCol (str) – Treatment Residual Column

  • weightCol (str) – The name of the weight column

confidenceLevel = Param(parent='undefined', name='confidenceLevel', doc='confidence level, default value is 0.975')
confounderVecCol = Param(parent='undefined', name='confounderVecCol', doc='Confounders to control for')
featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
getConfidenceLevel()[source]
Returns:

confidence level, default value is 0.975

Return type:

confidenceLevel

getConfounderVecCol()[source]
Returns:

Confounders to control for

Return type:

confounderVecCol

getFeaturesCol()[source]
Returns:

The name of the features column

Return type:

featuresCol

getHeterogeneityVecCol()[source]
Returns:

Vector to divide the treatment by

Return type:

heterogeneityVecCol

static getJavaPackage()[source]

Returns package name String.

getMaxDepth()[source]
Returns:

Max Depth of Tree

Return type:

maxDepth

getMaxIter()[source]
Returns:

maximum number of iterations (>= 0)

Return type:

maxIter

getMinSamplesLeaf()[source]
Returns:

Max Depth of Tree

Return type:

minSamplesLeaf

getNumTrees()[source]
Returns:

Number of trees

Return type:

numTrees

getOutcomeCol()[source]
Returns:

outcome column

Return type:

outcomeCol

getOutcomeModel()[source]
Returns:

outcome model to run

Return type:

outcomeModel

getOutcomeResidualCol()[source]
Returns:

Outcome Residual Column

Return type:

outcomeResidualCol

getOutputCol()[source]
Returns:

The name of the output column

Return type:

outputCol

getOutputHighCol()[source]
Returns:

Output Confidence Interval Low

Return type:

outputHighCol

getOutputLowCol()[source]
Returns:

Output Confidence Interval Low

Return type:

outputLowCol

getParallelism()[source]
Returns:

the number of threads to use when running parallel algorithms

Return type:

parallelism

getSampleSplitRatio()[source]
Returns:

Sample split ratio for cross-fitting. Default: [0.5, 0.5].

Return type:

sampleSplitRatio

getTreatmentCol()[source]
Returns:

treatment column

Return type:

treatmentCol

getTreatmentModel()[source]
Returns:

treatment model to run

Return type:

treatmentModel

getTreatmentResidualCol()[source]
Returns:

Treatment Residual Column

Return type:

treatmentResidualCol

getWeightCol()[source]
Returns:

The name of the weight column

Return type:

weightCol

heterogeneityVecCol = Param(parent='undefined', name='heterogeneityVecCol', doc='Vector to divide the treatment by')
maxDepth = Param(parent='undefined', name='maxDepth', doc='Max Depth of Tree')
maxIter = Param(parent='undefined', name='maxIter', doc='maximum number of iterations (>= 0)')
minSamplesLeaf = Param(parent='undefined', name='minSamplesLeaf', doc='Max Depth of Tree')
numTrees = Param(parent='undefined', name='numTrees', doc='Number of trees')
outcomeCol = Param(parent='undefined', name='outcomeCol', doc='outcome column')
outcomeModel = Param(parent='undefined', name='outcomeModel', doc='outcome model to run')
outcomeResidualCol = Param(parent='undefined', name='outcomeResidualCol', doc='Outcome Residual Column')
outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')
outputHighCol = Param(parent='undefined', name='outputHighCol', doc='Output Confidence Interval Low')
outputLowCol = Param(parent='undefined', name='outputLowCol', doc='Output Confidence Interval Low')
parallelism = Param(parent='undefined', name='parallelism', doc='the number of threads to use when running parallel algorithms')
classmethod read()[source]

Returns an MLReader instance for this class.

sampleSplitRatio = Param(parent='undefined', name='sampleSplitRatio', doc='Sample split ratio for cross-fitting. Default: [0.5, 0.5].')
setConfidenceLevel(value)[source]
Parameters:

confidenceLevel – confidence level, default value is 0.975

setConfounderVecCol(value)[source]
Parameters:

confounderVecCol – Confounders to control for

setFeaturesCol(value)[source]
Parameters:

featuresCol – The name of the features column

setHeterogeneityVecCol(value)[source]
Parameters:

heterogeneityVecCol – Vector to divide the treatment by

setMaxDepth(value)[source]
Parameters:

maxDepth – Max Depth of Tree

setMaxIter(value)[source]
Parameters:

maxIter – maximum number of iterations (>= 0)

setMinSamplesLeaf(value)[source]
Parameters:

minSamplesLeaf – Max Depth of Tree

setNumTrees(value)[source]
Parameters:

numTrees – Number of trees

setOutcomeCol(value)[source]
Parameters:

outcomeCol – outcome column

setOutcomeModel(value)[source]
Parameters:

outcomeModel – outcome model to run

setOutcomeResidualCol(value)[source]
Parameters:

outcomeResidualCol – Outcome Residual Column

setOutputCol(value)[source]
Parameters:

outputCol – The name of the output column

setOutputHighCol(value)[source]
Parameters:

outputHighCol – Output Confidence Interval Low

setOutputLowCol(value)[source]
Parameters:

outputLowCol – Output Confidence Interval Low

setParallelism(value)[source]
Parameters:

parallelism – the number of threads to use when running parallel algorithms

setParams(confidenceLevel=0.975, confounderVecCol='XW', featuresCol=None, heterogeneityVecCol='X', maxDepth=5, maxIter=1, minSamplesLeaf=10, numTrees=20, outcomeCol=None, outcomeModel=None, outcomeResidualCol='OutcomeResidual', outputCol='EffectAverage', outputHighCol='EffectUpperBound', outputLowCol='EffectLowerBound', parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, treatmentResidualCol='TreatmentResidual', weightCol=None)[source]

Set the (keyword only) parameters

setSampleSplitRatio(value)[source]
Parameters:

sampleSplitRatio – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

setTreatmentCol(value)[source]
Parameters:

treatmentCol – treatment column

setTreatmentModel(value)[source]
Parameters:

treatmentModel – treatment model to run

setTreatmentResidualCol(value)[source]
Parameters:

treatmentResidualCol – Treatment Residual Column

setWeightCol(value)[source]
Parameters:

weightCol – The name of the weight column

treatmentCol = Param(parent='undefined', name='treatmentCol', doc='treatment column')
treatmentModel = Param(parent='undefined', name='treatmentModel', doc='treatment model to run')
treatmentResidualCol = Param(parent='undefined', name='treatmentResidualCol', doc='Treatment Residual Column')
weightCol = Param(parent='undefined', name='weightCol', doc='The name of the weight column')

synapse.ml.causal.OrthoForestDMLModel module

class synapse.ml.causal.OrthoForestDMLModel.OrthoForestDMLModel(java_obj=None, confidenceLevel=0.975, confounderVecCol='XW', featuresCol=None, forest=None, heterogeneityVecCol='X', maxDepth=5, maxIter=1, minSamplesLeaf=10, numTrees=20, outcomeCol=None, outcomeModel=None, outcomeResidualCol='OutcomeResidual', outputCol='EffectAverage', outputHighCol='EffectUpperBound', outputLowCol='EffectLowerBound', parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, treatmentResidualCol='TreatmentResidual', weightCol=None)[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaModel

Parameters:
  • confidenceLevel (float) – confidence level, default value is 0.975

  • confounderVecCol (str) – Confounders to control for

  • featuresCol (str) – The name of the features column

  • forest (object) – Forest Trees produced in Ortho Forest DML Estimator

  • heterogeneityVecCol (str) – Vector to divide the treatment by

  • maxDepth (int) – Max Depth of Tree

  • maxIter (int) – maximum number of iterations (>= 0)

  • minSamplesLeaf (int) – Max Depth of Tree

  • numTrees (int) – Number of trees

  • outcomeCol (str) – outcome column

  • outcomeModel (object) – outcome model to run

  • outcomeResidualCol (str) – Outcome Residual Column

  • outputCol (str) – The name of the output column

  • outputHighCol (str) – Output Confidence Interval Low

  • outputLowCol (str) – Output Confidence Interval Low

  • parallelism (int) – the number of threads to use when running parallel algorithms

  • sampleSplitRatio (list) – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

  • treatmentCol (str) – treatment column

  • treatmentModel (object) – treatment model to run

  • treatmentResidualCol (str) – Treatment Residual Column

  • weightCol (str) – The name of the weight column

confidenceLevel = Param(parent='undefined', name='confidenceLevel', doc='confidence level, default value is 0.975')
confounderVecCol = Param(parent='undefined', name='confounderVecCol', doc='Confounders to control for')
featuresCol = Param(parent='undefined', name='featuresCol', doc='The name of the features column')
forest = Param(parent='undefined', name='forest', doc='Forest Trees produced in Ortho Forest DML Estimator')
getConfidenceLevel()[source]
Returns:

confidence level, default value is 0.975

Return type:

confidenceLevel

getConfounderVecCol()[source]
Returns:

Confounders to control for

Return type:

confounderVecCol

getFeaturesCol()[source]
Returns:

The name of the features column

Return type:

featuresCol

getForest()[source]
Returns:

Forest Trees produced in Ortho Forest DML Estimator

Return type:

forest

getHeterogeneityVecCol()[source]
Returns:

Vector to divide the treatment by

Return type:

heterogeneityVecCol

static getJavaPackage()[source]

Returns package name String.

getMaxDepth()[source]
Returns:

Max Depth of Tree

Return type:

maxDepth

getMaxIter()[source]
Returns:

maximum number of iterations (>= 0)

Return type:

maxIter

getMinSamplesLeaf()[source]
Returns:

Max Depth of Tree

Return type:

minSamplesLeaf

getNumTrees()[source]
Returns:

Number of trees

Return type:

numTrees

getOutcomeCol()[source]
Returns:

outcome column

Return type:

outcomeCol

getOutcomeModel()[source]
Returns:

outcome model to run

Return type:

outcomeModel

getOutcomeResidualCol()[source]
Returns:

Outcome Residual Column

Return type:

outcomeResidualCol

getOutputCol()[source]
Returns:

The name of the output column

Return type:

outputCol

getOutputHighCol()[source]
Returns:

Output Confidence Interval Low

Return type:

outputHighCol

getOutputLowCol()[source]
Returns:

Output Confidence Interval Low

Return type:

outputLowCol

getParallelism()[source]
Returns:

the number of threads to use when running parallel algorithms

Return type:

parallelism

getSampleSplitRatio()[source]
Returns:

Sample split ratio for cross-fitting. Default: [0.5, 0.5].

Return type:

sampleSplitRatio

getTreatmentCol()[source]
Returns:

treatment column

Return type:

treatmentCol

getTreatmentModel()[source]
Returns:

treatment model to run

Return type:

treatmentModel

getTreatmentResidualCol()[source]
Returns:

Treatment Residual Column

Return type:

treatmentResidualCol

getWeightCol()[source]
Returns:

The name of the weight column

Return type:

weightCol

heterogeneityVecCol = Param(parent='undefined', name='heterogeneityVecCol', doc='Vector to divide the treatment by')
maxDepth = Param(parent='undefined', name='maxDepth', doc='Max Depth of Tree')
maxIter = Param(parent='undefined', name='maxIter', doc='maximum number of iterations (>= 0)')
minSamplesLeaf = Param(parent='undefined', name='minSamplesLeaf', doc='Max Depth of Tree')
numTrees = Param(parent='undefined', name='numTrees', doc='Number of trees')
outcomeCol = Param(parent='undefined', name='outcomeCol', doc='outcome column')
outcomeModel = Param(parent='undefined', name='outcomeModel', doc='outcome model to run')
outcomeResidualCol = Param(parent='undefined', name='outcomeResidualCol', doc='Outcome Residual Column')
outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')
outputHighCol = Param(parent='undefined', name='outputHighCol', doc='Output Confidence Interval Low')
outputLowCol = Param(parent='undefined', name='outputLowCol', doc='Output Confidence Interval Low')
parallelism = Param(parent='undefined', name='parallelism', doc='the number of threads to use when running parallel algorithms')
classmethod read()[source]

Returns an MLReader instance for this class.

sampleSplitRatio = Param(parent='undefined', name='sampleSplitRatio', doc='Sample split ratio for cross-fitting. Default: [0.5, 0.5].')
setConfidenceLevel(value)[source]
Parameters:

confidenceLevel – confidence level, default value is 0.975

setConfounderVecCol(value)[source]
Parameters:

confounderVecCol – Confounders to control for

setFeaturesCol(value)[source]
Parameters:

featuresCol – The name of the features column

setForest(value)[source]
Parameters:

forest – Forest Trees produced in Ortho Forest DML Estimator

setHeterogeneityVecCol(value)[source]
Parameters:

heterogeneityVecCol – Vector to divide the treatment by

setMaxDepth(value)[source]
Parameters:

maxDepth – Max Depth of Tree

setMaxIter(value)[source]
Parameters:

maxIter – maximum number of iterations (>= 0)

setMinSamplesLeaf(value)[source]
Parameters:

minSamplesLeaf – Max Depth of Tree

setNumTrees(value)[source]
Parameters:

numTrees – Number of trees

setOutcomeCol(value)[source]
Parameters:

outcomeCol – outcome column

setOutcomeModel(value)[source]
Parameters:

outcomeModel – outcome model to run

setOutcomeResidualCol(value)[source]
Parameters:

outcomeResidualCol – Outcome Residual Column

setOutputCol(value)[source]
Parameters:

outputCol – The name of the output column

setOutputHighCol(value)[source]
Parameters:

outputHighCol – Output Confidence Interval Low

setOutputLowCol(value)[source]
Parameters:

outputLowCol – Output Confidence Interval Low

setParallelism(value)[source]
Parameters:

parallelism – the number of threads to use when running parallel algorithms

setParams(confidenceLevel=0.975, confounderVecCol='XW', featuresCol=None, forest=None, heterogeneityVecCol='X', maxDepth=5, maxIter=1, minSamplesLeaf=10, numTrees=20, outcomeCol=None, outcomeModel=None, outcomeResidualCol='OutcomeResidual', outputCol='EffectAverage', outputHighCol='EffectUpperBound', outputLowCol='EffectLowerBound', parallelism=10, sampleSplitRatio=[0.5, 0.5], treatmentCol=None, treatmentModel=None, treatmentResidualCol='TreatmentResidual', weightCol=None)[source]

Set the (keyword only) parameters

setSampleSplitRatio(value)[source]
Parameters:

sampleSplitRatio – Sample split ratio for cross-fitting. Default: [0.5, 0.5].

setTreatmentCol(value)[source]
Parameters:

treatmentCol – treatment column

setTreatmentModel(value)[source]
Parameters:

treatmentModel – treatment model to run

setTreatmentResidualCol(value)[source]
Parameters:

treatmentResidualCol – Treatment Residual Column

setWeightCol(value)[source]
Parameters:

weightCol – The name of the weight column

treatmentCol = Param(parent='undefined', name='treatmentCol', doc='treatment column')
treatmentModel = Param(parent='undefined', name='treatmentModel', doc='treatment model to run')
treatmentResidualCol = Param(parent='undefined', name='treatmentResidualCol', doc='Treatment Residual Column')
weightCol = Param(parent='undefined', name='weightCol', doc='The name of the weight column')

synapse.ml.causal.OrthoForestVariableTransformer module

class synapse.ml.causal.OrthoForestVariableTransformer.OrthoForestVariableTransformer(java_obj=None, outcomeResidualCol='OResid', outputCol='_tmp_tsOutcome', treatmentResidualCol='TResid', weightsCol='_tmp_twOutcome')[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaTransformer

Parameters:
  • outcomeResidualCol (str) – Outcome Residual Col

  • outputCol (str) – The name of the output column

  • treatmentResidualCol (str) – Treatment Residual Col

  • weightsCol (str) – Weights Col

static getJavaPackage()[source]

Returns package name String.

getOutcomeResidualCol()[source]
Returns:

Outcome Residual Col

Return type:

outcomeResidualCol

getOutputCol()[source]
Returns:

The name of the output column

Return type:

outputCol

getTreatmentResidualCol()[source]
Returns:

Treatment Residual Col

Return type:

treatmentResidualCol

getWeightsCol()[source]
Returns:

Weights Col

Return type:

weightsCol

outcomeResidualCol = Param(parent='undefined', name='outcomeResidualCol', doc='Outcome Residual Col')
outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')
classmethod read()[source]

Returns an MLReader instance for this class.

setOutcomeResidualCol(value)[source]
Parameters:

outcomeResidualCol – Outcome Residual Col

setOutputCol(value)[source]
Parameters:

outputCol – The name of the output column

setParams(outcomeResidualCol='OResid', outputCol='_tmp_tsOutcome', treatmentResidualCol='TResid', weightsCol='_tmp_twOutcome')[source]

Set the (keyword only) parameters

setTreatmentResidualCol(value)[source]
Parameters:

treatmentResidualCol – Treatment Residual Col

setWeightsCol(value)[source]
Parameters:

weightsCol – Weights Col

treatmentResidualCol = Param(parent='undefined', name='treatmentResidualCol', doc='Treatment Residual Col')
weightsCol = Param(parent='undefined', name='weightsCol', doc='Weights Col')

synapse.ml.causal.ResidualTransformer module

class synapse.ml.causal.ResidualTransformer.ResidualTransformer(java_obj=None, classIndex=1, observedCol='label', outputCol='residual', predictedCol='prediction')[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaTransformer

Parameters:
  • classIndex (int) – The index of the class to compute residual for classification outputs. Default value is 1.

  • observedCol (str) – observed data (label column)

  • outputCol (str) – The name of the output column

  • predictedCol (str) – predicted data (prediction or probability columns

classIndex = Param(parent='undefined', name='classIndex', doc='The index of the class to compute residual for classification outputs. Default value is 1.')
getClassIndex()[source]
Returns:

The index of the class to compute residual for classification outputs. Default value is 1.

Return type:

classIndex

static getJavaPackage()[source]

Returns package name String.

getObservedCol()[source]
Returns:

observed data (label column)

Return type:

observedCol

getOutputCol()[source]
Returns:

The name of the output column

Return type:

outputCol

getPredictedCol()[source]
Returns:

predicted data (prediction or probability columns

Return type:

predictedCol

observedCol = Param(parent='undefined', name='observedCol', doc='observed data (label column)')
outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')
predictedCol = Param(parent='undefined', name='predictedCol', doc='predicted data (prediction or probability columns')
classmethod read()[source]

Returns an MLReader instance for this class.

setClassIndex(value)[source]
Parameters:

classIndex – The index of the class to compute residual for classification outputs. Default value is 1.

setObservedCol(value)[source]
Parameters:

observedCol – observed data (label column)

setOutputCol(value)[source]
Parameters:

outputCol – The name of the output column

setParams(classIndex=1, observedCol='label', outputCol='residual', predictedCol='prediction')[source]

Set the (keyword only) parameters

setPredictedCol(value)[source]
Parameters:

predictedCol – predicted data (prediction or probability columns

synapse.ml.causal.SyntheticControlEstimator module

class synapse.ml.causal.SyntheticControlEstimator.SyntheticControlEstimator(java_obj=None, epsilon=1e-10, handleMissingOutcome='zero', localSolverThreshold=1000000, maxIter=100, numIterNoChange=None, outcomeCol=None, postTreatmentCol=None, stepSize=1.0, timeCol=None, tol=0.001, treatmentCol=None, unitCol=None)[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaEstimator

Parameters:
  • epsilon (float) – This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.

  • handleMissingOutcome (str) – How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)

  • localSolverThreshold (long) – threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.

  • maxIter (int) – maximum number of iterations (>= 0)

  • numIterNoChange (int) – Early termination when number of iterations without change reached.

  • outcomeCol (str) – outcome column

  • postTreatmentCol (str) – post treatment indicator column

  • stepSize (float) – Step size to be used for each iteration of optimization (> 0)

  • timeCol (str) – Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.

  • tol (float) – the convergence tolerance for iterative algorithms (>= 0)

  • treatmentCol (str) – treatment column

  • unitCol (str) – Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.

epsilon = Param(parent='undefined', name='epsilon', doc='This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.')
getEpsilon()[source]
Returns:

This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.

Return type:

epsilon

getHandleMissingOutcome()[source]
Returns:

How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)

Return type:

handleMissingOutcome

static getJavaPackage()[source]

Returns package name String.

getLocalSolverThreshold()[source]
Returns:

threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.

Return type:

localSolverThreshold

getMaxIter()[source]
Returns:

maximum number of iterations (>= 0)

Return type:

maxIter

getNumIterNoChange()[source]
Returns:

Early termination when number of iterations without change reached.

Return type:

numIterNoChange

getOutcomeCol()[source]
Returns:

outcome column

Return type:

outcomeCol

getPostTreatmentCol()[source]
Returns:

post treatment indicator column

Return type:

postTreatmentCol

getStepSize()[source]
Returns:

Step size to be used for each iteration of optimization (> 0)

Return type:

stepSize

getTimeCol()[source]
Returns:

Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.

Return type:

timeCol

getTol()[source]
Returns:

the convergence tolerance for iterative algorithms (>= 0)

Return type:

tol

getTreatmentCol()[source]
Returns:

treatment column

Return type:

treatmentCol

getUnitCol()[source]
Returns:

Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.

Return type:

unitCol

handleMissingOutcome = Param(parent='undefined', name='handleMissingOutcome', doc='How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)')
localSolverThreshold = Param(parent='undefined', name='localSolverThreshold', doc='threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.')
maxIter = Param(parent='undefined', name='maxIter', doc='maximum number of iterations (>= 0)')
numIterNoChange = Param(parent='undefined', name='numIterNoChange', doc='Early termination when number of iterations without change reached.')
outcomeCol = Param(parent='undefined', name='outcomeCol', doc='outcome column')
postTreatmentCol = Param(parent='undefined', name='postTreatmentCol', doc='post treatment indicator column')
classmethod read()[source]

Returns an MLReader instance for this class.

setEpsilon(value)[source]
Parameters:

epsilon – This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.

setHandleMissingOutcome(value)[source]
Parameters:

handleMissingOutcome – How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)

setLocalSolverThreshold(value)[source]
Parameters:

localSolverThreshold – threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.

setMaxIter(value)[source]
Parameters:

maxIter – maximum number of iterations (>= 0)

setNumIterNoChange(value)[source]
Parameters:

numIterNoChange – Early termination when number of iterations without change reached.

setOutcomeCol(value)[source]
Parameters:

outcomeCol – outcome column

setParams(epsilon=1e-10, handleMissingOutcome='zero', localSolverThreshold=1000000, maxIter=100, numIterNoChange=None, outcomeCol=None, postTreatmentCol=None, stepSize=1.0, timeCol=None, tol=0.001, treatmentCol=None, unitCol=None)[source]

Set the (keyword only) parameters

setPostTreatmentCol(value)[source]
Parameters:

postTreatmentCol – post treatment indicator column

setStepSize(value)[source]
Parameters:

stepSize – Step size to be used for each iteration of optimization (> 0)

setTimeCol(value)[source]
Parameters:

timeCol – Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.

setTol(value)[source]
Parameters:

tol – the convergence tolerance for iterative algorithms (>= 0)

setTreatmentCol(value)[source]
Parameters:

treatmentCol – treatment column

setUnitCol(value)[source]
Parameters:

unitCol – Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.

stepSize = Param(parent='undefined', name='stepSize', doc='Step size to be used for each iteration of optimization (> 0)')
timeCol = Param(parent='undefined', name='timeCol', doc='Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.')
tol = Param(parent='undefined', name='tol', doc='the convergence tolerance for iterative algorithms (>= 0)')
treatmentCol = Param(parent='undefined', name='treatmentCol', doc='treatment column')
unitCol = Param(parent='undefined', name='unitCol', doc='Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.')

synapse.ml.causal.SyntheticDiffInDiffEstimator module

class synapse.ml.causal.SyntheticDiffInDiffEstimator.SyntheticDiffInDiffEstimator(java_obj=None, epsilon=1e-10, handleMissingOutcome='zero', localSolverThreshold=1000000, maxIter=100, numIterNoChange=None, outcomeCol=None, postTreatmentCol=None, stepSize=1.0, timeCol=None, tol=0.001, treatmentCol=None, unitCol=None, zeta=None)[source]

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaEstimator

Parameters:
  • epsilon (float) – This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.

  • handleMissingOutcome (str) – How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)

  • localSolverThreshold (long) – threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.

  • maxIter (int) – maximum number of iterations (>= 0)

  • numIterNoChange (int) – Early termination when number of iterations without change reached.

  • outcomeCol (str) – outcome column

  • postTreatmentCol (str) – post treatment indicator column

  • stepSize (float) – Step size to be used for each iteration of optimization (> 0)

  • timeCol (str) – Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.

  • tol (float) – the convergence tolerance for iterative algorithms (>= 0)

  • treatmentCol (str) – treatment column

  • unitCol (str) – Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.

  • zeta (float) – The zeta value for regularization term when fitting unit weights. If not specified, a default value will be computed based on formula (2.2) specified in https://www.nber.org/system/files/working_papers/w25532/w25532.pdf. For large scale data, one may want to tune the zeta value, minimizing the loss of the unit weights regression.

epsilon = Param(parent='undefined', name='epsilon', doc='This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.')
getEpsilon()[source]
Returns:

This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.

Return type:

epsilon

getHandleMissingOutcome()[source]
Returns:

How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)

Return type:

handleMissingOutcome

static getJavaPackage()[source]

Returns package name String.

getLocalSolverThreshold()[source]
Returns:

threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.

Return type:

localSolverThreshold

getMaxIter()[source]
Returns:

maximum number of iterations (>= 0)

Return type:

maxIter

getNumIterNoChange()[source]
Returns:

Early termination when number of iterations without change reached.

Return type:

numIterNoChange

getOutcomeCol()[source]
Returns:

outcome column

Return type:

outcomeCol

getPostTreatmentCol()[source]
Returns:

post treatment indicator column

Return type:

postTreatmentCol

getStepSize()[source]
Returns:

Step size to be used for each iteration of optimization (> 0)

Return type:

stepSize

getTimeCol()[source]
Returns:

Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.

Return type:

timeCol

getTol()[source]
Returns:

the convergence tolerance for iterative algorithms (>= 0)

Return type:

tol

getTreatmentCol()[source]
Returns:

treatment column

Return type:

treatmentCol

getUnitCol()[source]
Returns:

Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.

Return type:

unitCol

getZeta()[source]
Returns:

The zeta value for regularization term when fitting unit weights. If not specified, a default value will be computed based on formula (2.2) specified in https://www.nber.org/system/files/working_papers/w25532/w25532.pdf. For large scale data, one may want to tune the zeta value, minimizing the loss of the unit weights regression.

Return type:

zeta

handleMissingOutcome = Param(parent='undefined', name='handleMissingOutcome', doc='How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)')
localSolverThreshold = Param(parent='undefined', name='localSolverThreshold', doc='threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.')
maxIter = Param(parent='undefined', name='maxIter', doc='maximum number of iterations (>= 0)')
numIterNoChange = Param(parent='undefined', name='numIterNoChange', doc='Early termination when number of iterations without change reached.')
outcomeCol = Param(parent='undefined', name='outcomeCol', doc='outcome column')
postTreatmentCol = Param(parent='undefined', name='postTreatmentCol', doc='post treatment indicator column')
classmethod read()[source]

Returns an MLReader instance for this class.

setEpsilon(value)[source]
Parameters:

epsilon – This value is added to the weights when we fit the final linear model for SyntheticControlEstimator and SyntheticDiffInDiffEstimator in order to avoid zero weights.

setHandleMissingOutcome(value)[source]
Parameters:

handleMissingOutcome – How to handle missing outcomes. Options are skip (which will filter out units with missing outcomes), zero (fill in missing outcomes with zero), or impute (impute with nearest available outcomes, or mean if two nearest outcomes are available)

setLocalSolverThreshold(value)[source]
Parameters:

localSolverThreshold – threshold for using local solver on driver node. Local solver is faster but relies on part of data being collected on driver node.

setMaxIter(value)[source]
Parameters:

maxIter – maximum number of iterations (>= 0)

setNumIterNoChange(value)[source]
Parameters:

numIterNoChange – Early termination when number of iterations without change reached.

setOutcomeCol(value)[source]
Parameters:

outcomeCol – outcome column

setParams(epsilon=1e-10, handleMissingOutcome='zero', localSolverThreshold=1000000, maxIter=100, numIterNoChange=None, outcomeCol=None, postTreatmentCol=None, stepSize=1.0, timeCol=None, tol=0.001, treatmentCol=None, unitCol=None, zeta=None)[source]

Set the (keyword only) parameters

setPostTreatmentCol(value)[source]
Parameters:

postTreatmentCol – post treatment indicator column

setStepSize(value)[source]
Parameters:

stepSize – Step size to be used for each iteration of optimization (> 0)

setTimeCol(value)[source]
Parameters:

timeCol – Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.

setTol(value)[source]
Parameters:

tol – the convergence tolerance for iterative algorithms (>= 0)

setTreatmentCol(value)[source]
Parameters:

treatmentCol – treatment column

setUnitCol(value)[source]
Parameters:

unitCol – Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.

setZeta(value)[source]
Parameters:

zeta – The zeta value for regularization term when fitting unit weights. If not specified, a default value will be computed based on formula (2.2) specified in https://www.nber.org/system/files/working_papers/w25532/w25532.pdf. For large scale data, one may want to tune the zeta value, minimizing the loss of the unit weights regression.

stepSize = Param(parent='undefined', name='stepSize', doc='Step size to be used for each iteration of optimization (> 0)')
timeCol = Param(parent='undefined', name='timeCol', doc='Specify the column that identifies the time when outcome is measured in the panel data. For example, if the outcome is measured daily, this column could be the Date column.')
tol = Param(parent='undefined', name='tol', doc='the convergence tolerance for iterative algorithms (>= 0)')
treatmentCol = Param(parent='undefined', name='treatmentCol', doc='treatment column')
unitCol = Param(parent='undefined', name='unitCol', doc='Specify the name of the column which contains an identifier for each observed unit in the panel data. For example, if the observed units are users, this column could be the UserId column.')
zeta = Param(parent='undefined', name='zeta', doc='The zeta value for regularization term when fitting unit weights. If not specified, a default value will be computed based on formula (2.2) specified in https://www.nber.org/system/files/working_papers/w25532/w25532.pdf. For large scale data, one may want to tune the zeta value, minimizing the loss of the unit weights regression.')

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.