synapse.ml.core.schema package

Submodules

synapse.ml.core.schema.TypeConversionUtils module

synapse.ml.core.schema.TypeConversionUtils.complexTypeConverter(name, value, cache)[source]

Type conversion for complex types

Parameters

  • name –

  • value –

  • cache –

Returns

Return type

_java_obj

synapse.ml.core.schema.TypeConversionUtils.generateTypeConverter(name, cache, typeConverter)[source]

Type converter

Parameters

  • name (str) –

  • cache –

  • typeConverter –

Returns

Function to convert the type

Return type

lambda

synapse.ml.core.schema.Utils module

class synapse.ml.core.schema.Utils.ComplexParamsMixin[source]

Bases: pyspark.ml.util.MLReadable

class synapse.ml.core.schema.Utils.JavaMMLReadable[source]

Bases: pyspark.ml.util.MLReadable

(Private) Mixin for instances that provide JavaMLReader.

classmethod read()[source]

Returns an MLReader instance for this class.

class synapse.ml.core.schema.Utils.JavaMMLReader(clazz)[source]

Bases: pyspark.ml.util.JavaMLReader

(Private) Specialization of MLReader for JavaParams types

synapse.ml.core.schema.Utils.from_java(java_stage, stage_name)[source]

Given a Java object, create and return a Python wrapper of it. Used for ML persistence. Meta-algorithms such as Pipeline should override this method as a classmethod.

Parameters

  • java_stage (str) –

  • stage_name (str) –

Returns

The python wrapper

Return type

object

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.