synapse.ml.onnx package

Submodules

synapse.ml.onnx.ImageFeaturizer module

class synapse.ml.onnx.ImageFeaturizer.ImageFeaturizer(java_obj=None, autoConvertToColor=True, channelNormalizationMeans=[0.485, 0.456, 0.406], channelNormalizationStds=[0.229, 0.224, 0.225], colorScaleFactor=0.00392156862745098, dropNa=True, featureTensorName=None, headless=True, ignoreDecodingErrors=False, imageHeight=None, imageTensorName=None, imageWidth=None, inputCol=None, onnxModel=None, outputCol='ImageFeaturizer_52c0a75160ce_output', outputTensorName='')[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters

name¶ (str) – The name of the model in the OnnxHub
location¶ (str) – The location of the model, either on local or HDFS

setMiniBatchSize(size)[source]

setModel(name)[source]

setModelLocation(location)[source]

synapse.ml.onnx.ONNXModel module

class synapse.ml.onnx.ONNXModel.MapInfo(key_type: str, value_type: str, size: int = - 1)[source]

Bases: synapse.ml.onnx.ONNXModel.ValueInfo

classmethod from_java(java_map_info: py4j.java_gateway.JavaObject) → synapse.ml.onnx.ONNXModel.MapInfo[source]

class synapse.ml.onnx.ONNXModel.NodeInfo(name: str, value_info: py4j.java_gateway.JavaObject)[source]: Bases: object

class synapse.ml.onnx.ONNXModel.ONNXModel(java_obj=None, argMaxDict=None, deviceType=None, feedDict=None, fetchDict=None, miniBatcher=None, modelPayload=None, optimizationLevel='ALL_OPT', softMaxDict=None)[source]

Bases: pyspark.ml.util.MLReadable[pyspark.ml.util.RL]

Parameters

SparkSession¶ (SparkSession) – The SparkSession that will be used to find the model
location¶ (str) – The location of the model, either on local or HDFS

getModelInputs()[source]

getModelOutputs() → Mapping[str, synapse.ml.onnx.ONNXModel.NodeInfo][source]

setMiniBatchSize(n)[source]

setModelLocation(location)[source]

class synapse.ml.onnx.ONNXModel.SequenceInfo(length: int, sequence_of_maps: bool, map_info: synapse.ml.onnx.ONNXModel.MapInfo, sequence_type: str)[source]

Bases: synapse.ml.onnx.ONNXModel.ValueInfo

classmethod from_java(java_sequence_info: py4j.java_gateway.JavaObject) → synapse.ml.onnx.ONNXModel.SequenceInfo[source]

class synapse.ml.onnx.ONNXModel.TensorInfo(shape: List[int], type: str)[source]

Bases: synapse.ml.onnx.ONNXModel.ValueInfo

classmethod from_java(java_tensor_info: py4j.java_gateway.JavaObject) → synapse.ml.onnx.ONNXModel.TensorInfo[source]

class synapse.ml.onnx.ONNXModel.ValueInfo[source]

Bases: object

classmethod from_java(java_value_info: py4j.java_gateway.JavaObject) → synapse.ml.onnx.ONNXModel.ValueInfo[source]

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.