synapse.ml.services.language package

Submodules

synapse.ml.services.language.AnalyzeText module

class synapse.ml.services.language.AnalyzeText.AnalyzeText(java_obj=None, AADToken=None, AADTokenCol=None, CustomAuthHeader=None, CustomAuthHeaderCol=None, apiVersion=None, apiVersionCol=None, batchSize=10, concurrency=1, concurrentTimeout=None, countryHint=None, countryHintCol=None, customHeaders=None, customHeadersCol=None, customUrlRoot=None, domain=None, domainCol=None, errorCol='AnalyzeText_6142f1b6d1c3_error', handler=None, kind=None, language=None, languageCol=None, loggingOptOut=None, loggingOptOutCol=None, modelVersion=None, modelVersionCol=None, opinionMining=None, opinionMiningCol=None, outputCol='AnalyzeText_6142f1b6d1c3_output', piiCategories=None, piiCategoriesCol=None, showStats=None, showStatsCol=None, stringIndexType=None, stringIndexTypeCol=None, subscriptionKey=None, subscriptionKeyCol=None, text=None, textCol=None, timeout=60.0, url=None)

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaTransformer

Parameters:

• AADToken (object) – AAD Token used for authentication

• CustomAuthHeader (object) – A Custom Value for Authorization Header

• apiVersion (object) – version of the api

• batchSize (int) – The max size of the buffer

• concurrency (int) – max number of concurrent calls

• concurrentTimeout (float) – max number seconds to wait on futures if concurrency >= 1

• countryHint (object) – the countryHint for language detection

• customHeaders (object) – Map of Custom Header Key-Value Tuples.

• customUrlRoot (str) – The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.

• domain (object) – if specified, will set the PII domain to include only a subset of the entity categories. Possible values include: ‘PHI’, ‘none’.

• errorCol (str) – column to hold http errors

• handler (object) – Which strategy to use when handling requests

• kind (str) – Enumeration of supported Text Analysis tasks

• language (object) – the language code of the text (optional for some services)

• loggingOptOut (object) – loggingOptOut for task

• modelVersion (object) – Version of the model

• opinionMining (object) – opinionMining option for SentimentAnalysisTask

• outputCol (str) – The name of the output column

• piiCategories (object) – describes the PII categories to return

• showStats (object) – Whether to include detailed statistics in the response

• stringIndexType (object) – Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

• subscriptionKey (object) – the API key to use

• text (object) – the text in the request body

• timeout (float) – number of seconds to wait before closing the connection

• url (str) – Url of the service

AADToken = Param(parent='undefined', name='AADToken', doc='ServiceParam: AAD Token used for authentication')

CustomAuthHeader = Param(parent='undefined', name='CustomAuthHeader', doc='ServiceParam: A Custom Value for Authorization Header')

apiVersion = Param(parent='undefined', name='apiVersion', doc='ServiceParam: version of the api')

batchSize = Param(parent='undefined', name='batchSize', doc='The max size of the buffer')

concurrency = Param(parent='undefined', name='concurrency', doc='max number of concurrent calls')

concurrentTimeout = Param(parent='undefined', name='concurrentTimeout', doc='max number seconds to wait on futures if concurrency >= 1')

countryHint = Param(parent='undefined', name='countryHint', doc='ServiceParam: the countryHint for language detection')

customHeaders = Param(parent='undefined', name='customHeaders', doc='ServiceParam: Map of Custom Header Key-Value Tuples.')

customUrlRoot = Param(parent='undefined', name='customUrlRoot', doc='The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.')

domain = Param(parent='undefined', name='domain', doc="ServiceParam: if specified, will set the PII domain to include only a subset of the entity categories. Possible values include: 'PHI', 'none'.")

errorCol = Param(parent='undefined', name='errorCol', doc='column to hold http errors')

getAADToken()

Returns:

AAD Token used for authentication

Return type:

AADToken

getApiVersion()

Returns:

version of the api

Return type:

apiVersion

getBatchSize()

Returns:

The max size of the buffer

Return type:

batchSize

getConcurrency()

Returns:

max number of concurrent calls

Return type:

concurrency

getConcurrentTimeout()

Returns:

max number seconds to wait on futures if concurrency >= 1

Return type:

concurrentTimeout

getCountryHint()

Returns:

the countryHint for language detection

Return type:

countryHint

getCustomAuthHeader()

Returns:

A Custom Value for Authorization Header

Return type:

CustomAuthHeader

getCustomHeaders()

Returns:

Map of Custom Header Key-Value Tuples.

Return type:

customHeaders

getCustomUrlRoot()

Returns:

The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.

Return type:

customUrlRoot

getDomain()

Returns:

if specified, will set the PII domain to include only a subset of the entity categories. Possible values include: ‘PHI’, ‘none’.

Return type:

domain

getErrorCol()

Returns:

column to hold http errors

Return type:

errorCol

getHandler()

Returns:

Which strategy to use when handling requests

Return type:

handler

static getJavaPackage()

Returns package name String.

getKind()

Returns:

Enumeration of supported Text Analysis tasks

Return type:

kind

getLanguage()

Returns:

the language code of the text (optional for some services)

Return type:

language

getLoggingOptOut()

Returns:

loggingOptOut for task

Return type:

loggingOptOut

getModelVersion()

Returns:

Version of the model

Return type:

modelVersion

getOpinionMining()

Returns:

opinionMining option for SentimentAnalysisTask

Return type:

opinionMining

getOutputCol()

Returns:

The name of the output column

Return type:

outputCol

getPiiCategories()

Returns:

describes the PII categories to return

Return type:

piiCategories

getShowStats()

Returns:

Whether to include detailed statistics in the response

Return type:

showStats

getStringIndexType()

Returns:

Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

Return type:

stringIndexType

getSubscriptionKey()

Returns:

the API key to use

Return type:

subscriptionKey

getText()

Returns:

the text in the request body

Return type:

text

getTimeout()

Returns:

number of seconds to wait before closing the connection

Return type:

timeout

getUrl()

Returns:

Url of the service

Return type:

url

handler = Param(parent='undefined', name='handler', doc='Which strategy to use when handling requests')

kind = Param(parent='undefined', name='kind', doc='Enumeration of supported Text Analysis tasks')

language = Param(parent='undefined', name='language', doc='ServiceParam: the language code of the text (optional for some services)')

loggingOptOut = Param(parent='undefined', name='loggingOptOut', doc='ServiceParam: loggingOptOut for task')

modelVersion = Param(parent='undefined', name='modelVersion', doc='ServiceParam: Version of the model')

opinionMining = Param(parent='undefined', name='opinionMining', doc='ServiceParam: opinionMining option for SentimentAnalysisTask')

outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')

piiCategories = Param(parent='undefined', name='piiCategories', doc='ServiceParam: describes the PII categories to return')

classmethod read()

Returns an MLReader instance for this class.

setAADToken(value)

Parameters:

AADToken – AAD Token used for authentication

setAADTokenCol(value)

Parameters:

AADToken – AAD Token used for authentication

setApiVersion(value)

Parameters:

apiVersion – version of the api

setApiVersionCol(value)

Parameters:

apiVersion – version of the api

setBatchSize(value)

Parameters:

batchSize – The max size of the buffer

setConcurrency(value)

Parameters:

concurrency – max number of concurrent calls

setConcurrentTimeout(value)

Parameters:

concurrentTimeout – max number seconds to wait on futures if concurrency >= 1

setCountryHint(value)

Parameters:

countryHint – the countryHint for language detection

setCountryHintCol(value)

Parameters:

countryHint – the countryHint for language detection

setCustomAuthHeader(value)

Parameters:

CustomAuthHeader – A Custom Value for Authorization Header

setCustomAuthHeaderCol(value)

Parameters:

CustomAuthHeader – A Custom Value for Authorization Header

setCustomHeaders(value)

Parameters:

customHeaders – Map of Custom Header Key-Value Tuples.

setCustomHeadersCol(value)

Parameters:

customHeaders – Map of Custom Header Key-Value Tuples.

setCustomServiceName(value)

setCustomUrlRoot(value)

Parameters:

customUrlRoot – The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.

setDefaultInternalEndpoint(value)

setDomain(value)

Parameters:

domain – if specified, will set the PII domain to include only a subset of the entity categories. Possible values include: ‘PHI’, ‘none’.

setDomainCol(value)

Parameters:

domain – if specified, will set the PII domain to include only a subset of the entity categories. Possible values include: ‘PHI’, ‘none’.

setEndpoint(value)

setErrorCol(value)

Parameters:

errorCol – column to hold http errors

setHandler(value)

Parameters:

handler – Which strategy to use when handling requests

setKind(value)

Parameters:

kind – Enumeration of supported Text Analysis tasks

setLanguage(value)

Parameters:

language – the language code of the text (optional for some services)

setLanguageCol(value)

Parameters:

language – the language code of the text (optional for some services)

setLocation(value)

setLoggingOptOut(value)

Parameters:

loggingOptOut – loggingOptOut for task

setLoggingOptOutCol(value)

Parameters:

loggingOptOut – loggingOptOut for task

setModelVersion(value)

Parameters:

modelVersion – Version of the model

setModelVersionCol(value)

Parameters:

modelVersion – Version of the model

setOpinionMining(value)

Parameters:

opinionMining – opinionMining option for SentimentAnalysisTask

setOpinionMiningCol(value)

Parameters:

opinionMining – opinionMining option for SentimentAnalysisTask

setOutputCol(value)

Parameters:

outputCol – The name of the output column

setParams(AADToken=None, AADTokenCol=None, CustomAuthHeader=None, CustomAuthHeaderCol=None, apiVersion=None, apiVersionCol=None, batchSize=10, concurrency=1, concurrentTimeout=None, countryHint=None, countryHintCol=None, customHeaders=None, customHeadersCol=None, customUrlRoot=None, domain=None, domainCol=None, errorCol='AnalyzeText_6142f1b6d1c3_error', handler=None, kind=None, language=None, languageCol=None, loggingOptOut=None, loggingOptOutCol=None, modelVersion=None, modelVersionCol=None, opinionMining=None, opinionMiningCol=None, outputCol='AnalyzeText_6142f1b6d1c3_output', piiCategories=None, piiCategoriesCol=None, showStats=None, showStatsCol=None, stringIndexType=None, stringIndexTypeCol=None, subscriptionKey=None, subscriptionKeyCol=None, text=None, textCol=None, timeout=60.0, url=None)

Set the (keyword only) parameters

setPiiCategories(value)

Parameters:

piiCategories – describes the PII categories to return

setPiiCategoriesCol(value)

Parameters:

piiCategories – describes the PII categories to return

setShowStats(value)

Parameters:

showStats – Whether to include detailed statistics in the response

setShowStatsCol(value)

Parameters:

showStats – Whether to include detailed statistics in the response

setStringIndexType(value)

Parameters:

stringIndexType – Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

setStringIndexTypeCol(value)

Parameters:

stringIndexType – Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

setSubscriptionKey(value)

Parameters:

subscriptionKey – the API key to use

setSubscriptionKeyCol(value)

Parameters:

subscriptionKey – the API key to use

setText(value)

Parameters:

text – the text in the request body

setTextCol(value)

Parameters:

text – the text in the request body

setTimeout(value)

Parameters:

timeout – number of seconds to wait before closing the connection

setUrl(value)

Parameters:

url – Url of the service

showStats = Param(parent='undefined', name='showStats', doc='ServiceParam: Whether to include detailed statistics in the response')

stringIndexType = Param(parent='undefined', name='stringIndexType', doc='ServiceParam: Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets')

subscriptionKey = Param(parent='undefined', name='subscriptionKey', doc='ServiceParam: the API key to use')

text = Param(parent='undefined', name='text', doc='ServiceParam: the text in the request body')

timeout = Param(parent='undefined', name='timeout', doc='number of seconds to wait before closing the connection')

url = Param(parent='undefined', name='url', doc='Url of the service')

synapse.ml.services.language.AnalyzeTextLongRunningOperations module

class synapse.ml.services.language.AnalyzeTextLongRunningOperations.AnalyzeTextLongRunningOperations(java_obj=None, AADToken=None, AADTokenCol=None, CustomAuthHeader=None, CustomAuthHeaderCol=None, apiVersion=None, apiVersionCol=None, backoffs=[100, 500, 1000], batchSize=10, concurrency=1, concurrentTimeout=None, customHeaders=None, customHeadersCol=None, customUrlRoot=None, deploymentName=None, deploymentNameCol=None, domain=None, domainCol=None, errorCol='AnalyzeTextLongRunningOperations_88fcc2f9ce87_error', excludeNormalizedValues=None, excludeNormalizedValuesCol=None, exclusionList=None, exclusionListCol=None, inclusionList=None, inclusionListCol=None, initialPollingDelay=300, kind=None, language=None, languageCol=None, loggingOptOut=None, loggingOptOutCol=None, maxPollingRetries=1000, modelVersion=None, modelVersionCol=None, opinionMining=None, opinionMiningCol=None, outputCol='AnalyzeTextLongRunningOperations_88fcc2f9ce87_output', overlapPolicy=None, overlapPolicyCol=None, piiCategories=None, piiCategoriesCol=None, pollingDelay=1000, projectName=None, projectNameCol=None, sentenceCount=None, sentenceCountCol=None, showStats=None, showStatsCol=None, sortBy=None, sortByCol=None, stringIndexType=None, stringIndexTypeCol=None, subscriptionKey=None, subscriptionKeyCol=None, summaryLength=None, summaryLengthCol=None, suppressMaxRetriesException=False, text=None, textCol=None, timeout=60.0, url=None)

Bases: ComplexParamsMixin, JavaMLReadable, JavaMLWritable, JavaTransformer

Parameters:

• AADToken (object) – AAD Token used for authentication

• CustomAuthHeader (object) – A Custom Value for Authorization Header

• apiVersion (object) – version of the api

• backoffs (list) – array of backoffs to use in the handler

• batchSize (int) – The max size of the buffer

• concurrency (int) – max number of concurrent calls

• concurrentTimeout (float) – max number seconds to wait on futures if concurrency >= 1

• customHeaders (object) – Map of Custom Header Key-Value Tuples.

• customUrlRoot (str) – The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.

• deploymentName (object) – This field indicates the deployment name for the model. This is a required field.

• domain (object) – The domain of the PII entity recognition request.

• errorCol (str) – column to hold http errors

• excludeNormalizedValues (object) – (Optional) request parameter that allows the user to provide settings for running the inference. If set to true, the service will exclude normalized

• exclusionList (object) – (Optional) request parameter that filters out any entities that are included the excludeList. When a user specifies an excludeList, they cannot get a prediction returned with an entity in that list. We will apply inclusionList before exclusionList

• inclusionList (object) – (Optional) request parameter that limits the output to the requested entity types included in this list. We will apply inclusionList before exclusionList

• initialPollingDelay (int) – number of milliseconds to wait before first poll for result

• kind (str) – Enumeration of supported Text Analysis tasks

• language (object) – the language code of the text (optional for some services)

• loggingOptOut (object) – loggingOptOut for task

• maxPollingRetries (int) – number of times to poll

• modelVersion (object) – Version of the model

• opinionMining (object) – Whether to use opinion mining in the request or not.

• outputCol (str) – The name of the output column

• overlapPolicy (object) – (Optional) describes the type of overlap policy to apply to the ner output.

• piiCategories (object) – describes the PII categories to return

• pollingDelay (int) – number of milliseconds to wait between polling

• projectName (object) – This field indicates the project name for the model. This is a required field

• sentenceCount (object) – Specifies the number of sentences in the extracted summary.

• showStats (object) – Whether to include detailed statistics in the response

• sortBy (object) – Specifies how to sort the extracted summaries. This can be either ‘Rank’ or ‘Offset’.

• stringIndexType (object) – Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

• subscriptionKey (object) – the API key to use

• summaryLength (object) – (NOTE: Recommended to use summaryLength over sentenceCount) Controls the approximate length of the output summaries.

• suppressMaxRetriesException (bool) – set true to suppress the maxumimum retries exception and report in the error column

• text (object) – the text in the request body

• timeout (float) – number of seconds to wait before closing the connection

• url (str) – Url of the service

AADToken = Param(parent='undefined', name='AADToken', doc='ServiceParam: AAD Token used for authentication')

CustomAuthHeader = Param(parent='undefined', name='CustomAuthHeader', doc='ServiceParam: A Custom Value for Authorization Header')

apiVersion = Param(parent='undefined', name='apiVersion', doc='ServiceParam: version of the api')

backoffs = Param(parent='undefined', name='backoffs', doc='array of backoffs to use in the handler')

batchSize = Param(parent='undefined', name='batchSize', doc='The max size of the buffer')

concurrency = Param(parent='undefined', name='concurrency', doc='max number of concurrent calls')

concurrentTimeout = Param(parent='undefined', name='concurrentTimeout', doc='max number seconds to wait on futures if concurrency >= 1')

customHeaders = Param(parent='undefined', name='customHeaders', doc='ServiceParam: Map of Custom Header Key-Value Tuples.')

customUrlRoot = Param(parent='undefined', name='customUrlRoot', doc='The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.')

deploymentName = Param(parent='undefined', name='deploymentName', doc='ServiceParam: This field indicates the deployment name for the model. This is a required field.')

domain = Param(parent='undefined', name='domain', doc='ServiceParam: The domain of the PII entity recognition request.')

errorCol = Param(parent='undefined', name='errorCol', doc='column to hold http errors')

excludeNormalizedValues = Param(parent='undefined', name='excludeNormalizedValues', doc='ServiceParam: (Optional) request parameter that allows the user to provide settings for running the inference. If set to true, the service will exclude normalized')

exclusionList = Param(parent='undefined', name='exclusionList', doc='ServiceParam: (Optional) request parameter that filters out any entities that are included the excludeList. When a user specifies an excludeList, they cannot get a prediction returned with an entity in that list. We will apply inclusionList before exclusionList')

getAADToken()

Returns:

AAD Token used for authentication

Return type:

AADToken

getApiVersion()

Returns:

version of the api

Return type:

apiVersion

getBackoffs()

Returns:

array of backoffs to use in the handler

Return type:

backoffs

getBatchSize()

Returns:

The max size of the buffer

Return type:

batchSize

getConcurrency()

Returns:

max number of concurrent calls

Return type:

concurrency

getConcurrentTimeout()

Returns:

max number seconds to wait on futures if concurrency >= 1

Return type:

concurrentTimeout

getCustomAuthHeader()

Returns:

A Custom Value for Authorization Header

Return type:

CustomAuthHeader

getCustomHeaders()

Returns:

Map of Custom Header Key-Value Tuples.

Return type:

customHeaders

getCustomUrlRoot()

Returns:

The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.

Return type:

customUrlRoot

getDeploymentName()

Returns:

This field indicates the deployment name for the model. This is a required field.

Return type:

deploymentName

getDomain()

Returns:

The domain of the PII entity recognition request.

Return type:

domain

getErrorCol()

Returns:

column to hold http errors

Return type:

errorCol

getExcludeNormalizedValues()

Returns:

(Optional) request parameter that allows the user to provide settings for running the inference. If set to true, the service will exclude normalized

Return type:

excludeNormalizedValues

getExclusionList()

Returns:

(Optional) request parameter that filters out any entities that are included the excludeList. When a user specifies an excludeList, they cannot get a prediction returned with an entity in that list. We will apply inclusionList before exclusionList

Return type:

exclusionList

getInclusionList()

Returns:

(Optional) request parameter that limits the output to the requested entity types included in this list. We will apply inclusionList before exclusionList

Return type:

inclusionList

getInitialPollingDelay()

Returns:

number of milliseconds to wait before first poll for result

Return type:

initialPollingDelay

static getJavaPackage()

Returns package name String.

getKind()

Returns:

Enumeration of supported Text Analysis tasks

Return type:

kind

getLanguage()

Returns:

the language code of the text (optional for some services)

Return type:

language

getLoggingOptOut()

Returns:

loggingOptOut for task

Return type:

loggingOptOut

getMaxPollingRetries()

Returns:

number of times to poll

Return type:

maxPollingRetries

getModelVersion()

Returns:

Version of the model

Return type:

modelVersion

getOpinionMining()

Returns:

Whether to use opinion mining in the request or not.

Return type:

opinionMining

getOutputCol()

Returns:

The name of the output column

Return type:

outputCol

getOverlapPolicy()

Returns:

(Optional) describes the type of overlap policy to apply to the ner output.

Return type:

overlapPolicy

getPiiCategories()

Returns:

describes the PII categories to return

Return type:

piiCategories

getPollingDelay()

Returns:

number of milliseconds to wait between polling

Return type:

pollingDelay

getProjectName()

Returns:

This field indicates the project name for the model. This is a required field

Return type:

projectName

getSentenceCount()

Returns:

Specifies the number of sentences in the extracted summary.

Return type:

sentenceCount

getShowStats()

Returns:

Whether to include detailed statistics in the response

Return type:

showStats

getSortBy()

Returns:

Specifies how to sort the extracted summaries. This can be either ‘Rank’ or ‘Offset’.

Return type:

sortBy

getStringIndexType()

Returns:

Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

Return type:

stringIndexType

getSubscriptionKey()

Returns:

the API key to use

Return type:

subscriptionKey

getSummaryLength()

Returns:

(NOTE: Recommended to use summaryLength over sentenceCount) Controls the approximate length of the output summaries.

Return type:

summaryLength

getSuppressMaxRetriesException()

Returns:

set true to suppress the maxumimum retries exception and report in the error column

Return type:

suppressMaxRetriesException

getText()

Returns:

the text in the request body

Return type:

text

getTimeout()

Returns:

number of seconds to wait before closing the connection

Return type:

timeout

getUrl()

Returns:

Url of the service

Return type:

url

inclusionList = Param(parent='undefined', name='inclusionList', doc='ServiceParam: (Optional) request parameter that limits the output to the requested entity types included in this list. We will apply inclusionList before exclusionList')

initialPollingDelay = Param(parent='undefined', name='initialPollingDelay', doc='number of milliseconds to wait before first poll for result')

kind = Param(parent='undefined', name='kind', doc='Enumeration of supported Text Analysis tasks')

language = Param(parent='undefined', name='language', doc='ServiceParam: the language code of the text (optional for some services)')

loggingOptOut = Param(parent='undefined', name='loggingOptOut', doc='ServiceParam: loggingOptOut for task')

maxPollingRetries = Param(parent='undefined', name='maxPollingRetries', doc='number of times to poll')

modelVersion = Param(parent='undefined', name='modelVersion', doc='ServiceParam: Version of the model')

opinionMining = Param(parent='undefined', name='opinionMining', doc='ServiceParam: Whether to use opinion mining in the request or not.')

outputCol = Param(parent='undefined', name='outputCol', doc='The name of the output column')

overlapPolicy = Param(parent='undefined', name='overlapPolicy', doc='ServiceParam: (Optional) describes the type of overlap policy to apply to the ner output.')

piiCategories = Param(parent='undefined', name='piiCategories', doc='ServiceParam: describes the PII categories to return')

pollingDelay = Param(parent='undefined', name='pollingDelay', doc='number of milliseconds to wait between polling')

projectName = Param(parent='undefined', name='projectName', doc='ServiceParam: This field indicates the project name for the model. This is a required field')

classmethod read()

Returns an MLReader instance for this class.

sentenceCount = Param(parent='undefined', name='sentenceCount', doc='ServiceParam: Specifies the number of sentences in the extracted summary.')

setAADToken(value)

Parameters:

AADToken – AAD Token used for authentication

setAADTokenCol(value)

Parameters:

AADToken – AAD Token used for authentication

setApiVersion(value)

Parameters:

apiVersion – version of the api

setApiVersionCol(value)

Parameters:

apiVersion – version of the api

setBackoffs(value)

Parameters:

backoffs – array of backoffs to use in the handler

setBatchSize(value)

Parameters:

batchSize – The max size of the buffer

setConcurrency(value)

Parameters:

concurrency – max number of concurrent calls

setConcurrentTimeout(value)

Parameters:

concurrentTimeout – max number seconds to wait on futures if concurrency >= 1

setCustomAuthHeader(value)

Parameters:

CustomAuthHeader – A Custom Value for Authorization Header

setCustomAuthHeaderCol(value)

Parameters:

CustomAuthHeader – A Custom Value for Authorization Header

setCustomHeaders(value)

Parameters:

customHeaders – Map of Custom Header Key-Value Tuples.

setCustomHeadersCol(value)

Parameters:

customHeaders – Map of Custom Header Key-Value Tuples.

setCustomServiceName(value)

setCustomUrlRoot(value)

Parameters:

customUrlRoot – The custom URL root for the service. This will not append OpenAI specific model path completions (i.e. /chat/completions) to the URL.

setDefaultInternalEndpoint(value)

setDeploymentName(value)

Parameters:

deploymentName – This field indicates the deployment name for the model. This is a required field.

setDeploymentNameCol(value)

Parameters:

deploymentName – This field indicates the deployment name for the model. This is a required field.

setDomain(value)

Parameters:

domain – The domain of the PII entity recognition request.

setDomainCol(value)

Parameters:

domain – The domain of the PII entity recognition request.

setEndpoint(value)

setErrorCol(value)

Parameters:

errorCol – column to hold http errors

setExcludeNormalizedValues(value)

Parameters:

excludeNormalizedValues – (Optional) request parameter that allows the user to provide settings for running the inference. If set to true, the service will exclude normalized

setExcludeNormalizedValuesCol(value)

Parameters:

excludeNormalizedValues – (Optional) request parameter that allows the user to provide settings for running the inference. If set to true, the service will exclude normalized

setExclusionList(value)

Parameters:

exclusionList – (Optional) request parameter that filters out any entities that are included the excludeList. When a user specifies an excludeList, they cannot get a prediction returned with an entity in that list. We will apply inclusionList before exclusionList

setExclusionListCol(value)

Parameters:

exclusionList – (Optional) request parameter that filters out any entities that are included the excludeList. When a user specifies an excludeList, they cannot get a prediction returned with an entity in that list. We will apply inclusionList before exclusionList

setInclusionList(value)

Parameters:

inclusionList – (Optional) request parameter that limits the output to the requested entity types included in this list. We will apply inclusionList before exclusionList

setInclusionListCol(value)

Parameters:

inclusionList – (Optional) request parameter that limits the output to the requested entity types included in this list. We will apply inclusionList before exclusionList

setInitialPollingDelay(value)

Parameters:

initialPollingDelay – number of milliseconds to wait before first poll for result

setKind(value)

Parameters:

kind – Enumeration of supported Text Analysis tasks

setLanguage(value)

Parameters:

language – the language code of the text (optional for some services)

setLanguageCol(value)

Parameters:

language – the language code of the text (optional for some services)

setLocation(value)

setLoggingOptOut(value)

Parameters:

loggingOptOut – loggingOptOut for task

setLoggingOptOutCol(value)

Parameters:

loggingOptOut – loggingOptOut for task

setMaxPollingRetries(value)

Parameters:

maxPollingRetries – number of times to poll

setModelVersion(value)

Parameters:

modelVersion – Version of the model

setModelVersionCol(value)

Parameters:

modelVersion – Version of the model

setOpinionMining(value)

Parameters:

opinionMining – Whether to use opinion mining in the request or not.

setOpinionMiningCol(value)

Parameters:

opinionMining – Whether to use opinion mining in the request or not.

setOutputCol(value)

Parameters:

outputCol – The name of the output column

setOverlapPolicy(value)

Parameters:

overlapPolicy – (Optional) describes the type of overlap policy to apply to the ner output.

setOverlapPolicyCol(value)

Parameters:

overlapPolicy – (Optional) describes the type of overlap policy to apply to the ner output.

setParams(AADToken=None, AADTokenCol=None, CustomAuthHeader=None, CustomAuthHeaderCol=None, apiVersion=None, apiVersionCol=None, backoffs=[100, 500, 1000], batchSize=10, concurrency=1, concurrentTimeout=None, customHeaders=None, customHeadersCol=None, customUrlRoot=None, deploymentName=None, deploymentNameCol=None, domain=None, domainCol=None, errorCol='AnalyzeTextLongRunningOperations_88fcc2f9ce87_error', excludeNormalizedValues=None, excludeNormalizedValuesCol=None, exclusionList=None, exclusionListCol=None, inclusionList=None, inclusionListCol=None, initialPollingDelay=300, kind=None, language=None, languageCol=None, loggingOptOut=None, loggingOptOutCol=None, maxPollingRetries=1000, modelVersion=None, modelVersionCol=None, opinionMining=None, opinionMiningCol=None, outputCol='AnalyzeTextLongRunningOperations_88fcc2f9ce87_output', overlapPolicy=None, overlapPolicyCol=None, piiCategories=None, piiCategoriesCol=None, pollingDelay=1000, projectName=None, projectNameCol=None, sentenceCount=None, sentenceCountCol=None, showStats=None, showStatsCol=None, sortBy=None, sortByCol=None, stringIndexType=None, stringIndexTypeCol=None, subscriptionKey=None, subscriptionKeyCol=None, summaryLength=None, summaryLengthCol=None, suppressMaxRetriesException=False, text=None, textCol=None, timeout=60.0, url=None)

Set the (keyword only) parameters

setPiiCategories(value)

Parameters:

piiCategories – describes the PII categories to return

setPiiCategoriesCol(value)

Parameters:

piiCategories – describes the PII categories to return

setPollingDelay(value)

Parameters:

pollingDelay – number of milliseconds to wait between polling

setProjectName(value)

Parameters:

projectName – This field indicates the project name for the model. This is a required field

setProjectNameCol(value)

Parameters:

projectName – This field indicates the project name for the model. This is a required field

setSentenceCount(value)

Parameters:

sentenceCount – Specifies the number of sentences in the extracted summary.

setSentenceCountCol(value)

Parameters:

sentenceCount – Specifies the number of sentences in the extracted summary.

setShowStats(value)

Parameters:

showStats – Whether to include detailed statistics in the response

setShowStatsCol(value)

Parameters:

showStats – Whether to include detailed statistics in the response

setSortBy(value)

Parameters:

sortBy – Specifies how to sort the extracted summaries. This can be either ‘Rank’ or ‘Offset’.

setSortByCol(value)

Parameters:

sortBy – Specifies how to sort the extracted summaries. This can be either ‘Rank’ or ‘Offset’.

setStringIndexType(value)

Parameters:

stringIndexType – Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

setStringIndexTypeCol(value)

Parameters:

stringIndexType – Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets

setSubscriptionKey(value)

Parameters:

subscriptionKey – the API key to use

setSubscriptionKeyCol(value)

Parameters:

subscriptionKey – the API key to use

setSummaryLength(value)

Parameters:

summaryLength – (NOTE: Recommended to use summaryLength over sentenceCount) Controls the approximate length of the output summaries.

setSummaryLengthCol(value)

Parameters:

summaryLength – (NOTE: Recommended to use summaryLength over sentenceCount) Controls the approximate length of the output summaries.

setSuppressMaxRetriesException(value)

Parameters:

suppressMaxRetriesException – set true to suppress the maxumimum retries exception and report in the error column

setText(value)

Parameters:

text – the text in the request body

setTextCol(value)

Parameters:

text – the text in the request body

setTimeout(value)

Parameters:

timeout – number of seconds to wait before closing the connection

setUrl(value)

Parameters:

url – Url of the service

showStats = Param(parent='undefined', name='showStats', doc='ServiceParam: Whether to include detailed statistics in the response')

sortBy = Param(parent='undefined', name='sortBy', doc="ServiceParam: Specifies how to sort the extracted summaries. This can be either 'Rank' or 'Offset'.")

stringIndexType = Param(parent='undefined', name='stringIndexType', doc='ServiceParam: Specifies the method used to interpret string offsets. Defaults to Text Elements(Graphemes) according to Unicode v8.0.0.For more information see https://aka.ms/text-analytics-offsets')

subscriptionKey = Param(parent='undefined', name='subscriptionKey', doc='ServiceParam: the API key to use')

summaryLength = Param(parent='undefined', name='summaryLength', doc='ServiceParam: (NOTE: Recommended to use summaryLength over sentenceCount) Controls the approximate length of the output summaries.')

suppressMaxRetriesException = Param(parent='undefined', name='suppressMaxRetriesException', doc='set true to suppress the maxumimum retries exception and report in the error column')

text = Param(parent='undefined', name='text', doc='ServiceParam: the text in the request body')

timeout = Param(parent='undefined', name='timeout', doc='number of seconds to wait before closing the connection')

url = Param(parent='undefined', name='url', doc='Url of the service')

Module contents

SynapseML is an ecosystem of tools aimed towards expanding the distributed computing framework Apache Spark in several new directions. SynapseML adds many deep learning and data science tools to the Spark ecosystem, including seamless integration of Spark Machine Learning pipelines with Microsoft Cognitive Toolkit (CNTK), LightGBM and OpenCV. These tools enable powerful and highly-scalable predictive and analytical models for a variety of datasources.

SynapseML also brings new networking capabilities to the Spark Ecosystem. With the HTTP on Spark project, users can embed any web service into their SparkML models. In this vein, SynapseML provides easy to use SparkML transformers for a wide variety of Microsoft Cognitive Services. For production grade deployment, the Spark Serving project enables high throughput, sub-millisecond latency web services, backed by your Spark cluster.

SynapseML requires Scala 2.12, Spark 3.0+, and Python 3.6+.