Packages

class ImageFeaturizer extends Transformer with HasInputCol with HasOutputCol with Wrappable with ComplexParamsWritable with BasicLogging

The ImageFeaturizer relies on a CNTK model to do the featurization, one can set this model using the modelLocation parameter. To map the nodes of the CNTK model onto the standard "layers" structure of a feed forward neural net, one needs to supply a list of node names that range from the output node, back towards the input node of the CNTK Function. This list does not need to be exhaustive, and is provided to you if you use a model downloaded from the ModelDownloader, one can find this layer list in the schema of the downloaded model.

The ImageFeaturizer takes an input column of images (the type returned by the ImageReader), and automatically resizes them to fit the CMTKModel's inputs. It then feeds them through a pre-trained CNTK model. One can truncate the model using the cutOutputLayers parameter that determines how many layers to truncate from the output of the network. For example, layer=0 means that no layers are removed, layer=2 means that the image featurizer returns the activations of the layer that is two layers from the output layer.

Linear Supertypes
BasicLogging, ComplexParamsWritable, MLWritable, Wrappable, DotnetWrappable, RWrappable, PythonWrappable, BaseWrappable, HasOutputCol, HasInputCol, Transformer, PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ImageFeaturizer
  2. BasicLogging
  3. ComplexParamsWritable
  4. MLWritable
  5. Wrappable
  6. DotnetWrappable
  7. RWrappable
  8. PythonWrappable
  9. BaseWrappable
  10. HasOutputCol
  11. HasInputCol
  12. Transformer
  13. PipelineStage
  14. Logging
  15. Params
  16. Serializable
  17. Serializable
  18. Identifiable
  19. AnyRef
  20. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ImageFeaturizer()
  2. new ImageFeaturizer(uid: String)

    uid

    the uid of the image transformer

Value Members

  1. final def clear(param: Param[_]): ImageFeaturizer.this.type
    Definition Classes
    Params
  2. val cntkModel: TransformerParam
  3. def copy(extra: ParamMap): Transformer
    Definition Classes
    ImageFeaturizer → Transformer → PipelineStage → Params
  4. val cutOutputLayers: IntParam

    The number of layer to cut off the end of the network; 0 leaves the network intact, 1 removes the output layer, etc.

  5. def dotnetAdditionalMethods: String
    Definition Classes
    DotnetWrappable
  6. val dropNa: BooleanParam
  7. val emptyCntkModel: CNTKModel
  8. def explainParam(param: Param[_]): String
    Definition Classes
    Params
  9. def explainParams(): String
    Definition Classes
    Params
  10. final def extractParamMap(): ParamMap
    Definition Classes
    Params
  11. final def extractParamMap(extra: ParamMap): ParamMap
    Definition Classes
    Params
  12. final def get[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  13. def getCntkModel: CNTKModel

  14. def getCutOutputLayers: Int

  15. final def getDefault[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  16. def getDropNa: Boolean

  17. def getInputCol: String

    Definition Classes
    HasInputCol
  18. def getInputNode: Int

  19. def getLayerNames: Array[String]

  20. def getMiniBatchSize: Int

  21. def getModel: SerializableFunction

  22. final def getOrDefault[T](param: Param[T]): T
    Definition Classes
    Params
  23. def getOutputCol: String

    Definition Classes
    HasOutputCol
  24. def getParam(paramName: String): Param[Any]
    Definition Classes
    Params
  25. def getParamInfo(p: Param[_]): ParamInfo[_]
    Definition Classes
    BaseWrappable
  26. final def hasDefault[T](param: Param[T]): Boolean
    Definition Classes
    Params
  27. def hasParam(paramName: String): Boolean
    Definition Classes
    Params
  28. val inputCol: Param[String]

    The name of the input column

    The name of the input column

    Definition Classes
    HasInputCol
  29. final def isDefined(param: Param[_]): Boolean
    Definition Classes
    Params
  30. final def isSet(param: Param[_]): Boolean
    Definition Classes
    Params
  31. val layerNames: StringArrayParam

    Array with valid CNTK nodes to choose from; the first entries of this array should be closer to the output node.

  32. def logClass(): Unit
    Definition Classes
    BasicLogging
  33. def logFit[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  34. def logPredict[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  35. def logTrain[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  36. def logTransform[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  37. def logVerb[T](verb: String, f: ⇒ T): T
    Definition Classes
    BasicLogging
  38. def makeDotnetFile(conf: CodegenConfig): Unit
    Definition Classes
    DotnetWrappable
  39. def makePyFile(conf: CodegenConfig): Unit
    Definition Classes
    PythonWrappable
  40. def makeRFile(conf: CodegenConfig): Unit
    Definition Classes
    RWrappable
  41. val outputCol: Param[String]

    The name of the output column

    The name of the output column

    Definition Classes
    HasOutputCol
  42. lazy val params: Array[Param[_]]
    Definition Classes
    Params
  43. def pyAdditionalMethods: String
    Definition Classes
    PythonWrappable
  44. def pyInitFunc(): String
    Definition Classes
    PythonWrappable
  45. def save(path: String): Unit
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  46. final def set[T](param: Param[T], value: T): ImageFeaturizer.this.type
    Definition Classes
    Params
  47. def setCntkModel(value: CNTKModel): ImageFeaturizer.this.type

  48. def setCutOutputLayers(value: Int): ImageFeaturizer.this.type

  49. def setDropNa(value: Boolean): ImageFeaturizer.this.type

  50. def setInputCol(value: String): ImageFeaturizer.this.type

    Definition Classes
    HasInputCol
  51. def setInputNode(value: Int): ImageFeaturizer.this.type

  52. def setLayerNames(value: Array[String]): ImageFeaturizer.this.type

  53. def setMiniBatchSize(value: Int): ImageFeaturizer.this.type

  54. def setModel(model: SerializableFunction): ImageFeaturizer.this.type

  55. def setModel(modelSchema: ModelSchema): ImageFeaturizer.this.type
  56. def setModelLocation(path: String): ImageFeaturizer.this.type
  57. def setOutputCol(value: String): ImageFeaturizer.this.type

    Definition Classes
    HasOutputCol
  58. def toString(): String
    Definition Classes
    Identifiable → AnyRef → Any
  59. def transform(dataset: Dataset[_]): DataFrame
    Definition Classes
    ImageFeaturizer → Transformer
  60. def transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" )
  61. def transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" ) @varargs()
  62. def transformSchema(schema: StructType): StructType

    Add the features column to the schema

    Add the features column to the schema

    returns

    schema with features column

    Definition Classes
    ImageFeaturizer → PipelineStage
  63. val uid: String
    Definition Classes
    ImageFeaturizer → BasicLogging → Identifiable
  64. val ver: String
    Definition Classes
    BasicLogging
  65. def write: MLWriter
    Definition Classes
    ComplexParamsWritable → MLWritable