Packages

class ImageFeaturizer extends Transformer with HasInputCol with HasOutputCol with Wrappable with ComplexParamsWritable with BasicLogging

The ImageFeaturizer relies on a CNTK model to do the featurization, one can set this model using the modelLocation parameter. To map the nodes of the CNTK model onto the standard "layers" structure of a feed forward neural net, one needs to supply a list of node names that range from the output node, back towards the input node of the CNTK Function. This list does not need to be exhaustive, and is provided to you if you use a model downloaded from the ModelDownloader, one can find this layer list in the schema of the downloaded model.

The ImageFeaturizer takes an input column of images (the type returned by the ImageReader), and automatically resizes them to fit the CMTKModel's inputs. It then feeds them through a pre-trained CNTK model. One can truncate the model using the cutOutputLayers parameter that determines how many layers to truncate from the output of the network. For example, layer=0 means that no layers are removed, layer=2 means that the image featurizer returns the activations of the layer that is two layers from the output layer.

Linear Supertypes
BasicLogging, ComplexParamsWritable, MLWritable, Wrappable, RWrappable, PythonWrappable, BaseWrappable, HasOutputCol, HasInputCol, Transformer, PipelineStage, Logging, Params, Serializable, Serializable, Identifiable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. ImageFeaturizer
  2. BasicLogging
  3. ComplexParamsWritable
  4. MLWritable
  5. Wrappable
  6. RWrappable
  7. PythonWrappable
  8. BaseWrappable
  9. HasOutputCol
  10. HasInputCol
  11. Transformer
  12. PipelineStage
  13. Logging
  14. Params
  15. Serializable
  16. Serializable
  17. Identifiable
  18. AnyRef
  19. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new ImageFeaturizer()
  2. new ImageFeaturizer(uid: String)

    uid

    the uid of the image transformer

Value Members

  1. final def clear(param: Param[_]): ImageFeaturizer.this.type
    Definition Classes
    Params
  2. val cntkModel: TransformerParam
  3. def copy(extra: ParamMap): Transformer
    Definition Classes
    ImageFeaturizer → Transformer → PipelineStage → Params
  4. val cutOutputLayers: IntParam

    The number of layer to cut off the end of the network; 0 leaves the network intact, 1 removes the output layer, etc.

  5. val dropNa: BooleanParam
  6. val emptyCntkModel: CNTKModel
  7. def explainParam(param: Param[_]): String
    Definition Classes
    Params
  8. def explainParams(): String
    Definition Classes
    Params
  9. final def extractParamMap(): ParamMap
    Definition Classes
    Params
  10. final def extractParamMap(extra: ParamMap): ParamMap
    Definition Classes
    Params
  11. final def get[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  12. def getCntkModel: CNTKModel

  13. def getCutOutputLayers: Int

  14. final def getDefault[T](param: Param[T]): Option[T]
    Definition Classes
    Params
  15. def getDropNa: Boolean

  16. def getInputCol: String

    Definition Classes
    HasInputCol
  17. def getInputNode: Int

  18. def getLayerNames: Array[String]

  19. def getMiniBatchSize: Int

  20. def getModel: SerializableFunction

  21. final def getOrDefault[T](param: Param[T]): T
    Definition Classes
    Params
  22. def getOutputCol: String

    Definition Classes
    HasOutputCol
  23. def getParam(paramName: String): Param[Any]
    Definition Classes
    Params
  24. final def hasDefault[T](param: Param[T]): Boolean
    Definition Classes
    Params
  25. def hasParam(paramName: String): Boolean
    Definition Classes
    Params
  26. val inputCol: Param[String]

    The name of the input column

    The name of the input column

    Definition Classes
    HasInputCol
  27. final def isDefined(param: Param[_]): Boolean
    Definition Classes
    Params
  28. final def isSet(param: Param[_]): Boolean
    Definition Classes
    Params
  29. val layerNames: StringArrayParam

    Array with valid CNTK nodes to choose from; the first entries of this array should be closer to the output node.

  30. def logClass(): Unit
    Definition Classes
    BasicLogging
  31. def logFit[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  32. def logPredict[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  33. def logTrain[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  34. def logTransform[T](f: ⇒ T): T
    Definition Classes
    BasicLogging
  35. def logVerb[T](verb: String, f: ⇒ T): T
    Definition Classes
    BasicLogging
  36. def makePyFile(conf: CodegenConfig): Unit
    Definition Classes
    PythonWrappable
  37. def makeRFile(conf: CodegenConfig): Unit
    Definition Classes
    RWrappable
  38. val outputCol: Param[String]

    The name of the output column

    The name of the output column

    Definition Classes
    HasOutputCol
  39. lazy val params: Array[Param[_]]
    Definition Classes
    Params
  40. def pyAdditionalMethods: String
    Definition Classes
    PythonWrappable
  41. def pyInitFunc(): String
    Definition Classes
    PythonWrappable
  42. def save(path: String): Unit
    Definition Classes
    MLWritable
    Annotations
    @Since( "1.6.0" ) @throws( ... )
  43. final def set[T](param: Param[T], value: T): ImageFeaturizer.this.type
    Definition Classes
    Params
  44. def setCntkModel(value: CNTKModel): ImageFeaturizer.this.type

  45. def setCutOutputLayers(value: Int): ImageFeaturizer.this.type

  46. def setDropNa(value: Boolean): ImageFeaturizer.this.type

  47. def setInputCol(value: String): ImageFeaturizer.this.type

    Definition Classes
    HasInputCol
  48. def setInputNode(value: Int): ImageFeaturizer.this.type

  49. def setLayerNames(value: Array[String]): ImageFeaturizer.this.type

  50. def setMiniBatchSize(value: Int): ImageFeaturizer.this.type

  51. def setModel(model: SerializableFunction): ImageFeaturizer.this.type

  52. def setModel(modelSchema: ModelSchema): ImageFeaturizer.this.type
  53. def setModelLocation(path: String): ImageFeaturizer.this.type
  54. def setOutputCol(value: String): ImageFeaturizer.this.type

    Definition Classes
    HasOutputCol
  55. def toString(): String
    Definition Classes
    Identifiable → AnyRef → Any
  56. def transform(dataset: Dataset[_]): DataFrame
    Definition Classes
    ImageFeaturizer → Transformer
  57. def transform(dataset: Dataset[_], paramMap: ParamMap): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" )
  58. def transform(dataset: Dataset[_], firstParamPair: ParamPair[_], otherParamPairs: ParamPair[_]*): DataFrame
    Definition Classes
    Transformer
    Annotations
    @Since( "2.0.0" ) @varargs()
  59. def transformSchema(schema: StructType): StructType

    Add the features column to the schema

    Add the features column to the schema

    returns

    schema with features column

    Definition Classes
    ImageFeaturizer → PipelineStage
  60. val uid: String
    Definition Classes
    ImageFeaturizer → BasicLogging → Identifiable
  61. val ver: String
    Definition Classes
    BasicLogging
  62. def write: MLWriter
    Definition Classes
    ComplexParamsWritable → MLWritable