如何配置DL4j用于本地图像

huangapple 未分类评论47阅读模式
英文:

How to configure DL4j for local images

问题

我正在尝试使用DeepLearning4j将32x32像素的图像分类为0-9之间的数字。
我查阅了许多示例和教程,但在将数据集适配到网络时总是遇到一些异常情况。

我目前正在尝试使用一个带有ParentPathLabelGenerator的ImageRecordReader和RecordReaderDataSetIterator。

图像似乎加载得很好,但在适配时我总是遇到DL4JInvalidInputException异常。
    File parentDir = new File(dataPath);
    FileSplit filesInDir = new FileSplit(parentDir, NativeImageLoader.ALLOWED_FORMATS);
    ParentPathLabelGenerator labelMaker = new ParentPathLabelGenerator();

    BalancedPathFilter pathFilter = new BalancedPathFilter(new Random(), labelMaker, 100);
    InputSplit[] filesInDirSplit = filesInDir.sample(pathFilter, 80, 20);
    InputSplit trainData = filesInDirSplit[0];
    InputSplit testData = filesInDirSplit[1];

    ImageRecordReader recordReader = new ImageRecordReader(numRows, numColumns, 3, labelMaker);
    recordReader.initialize(trainData);

    DataSetIterator dataIter = new RecordReaderDataSetIterator(recordReader, 1, 1, outputNum);

当使用DenseLayer时:

Exception in thread "main" org.deeplearning4j.exception.DL4JInvalidInputException: 非矩阵输入;预期矩阵(秩为2),但得到秩为4的数组,形状为[1, 3, 32, 32]。缺少预处理器或错误的输入类型?(图层名称:layer0,图层索引:0,图层类型:DenseLayer)


当使用ConvolutionLayer时,错误出现在OutputLayer:

Exception in thread "main" org.deeplearning4j.exception.DL4JInvalidInputException: 非矩阵输入;预期矩阵(秩为2),但得到秩为4的数组,形状为[1, 1000, 28, 28]。缺少预处理器或错误的输入类型?(图层名称:layer1,图层索引:1,图层类型:OutputLayer)


是我加载图像的尝试有问题,还是我的网络配置有误?

配置:

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
.list()
.layer(0, new ConvolutionLayer.Builder()
.nIn(3) // 输入数据点数。
.nOut(1000) // 输出数据点数。
.activation(Activation.RELU) // 激活函数。
.weightInit(WeightInit.XAVIER) // 权重初始化。
.build())
.layer(1, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
.nIn(1000)
.nOut(outputNum)
.activation(Activation.SOFTMAX)
.weightInit(WeightInit.XAVIER)
.build())
.build();



<details>
<summary>英文:</summary>

I&#39;m trying to use DeepLearning4j to categorize 32x32 images in numbers from 0-9.
I&#39;ve looked up a number of examples and tutorials, but always run into some exception when fitting the dataset to a network.

Im currently trying to use a ImageRecordReader with ParentPathLabelGenerator and RecordReaderDataSetIterator.

The images seem to load fine but i always run into a DL4JInvalidInputException when fitting.
    File parentDir = new File(dataPath);
    FileSplit filesInDir = new FileSplit(parentDir, NativeImageLoader.ALLOWED_FORMATS);
    ParentPathLabelGenerator labelMaker = new ParentPathLabelGenerator();

    BalancedPathFilter pathFilter = new BalancedPathFilter(new Random(), labelMaker, 100);
    InputSplit[] filesInDirSplit = filesInDir.sample(pathFilter, 80, 20);
    InputSplit trainData = filesInDirSplit[0];
    InputSplit testData = filesInDirSplit[1];

    ImageRecordReader recordReader = new ImageRecordReader(numRows, numColumns, 3, labelMaker);
    recordReader.initialize(trainData);

    DataSetIterator dataIter = new RecordReaderDataSetIterator(recordReader, 1, 1, outputNum);

When using DenseLayer:

Exception in thread "main" org.deeplearning4j.exception.DL4JInvalidInputException: Input that is not a matrix; expected matrix (rank 2), got rank 4 array with shape [1, 3, 32, 32]. Missing preprocessor or wrong input type? (layer name: layer0, layer index: 0, layer type: DenseLayer)


When using ConvolutionLayer the error occures at the OutputLayer:

Exception in thread "main" org.deeplearning4j.exception.DL4JInvalidInputException: Input that is not a matrix; expected matrix (rank 2), got rank 4 array with shape [1, 1000, 28, 28]. Missing preprocessor or wrong input type? (layer name: layer1, layer index: 1, layer type: OutputLayer)


Is my attempt at loading the images incorrect or is my network misconfigured?

Configuration:

MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
.list()
.layer(0, new ConvolutionLayer.Builder()
.nIn(3) // Number of input datapoints.
.nOut(1000) // Number of output datapoints.
.activation(Activation.RELU) // Activation function.
.weightInit(WeightInit.XAVIER) // Weight initialization.
.build())
.layer(1, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
.nIn(1000)
.nOut(outputNum)
.activation(Activation.SOFTMAX)
.weightInit(WeightInit.XAVIER)
.build())
.build();


</details>


# 答案1
**得分**: 0

使用`.setInputType`配置选项来定义网络是最简单的方法。它将为您设置所有必要的预处理器,并且还会计算所有正确的`.nIn`值。

再次查看这个示例:https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/mnist/MnistClassifier.java#L156

当您使用`.setInputType`的方式来设置网络时,您无需设置任何`.nIn`值 - 但您仍然可以在我链接的示例中看到这一点,但通常没有充分的理由这样做。

<details>
<summary>英文:</summary>

The easiest way is to use the `.setInputType` configuration option when defining the network. It will set up all the necessary pre-processors for you, and it will calculate all the correct `.nIn` values too.

Take another look at this example https://github.com/eclipse/deeplearning4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/convolution/mnist/MnistClassifier.java#L156

When you use the `.setInputType` way of setting up your network, you don&#39;t have to set any `.nIn` values at all - you still can, as is evident in the example I&#39;ve linked, but usually there is no good reason to do so. 

</details>



huangapple
  • 本文由 发表于 2020年4月5日 21:26:11
  • 转载请务必保留本文链接:https://java.coder-hub.com/61043337.html
匿名

发表评论

匿名网友

:?: :razz: :sad: :evil: :!: :smile: :oops: :grin: :eek: :shock: :???: :cool: :lol: :mad: :twisted: :roll: :wink: :idea: :arrow: :neutral: :cry: :mrgreen:

确定