Fashion MNIST is a dataset of Zalando's article images—consisting of a training set of 60,000 examples and a test set of 10,000 examples. Each example is a 28x28 grayscale image, associated with a label from 10 classes. Fashion-MNIST is intended to serve as a direct drop-in replacement for the original MNIST dataset for benchmarking machine learning algorithms. It shares the same image size and structure of training and testing splits.
The picture below shows the formal representation of the Convolutional Neural Network for this dataset. The architecture consists of a convolutional block, followed by a MaxPooling and a Flattening operation; then there are 3 Fully Connected layers with ReLU activation functions. This network takes as input a 1x28x28 tensor and outputs a 1D vector of 10 elements.
In NeVer2 you can select the necessary blocks from the left toolbar; the requested nodes will appear on the canvas with default parameters. You can drag and drop them to any position in the canvas. To edit their parameters you can just double-click or right-click -> "Edit", while to delete them you can press DEL or right-click -> "Delete".
The first layer is defined by a Convolutional block, with the parameters defined as in the example; the input shape is 1,28,28 and the convolution is operated with a 5 pixel kernel filter.
After this you can add a MaxPool block and edit the parameters accordingly. You can connect the two blocks activating the drawing mode with the "Draw Line" button in the top-left corner or by pressing the CTRL+D shortcut and clicking first on the starting block and then on the next block.
The next node is a Flatten operation, that unrolls the 3D tensor into a 1D array; this operation is followed by a ReLU activation. In order to flatten the input to a 1D array the axis parameter is set to 0.
Finally the three Fully Connected layers resize the dimension to 1000, 100 and 10 neurons: this operation is performed by editing the out_features parameter. Each Fully Connected is followed by a ReLU activation.
Once the network is completed, you can train it. In the menu bar select "Learn..." -> "Train": the training window appears, where you can select the dataset and the learning parameters. Given the popularity of this dataset, both MNIST and fMNIST are available by default; the first time they will be downloaded and then will be stored in the NeVer2 working directory. There is a pre-defined dataset transform tailored for both convolutional and linear MNIST and fMNIST networks, which compose 2 or 3 transforms: pilToTensor and Normalize(1, 0.5) in both cases, Flatten in the linear case only.
The learning parameters can be set as follows:
Optimizer: at the moment only "Adam" (adaptive moment estimation) is supported, being
the most popular and effective gradient based optimization algorithm. Once selected, you can edit all
its related parameters.
Learning Rate Scheduler: like the Optimizer, only "ReduceLROnPlateau" is supported. It adjusts the
learning rate only when a plateau in model performance is detected, e.g. no change for a given number of
training epochs.
Loss Function: you can choose between "Cross Entropy" and "MSE Loss", depending on the network
structure.
Precision Metric: you can choose between "Inaccuracy" and "MSE loss".
Epochs: it defines the steps that the learning algorithm will perform through the training process.
Validation Percentage: (a number between 0 and 1) indicates the percentage of the dataset to use
as the validation set.
Training batch size and Validation batch size: it defines the dimension of the
training batches.
Cuda: it allows to take advantage of NVidia GPU architecture for the computation.
Train patience (optional): the number of epochs in which the loss may not decrease
before the training procedure is interrupted with early stopping.
Checkpoints root (optional): where to store the checkpoints of the training strategy, by
default the working directory of NeVer2.
Verbosity level (optional): how many batches between training log prints (default 1).
Once the parameters are set, clicking "Train Network" runs the training. Logs are printed in a box below the buttons.
The Best Loss Score label indicates the smallest training loss; in this case, with a 10 epoch long training the best loss is 0.0136, corresponding to 1.36%. Looking at the following diagram shows how increasing the training epochs improves significantly the training loss.
At the moment we do not support verification for Convolutional Neural Networks. This section will be updated in time.