Network building

NeVer2 allows to add sequential layers directly by clicking on the corresponding buttons in the left toolbar. The layer block will be placed in the correct position with default parameters which can be edited directly. Note that adding a layer will block the editing of previous layers to maintain consistency. It is always possible to remove nodes at the end of a network with the "DEL" key. This also applies to the input block, where it is possible to change the input identifier and to set the number of inputs of the network.

The first block is a Fully Connected layer consisting of 64 neurons. To do so, it is sufficient to set the out_features parameter to 64 and to click on Save.

To add the other layers we select the corresponding block in the left toolbar. ReLU activation functions have no parameters and new Fully Connected layers can be added directly. In the example below it is presented the complete network. In this example it is also visible how the last layer computes the output which is then shown in the output block.

Network Training

Once the network is completed, it is time to train it on the dataset. In the menu bar, selecting "Learn..." -> "Train" displays the window for the training setup. The first thing to do is to select the dataset: we made directly available the MNIST and Fashion MNIST datasets, but it is possible to open any dataset as a text file selecting "Custom data source...".

In order to process correctly the dataset, it is required to provide information on the Data type to expect and the Delimiter character, which are set by default as float and ','.

The learning parameters can be set as follows:

Optimizer: at the moment only "Adam" (adaptive moment estimation) is supported, being the most popular and effective gradient based optimization algorithm. Once selected, you can edit all its related parameters.

Learning Rate Scheduler: like the Optimizer, only "ReduceLROnPlateau" is supported. It adjusts the learning rate only when a plateau in model performance is detected, e.g. no change for a given number of training epochs.

Loss Function: you can choose between "Cross Entropy" and "MSE Loss", depending on the network structure.

Precision Metric: you can choose between "Inaccuracy" and "MSE loss".

Epochs: it defines the steps that the learning algorithm will perform through the training process.

Validation Percentage: (a number between 0 and 1) indicates the percentage of the dataset to use as the validation set.

Training batch size and Validation batch size: it defines the dimension of the training batches.

Cuda: it allows to take advantage of NVidia GPU architecture for the computation.

Train patience (optional): the number of epochs in which the loss may not decrease before the training procedure is interrupted with early stopping.

Checkpoints root (optional): where to store the checkpoints of the training strategy, by default the working directory of NeVer2.

Verbosity level (optional): how many batches between training log prints (default 1).

Once the parameters are set, clicking "Train Network" runs the training. Logs are printed in a box below the buttons.

The Best Loss Score label indicates the smallest training loss; in this case, with a 10 epoch long training the best loss is 0.00048, corresponding to 0.048%. Looking at the following diagram shows how increasing the training epochs improves slightly the training loss.

James

Overview

Network building

Network Training

Network Verification

In progress...