Fully-Connected Network Schema Details
A fully-connected network (FCN) is composed of a sequence of fully-connected layers (also referred to as Dense layers), followed by an output layer. However, there are many other components that can be added to these networks to improve their performance or to expand the types of problems that can be solved. In this section, we present the details of the Zero2Neuro Fully Connected Schema and show how the different components can be configured using network arguments.
Fully-Connected Module
The base of a FCN is implemented using a sequence of fully-connected modules, which add functionality on top of Fully Connected Layers:

Components:
- Fully Connected Layer (also referred to as a Dense Layer): a layer that contains the specified number of hidden neurons; each receives input from all neurons in the previous layer.
--hidden_activation: non-linear activation function--number_hidden_units: number of hidden units in the layer- L1/L2 weight regularization (default = none): explicit form of regularization that penalizes the absolute value of weights (L1) or the square of the weights (L2). This penalty is added to the loss function. Typical choices for L1 are 1 … 0.001; typical choices for L2 are 0.001 … 0.00001
- Dropout:
- ```–dropout`` (float): during training, randomly deactivate neurons within the layer with the specified probability (default = none). This implements an implicit form of regularization. Reasonable values for this argument are 0.1 … 0.5
- Batch Normalization:
--batch_normalization(switch): scale and shift each output neuron of the module so their values individually fall within a standard normal distribution across a training batch. While computationally expensive, batch normalization can improve the overall speed of training for very deep networks. Default: none.
Notes:
- All arguments are shared across all Fully-Connected Modules in the network, with the exception that each has its own number of neurons (
--num_hidden_unitsaccepts a list of hidden neuron numbers, one for each layer).
Fully-Connected Network

Components:
- Inputs:
--input_shapespecifies the shape of the input to the neural network. Typically, this is a single integer.
- Tokenizer: translation from an input string to a sequence of integers
--tokenizerturns on tokenization.- For this feature, the input shape must be (1,)
- Embedding + Flatten: translates a sequence of integers into a sequence of vectors that are then appended together to form one large vector
--embedding_dims(int): length of the individual vectors
- Batch Normalization (switch): Normalize the input feature values
--batch_normalization_input(switch): Turn on batch normalization for the input features.
- Input Dropout: randomly drop out input features during training. This can help the network to learn representations that rely on different subsets of input features.
--dropout_input(float): Dropout probability (0..1)
- Fully Connected Layer: Final output layer
--output_shape(int sequence): The shape of the network output- L1/L2 weight regularization (default = none): Same as for the Fully-Connected Modules.