Which of the following explains back propagation?

Backpropagation, short for "backward propagation of errors," is the core algorithm for training artificial neural networks. The process involves a forward pass where input data is fed through the network to produce an output, which is then compared to the correct output to calculate an error value (loss). In the subsequent backward pass, this error is propagated from the output layer back through the network's hidden layers. The algorithm uses the chain rule of calculus to calculate the gradient of the loss function with respect to each weight and bias in the network. These gradients are then used by an optimization algorithm, such as gradient descent, to update the weights and biases to minimize the overall error.

A. Convolutions are specific mathematical operations used in certain network types (CNNs); they are not the values that are propagated backward to update weights.

B. Accuracy is a performance metric calculated after a prediction is made. The algorithm propagates the raw error or loss value, not the accuracy percentage.

C. Nodes (or neurons) are the structural components of the network. The error signal is what passes backward between them, not the nodes themselves.

1. Rumelhart

D. E.

Hinton

G. E.

& Williams

R. J. (1986). Learning representations by back-propagating errors. Nature

323(6088)

533–536. In the abstract

the authors introduce the "back-propagation" learning procedure that "repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector." This difference is the error. https://doi.org/10.1038/323533a0

2. Stanford University. (n.d.). CS231n: Convolutional Neural Networks for Visual Recognition. Module 1: Neural Networks Part 3: Learning and Evaluation

Section: "Backpropagation". The course notes state

"Backpropagation is a clever way of computing gradients of expressions through recursive application of chain rule... The core idea is that we can compute the gradient of the loss function with respect to the weights of a neural network by propagating gradients backward through the network."

3. Nielsen

M. A. (2015). Neural Networks and Deep Learning. Determination Press. Chapter 2

"How the backpropagation algorithm works

" Section: "The four fundamental equations behind backpropagation." The text explains that the algorithm is a method for computing the gradient of the cost function

starting by computing the error (δL) in the output layer and then propagating this error backward to compute the error in earlier layers.