BACKPROBAGATION IN NEURAL NETWORK

Code Sample and Text

Backpropagation In Python

In the case of simple feedforward neural networks, the process of backpropagation can be done efficiently in Python.


First, we will import the necessary libraries.


import numpy as np

Next, we will define a simple neural network with one hidden layer. 


def sigmoid(x):
    return 1.0 / (1 + np.exp(-x))

def sigmoid_derivative(x):
    return x * (1.0 - x)

class NeuralNetwork:
    def __init__(self, x, y):
        self.input      = x
        self.weights1   = np.random.rand(self.input.shape[1],4) 
        self.weights2   = np.random.rand(4,1)                 
        self.y          = y
        self.output     = np.zeros(self.y.shape)

    def feedforward(self):
        self.layer1 = sigmoid(np.dot(self.input, self.weights1))
        self.output = sigmoid(np.dot(self.layer1, self.weights2))

    def backprop(self):
        # application of the chain rule to find derivative 
        #of the loss function with respect to weights2 and weights1
        d_weights2 = np.dot(self.layer1.T, (2*(self.y - self.output) * sigmoid_derivative(self.output)))
        d_weights1 = np.dot(self.input.T, (np.dot(2*(self.y - self.output) 
        * sigmoid_derivative(self.output), self.weights2.T) 
        * sigmoid_derivative(self.layer1)))

        # update the weights with the derivative of the loss function
        self.weights1 += d_weights1
        self.weights2 += d_weights2

Finally, we can use the NeuralNetwork class to train our neural network.


	# Training set
X = np.array([[0,0,1],
              [0,1,1],
              [1,0,1],
              [1,1,1]])
y = np.array([[0],[1],[1],[0]])

# Create neural network object
nn = NeuralNetwork(X, y)

# Train the network
for i in range(1500):
    nn.feedforward()
    nn.backprop()

print(nn.output)

In this code, we initialize our neural network with random weights. Then, we train the network by feeding forward the training inputs, comparing the network's output with the actual output, and backpropagating the errors to adjust the weights. Finally, we print the output of the trained neural network. Note: This code is for educational purposes and does not implement any optimizations such as stochastic gradient descent, mini-batch gradient descent, or Adam optimization algorithm. Also, the error calculation is done with the mean squared error, but other error functions like cross-entropy error can be used depending on the problem.

Comments

Post a Comment

Popular posts from this blog

CIFAR-10 Dataset Classification Using Convolutional Neural Networks (CNNs) With PyTorch

CIFAR-10 Image Classification with CNN (PyTorch) - Complete Guide CIFAR-10 Image Classification Using CNN with PyTorch This comprehensive guide demonstrates how to classify images from the CIFAR-10 dataset using Convolutional Neural Networks (CNNs) built with PyTorch. The project covers the entire machine learning pipeline from data loading to model deployment, including advanced techniques for improving performance. 🚀 Key Features Complete PyTorch implementation Data augmentation techniques Multiple CNN architectures Training visualization Model evaluation metrics Hyperparameter tuning Model saving/loading Deployment options 📊 Performance Baseline model: ~70% accuracy Improved mod...
Read more

Radial Basis Function Networks with PyTorch

Advanced Radial Basis Function Networks with PyTorch Advanced Radial Basis Function Networks (RBFNs) with PyTorch Introduction to RBF Networks Radial Basis Function Networks are a type of artificial neural network that uses radial basis functions as activation functions. They are particularly effective for pattern recognition and function approximation problems. Key Characteristics Three-layer architecture : Input layer, hidden RBF layer, and linear output layer Localized activation : Each neuron in the hidden layer responds only to inputs near its center Fast training : Often requires fewer iterations than multilayer perceptrons Universal approximation : Can approximate any continuous function given enough hidden units Mathematical Foundation : The RBF network implements a function of the form: f(x) = Σ w_i * φ(||x - c_i||) ...
Read more

Long-short-term-memory (LSTM) Word Prediction With PyTorch

LSTM Text Generation with PyTorch - Complete Guide LSTM Text Generation with PyTorch A comprehensive guide to building word-level language models with Long Short-Term Memory networks Introduction to LSTM Text Generation Long Short-Term Memory (LSTM) networks are a special kind of recurrent neural network (RNN) capable of learning long-term dependencies. They are particularly useful for sequence prediction problems like text generation, where the context from previous words is crucial for predicting the next word. Key Concepts Word-level modeling : Predicts the next word given previous words Embeddings : Dense vector representations of words ...
Read more