[{"_1":2,"_52":-5,"_53":-5},"loaderData",{"_3":4,"_7":8,"_51":-5},"root",{"_5":6},"path","/blogs/tags/neural-network","pages/Tag",{"_9":10,"_47":48,"_19":22,"_49":50},"posts",[11],{"_12":13,"_14":15,"_16":17,"_19":20,"_25":26,"_27":28,"_29":30,"_33":34,"_37":38,"_39":40},"id","clzsoiiid00da07pjuwru2lu6","publishDate","2024-08-14","category",[18],"tutorial","tag",[21,22,23,24],"rust","neural-network","machine-learning","deep-learning","slug","building-a-neural-network-from-scratch-in-rust","title","Building a Neural Network from Scratch in Rust","excerpt",{"_31":32},"text","Learn how to build a neural network from scratch in Rust, covering initialization, forward pass, backpropagation, and training on the XOR dataset.","coverImg",{"_35":36},"url","https://ap-south-1.graphassets.com/AHDh3MjtFRAmkhgRu025Qz/eAnLoIvdSnCXPXHHPyhQ","content","# Introduction\n\nIn this blog, we will build a simple neural network from scratch in Rust. We'll start by setting up our project, then implement the core components of a neural network, and finally train it on a basic dataset.\n\n# Project Setup\n\nFirst, let's set up a new Rust project. Open your terminal and run:\n\n```sh\ncargo new neural_network\ncd neural_network\n```\nIf you're unfamiler with ML Models in Rust, check our previous guide on [how to build a machine learning model in rust](/blogs/tutorial/how-to-build-a-machine-learning-model-in-rust).\n\nNext, we'll add the `ndarray` crate for numerical operations and `rand` crate for random number generation. Update your `Cargo.toml` file to include these dependencies:\n\n\n```toml\n[dependencies]\nndarray = \"0.15\"\nrand = \"0.8\"\n```\n\n# Implementing the Neural Network\nWe'll start by creating a network.rs file in the src directory to hold our neural network implementation.\n\n**Defining the Network Structure**\nCreate a Network struct that will hold our weights and biases:\n\n```rust\n// src/network.rs\nuse ndarray::{Array1, Array2, Axis};\nuse rand::thread_rng;\nuse rand::Rng;\n\npub struct Network {\n weights1: Array2,\n biases1: Array1,\n weights2: Array2,\n biases2: Array1,\n}\n\nimpl Network {\n pub fn new(input_size: usize, hidden_size: usize, output_size: usize) -> Self {\n let mut rng = thread_rng();\n\n let weights1 = Array2::from_shape_fn((hidden_size, input_size), |_| rng.gen_range(-1.0..1.0));\n let biases1 = Array1::from_shape_fn(hidden_size, |_| rng.gen_range(-1.0..1.0));\n let weights2 = Array2::from_shape_fn((output_size, hidden_size), |_| rng.gen_range(-1.0..1.0));\n let biases2 = Array1::from_shape_fn(output_size, |_| rng.gen_range(-1.0..1.0));\n\n Network {\n weights1,\n biases1,\n weights2,\n biases2,\n }\n }\n}\n```\n\n## Forward Pass\nImplement the forward pass of the network, which involves calculating the activations for each layer:\n\n```rust\nimpl Network {\n fn sigmoid(x: &Array1) -> Array1 {\n x.mapv(|x| 1.0 / (1.0 + (-x).exp()))\n }\n\n fn sigmoid_derivative(x: &Array1) -> Array1 {\n x * &(1.0 - x)\n }\n\n pub fn forward(&self, input: &Array1) -> (Array1, Array1, Array1) {\n let hidden_input = self.weights1.dot(input) + &self.biases1;\n let hidden_output = Self::sigmoid(&hidden_input);\n let final_input = self.weights2.dot(&hidden_output) + &self.biases2;\n let final_output = Self::sigmoid(&final_input);\n\n (hidden_output, final_input, final_output)\n }\n}\n```\n\n## Backpropagation\nAdd backpropagation to adjust the weights and biases based on the error:\n\n```rust\nimpl Network {\n pub fn train(&mut self, input: &Array1, target: &Array1, learning_rate: f64) {\n let (hidden_output, final_input, final_output) = self.forward(input);\n\n let output_errors = target - &final_output;\n let output_delta = &output_errors * &Self::sigmoid_derivative(&final_output);\n\n let hidden_errors = self.weights2.t().dot(&output_delta);\n let hidden_delta = &hidden_errors * &Self::sigmoid_derivative(&hidden_output);\n\n self.weights2 = &self.weights2 + &output_delta.insert_axis(Axis(1)).dot(&hidden_output.insert_axis(Axis(0))) * learning_rate;\n self.biases2 = &self.biases2 + &output_delta * learning_rate;\n\n self.weights1 = &self.weights1 + &hidden_delta.insert_axis(Axis(1)).dot(&input.insert_axis(Axis(0))) * learning_rate;\n self.biases1 = &self.biases1 + &hidden_delta * learning_rate;\n }\n}\n```\n\n## Training the Network\nNow, let's create a main file to train our network on a simple dataset.\n\n```rust\n// src/main.rs\nmod network;\nuse network::Network;\nuse ndarray::array;\n\nfn main() {\n let mut network = Network::new(2, 3, 1);\n\n let inputs = vec![\n array![0.0, 0.0],\n array![0.0, 1.0],\n array![1.0, 0.0],\n array![1.0, 1.0],\n ];\n\n let targets = vec![\n array![0.0],\n array![1.0],\n array![1.0],\n array![0.0],\n ];\n\n let learning_rate = 0.1;\n let epochs = 10000;\n\n for _ in 0..epochs {\n for (input, target) in inputs.iter().zip(targets.iter()) {\n network.train(input, target, learning_rate);\n }\n }\n\n for input in inputs.iter() {\n let (_, _, output) = network.forward(input);\n println!(\"{:?} -> {:?}\", input, output);\n }\n}\n```\n\n## Explanation\n\n- Network Initialization: We initialize the network with random weights and biases.\n- Forward Pass: We calculate the activations for the hidden and output layers using the sigmoid function.\n- Backpropagation: We calculate the error at the output and hidden layers, then update the weights and biases using gradient descent.\n- Training: We train the network on the XOR dataset for a specified number of epochs.\n- Testing: After training, we test the network on the same inputs to see how well it learned the XOR function.\n\n\n## Running the Code\n\n```sh\ncargo run\n```\nYou should see the network's predictions for the XOR function.\n\n# Conclusion\nIn this blog, we built a simple neural network from scratch in Rust. We covered the core components, including initialization, forward pass, and backpropagation. This example can be expanded to more complex networks and datasets, providing a solid foundation for neural network implementation in Rust.\n\nFeel free to experiment with different architectures, activation functions, and learning rates to see how they affect the network's performance.","author",{"_41":42,"_43":44,"_45":46},"name","Dexter","img","https://avatars.githubusercontent.com/u/71167911?v=4","social","https://github.com/dexter-xD","hasMorePosts",false,"page",1,"layouts/HomeLayout","actionData","errors"]