---
title: "Vineet Theodore (@vf30kf) | Hashnode"
description: "Jack of all trades. Read the latest articles by Vineet Theodore on Hashnode."
url: "https://hashnode.com/@vf30kf"
---

@vf30kf

# Vineet Theodore

@vf30kf · Joined June 2025

Jack of all trades

## Comments

[In table 3, the function for Swish needs sigmoid(beta\*x), unless beta is identically 1. From what I understand, beta is itself a trainable parameter, making it key to differentiate it from SiLU, where beta is identically 1. Thus, the derivative should be: sigmoid(beta\*x) plus beta times x times derivative\_of\_the\_sigmoid function, where derivative of the sigmoid equals sigmoid(beta times x) times (1 minus sigmoid(beta times x)) Or if sigmoid is y, then derivative of y is equal to y times (1-y)](https://hashnode.com/posts/comprehensive-guide-to-the-relu-activation-function-in-neural-networks-definition-role-and-type-explained/66e18a8cdb99bf3dc707d638/comment/684b43a1a132ee2871d880ea)

Comment · Article · Jun 12, 2025 · [Comprehensive Guide to the ReLU Activation Function in Neural Networks: Definition, Role, and Type Explained](https://hashnode.com/posts/comprehensive-guide-to-the-relu-activation-function-in-neural-networks-definition-role-and-type-explained/66e18a8cdb99bf3dc707d638)
