maths

🎲 Probability Basics

Probability is simply the mathematics of uncertainty. It helps us predict what might happen when we don't know for sure (like rolling dice or predicting stock prices).

Expected Value ($\text{E}[X]$)

The Concept: The long-term average outcome.

ELI5 Analogy: Imagine you play a carnival game where you have a 50% chance of winning $10 and a 50% chance of winning nothing. Your expected value is not what you might win on one try, but the average amount you expect to win if you played that game thousands of times.

The Math: You take every possible outcome, multiply it by how likely it is to happen, and then add them all up. It’s a weighted average.

Standard Deviation ($\sigma$)

The Concept: A measure of spread or variability. Specifically, it is the square root of the average squared distance each result is from the mean — $\sigma = \sqrt{E[(X - \mu)^2]}$.

ELI5 Analogy: Imagine two groups of people:

1. Group A: Everyone is exactly 6 feet tall. (Low standard deviation.)
2. Group B: Heights range wildly from 4 feet to 8 feet. (High standard deviation.)

If you calculate the average height for both, it might be the same. But the standard deviation tells you which group is more predictable and consistent. A low $\sigma$ means "very reliable."

Normal Distributions (The Bell Curve)

The Concept: The most common pattern of data in nature. Most things tend to cluster around a central average.

ELI5 Analogy: Think about the heights of adult humans, or the scores on an IQ test. You will rarely find someone who is extremely short or extremely tall (relative to the population). Instead, most people are clustered right in the middle. This symmetrical, bell-shaped curve is the Normal Distribution.

Why it matters: Because so many natural phenomena follow this pattern, we can use its predictable math to make educated guesses about things we haven't measured yet.

Log Likelihood

The Concept: A way to measure how "surprised" or "confident" a model is by the data it sees. It measures the probability of observing your actual data given your model's parameters.

ELI5 Analogy: Imagine you are trying to guess what kind of animal lives in a certain forest. You have two models:

1. Model A (High Likelihood): Predicts "deer" because 90% of the time, people who visit this forest see deer tracks. This model is very confident and matches the data well.
2. Model B (Low Likelihood): Predicts "polar bear." Seeing a polar bear in that forest would be extremely unlikely. The log likelihood for Model B would be very low.

The Math: We use logarithms ($\log$) because multiplying many small probabilities together can lead to numbers so tiny they become unmanageable for computers (they underflow). Taking the logarithm turns multiplication into addition, which is much easier and more stable for computation.

Maximum Likelihood Estimators (MLE)

The Concept: Finding the best possible settings (parameters) for your model that make the data you actually observed the most probable.

ELI5 Analogy: You are trying to figure out how many apples ($\theta$) a farmer picked last year, but all you have is a picture of the orchard today. MLE says: "Let's find the number of apples ($\theta$) that makes it most likely that this photo was taken."

The Goal: We don't just want parameters that fit the data; we want the parameters that make the data look like it came from a natural, probable process.

Random Variables

The Concept: A variable whose value is determined by chance (randomness). It’s not a number you measure directly; it represents a potential outcome.

ELI5 Analogy: If I flip a coin, the result (Heads or Tails) is my random variable. The actual physical coin isn't the variable; the outcome of the flip is the variable. We can calculate probabilities about this variable (e.g., $P(\text{Heads}) = 0.5$).

Central Limit Theorem (CLT)

The Concept: A powerful, magical rule that says: If you take enough independent random samples from a population with a finite mean and variance (no matter how weirdly shaped its distribution is), and then you calculate the average of those samples, the resulting averages will start to look like a Normal Distribution (the Bell Curve).

ELI5 Analogy: Imagine I give you bags of marbles. One bag might have red marbles 90% of the time and blue marbles 10% of the time. Another bag might be totally random. If you take enough samples from either bag, and then calculate the average proportion of red marbles in those samples, that average will always start to look like a perfect bell curve centered around the true mean.

Why it matters: It allows us to use the predictable math of the Normal Distribution even when we have no idea what the original data distribution looked like!

📐 Calculus Basics

Calculus is fundamentally the mathematics of change. It helps us answer questions like: "How fast?" or "If I change this, how much will that change?"

Gradients

The Concept: A vector (a list of numbers) that points in the direction of the steepest ascent (the fastest way up a hill). The magnitude of the gradient tells you how steep that climb is.

ELI5 Analogy: Imagine you are blindfolded and standing somewhere on a giant, hilly landscape. You want to find the highest point as quickly as possible. You don't know where it is, but if you feel around your feet, the direction that feels steepest going up is the gradient. If the gradient is zero, you are at the bottom of a valley or the top of a peak (a flat spot).

In ML: We use gradients to find the minimum error. Since we want to minimize error, we actually move in the opposite direction of the gradient (downhill).

The Chain Rule

The Concept: A rule for calculating derivatives (rates of change) when one function depends on another function, which itself might depend on a third function, and so on. It allows us to "chain" rates together.

ELI5 Analogy: Imagine you are driving a car whose speed is determined by how hard your foot presses the gas pedal, but the amount of acceleration you get from the gas pedal depends on how much fuel is left in the tank.

• Your Speed depends on your Acceleration.
• Your Acceleration depends on the Gas Pedal Input.
• The rate at which your speed changes (the derivative) must account for all three dependencies. The Chain Rule tells you how to multiply these rates together:

$\frac{d(\text{Speed})}{d(\text{Time})} = \frac{d(\text{Speed})}{d(\text{Acceleration})} \times \frac{d(\text{Acceleration})}{d(\text{Gas Pedal Input})} \times \frac{d(\text{Gas Pedal Input})}{d(\text{Time})}$

The Intuition for Backpropagation

The Concept: Backpropagation (short for "backward propagation") is the process of calculating the gradient (the error signal) through every single connection and parameter in a neural network. It uses the Chain Rule repeatedly.

ELI5 Analogy: Imagine you are running a complex assembly line that builds a toy car. The final product comes out, and it's broken (that's your Error). You need to figure out which worker (parameter) on the line was most responsible for the breakage.

1. Forward Pass: The raw materials go through the workers in order (Worker 1 $\rightarrow$ Worker 2 $\rightarrow$ ... $\rightarrow$ Final Product). This is how the prediction happens.
2. Error Calculation: You measure the final error at the end of the line.
3. Backward Pass (Backprop): Instead of starting from the beginning, you start at the broken product and work backward. The error signal flows backward through the chain:
   • The final error tells Worker N how badly they failed.
   • Worker N passes a fraction of that blame/error back to Worker N-1.
   • Worker N-1 then adjusts their own parameters based on the blame received, and passes the remaining blame further back... until you know exactly which worker (parameter) needs the most adjustment to fix the error.

The Key Takeaway: Backpropagation is nothing more than applying the Chain Rule repeatedly across all layers of a neural network to efficiently calculate $\frac{\partial \text{Error}}{\partial \text{Weight}}$, telling us how much each individual weight contributed to the final mistake.