Gradient Descent Calculator

Iterate downhill from a starting point with adaptive gradients, backtracking steps, a path visual, and convergence diagnostics.

Results are calculated automatically as you enter data.

Function f(x,y)

Start x Start y Maximum learning rate Iterations Gradient tolerance

Momentum mode Momentum beta

Step-by-step slider

Formula and interpretation

Result Enter a function and starting point to run gradient descent.

Disclaimer: All tools on this site are intended for educational and informational purposes only. Not professional advice. No guarantee of accuracy; verify units, formulas, assumptions, tolerances, rounding, and limitations before important use. See our Terms of Use.

Assumptions and limitations

Results are educational estimates, not professional engineering, construction, safety, scientific, legal, financial, or academic advice. No guarantee is made that outputs are accurate, complete, suitable, or error-free. Verify units, formulas, assumptions, tolerances, rounding, and limitations independently, and use qualified professional review for technical or safety-sensitive work. This result uses a browser-based numerical approximation. Check tolerance, convergence, units, and method limits before relying on it. Use of this calculator is subject to our Terms of Use.

▼ See explanations and tips below ▼

Related Calculators

What Is Gradient Descent?

Gradient descent is an iterative method for finding a lower value of a function. Instead of solving for the minimum in one algebraic step, it starts from an initial point, estimates which direction is uphill, and then moves in the opposite direction.

For a two-variable function $f(x,y)$, you can imagine standing on a surface where the height is the value of the function. The gradient points in the direction of steepest local increase. Gradient descent moves against that direction, so the next point is usually lower than the current one when the step size is reasonable.

This idea is used throughout numerical optimization, machine learning, statistics, engineering, economics, and applied mathematics. It is especially useful when a function is too complicated to minimize by hand or when only numerical evaluations of the function are available.

Why Gradient Descent Matters

Many real problems can be written as minimization problems. A model may try to minimize prediction error, an engineer may try to minimize cost or energy, and a student may want to understand how an algorithm searches for a minimum.

Gradient descent matters because it turns a difficult optimization problem into a repeatable sequence of small steps:

Evaluate the function near the current point.
Estimate the local slope.
Move downhill.
Repeat until the result is good enough or the chosen iteration limit is reached.

The method is simple, but its behavior depends strongly on the shape of the function, the starting point, the learning rate, and whether the gradient estimate is reliable.

Key Terms to Know

Objective function: The function being minimized. In this calculator, the objective is a two-variable expression written as $f(x,y)$.
Current point: The current coordinate pair $(x_k,y_k)$ at iteration $k$.
Gradient: A vector of partial derivatives. For $f(x,y)$, it describes how the function changes as $x$ and $y$ change.
Estimated gradient: A numerical approximation of the gradient, rather than an exact symbolic derivative.
Learning rate: The positive maximum trial step, usually written as $\alpha$. The calculator reduces it automatically when a full step does not satisfy the descent condition.
Iteration: One update step of the algorithm. Iteration $0$ is the starting point before any update is applied.
Gradient norm: The length of the gradient vector. A small gradient norm often suggests that the path is near a stationary point.
Momentum: A modification that carries part of the previous movement into the next update, like a velocity term.
Convergence: The process of the iterates settling toward a minimum or stationary point.

How Gradient Descent Works

For a point

$$ z_k = \begin{bmatrix}x_k \\ y_k\end{bmatrix}, $$

the basic gradient descent update is

$$ z_{k+1} = z_k - \alpha \nabla f(z_k). $$

In words, the next point equals the current point minus the learning rate times the gradient. The minus sign is important: the gradient points uphill, so subtracting it moves downhill.

For a two-variable function, the gradient is

$$ \nabla f(x,y) = \begin{bmatrix} \frac{\partial f}{\partial x}(x,y) \\ \frac{\partial f}{\partial y}(x,y) \end{bmatrix}. $$

This calculator estimates the two partial derivatives numerically with central finite differences. For the $x$ direction, it samples the function slightly to the right and slightly to the left of the current point:

$$ \widehat{\partial_x f}(x,y) \approx \frac{f(x+h_x,y)-f(x-h_x,y)}{2h_x}. $$

For the $y$ direction, it uses the same idea above and below the point:

$$ \widehat{\partial_y f}(x,y) \approx \frac{f(x,y+h_y)-f(x,y-h_y)}{2h_y}. $$

The calculator tries several adaptive finite-difference step sizes based on the cube root of floating-point precision and the coordinate scale. It compares centered estimates with Richardson extrapolation, tracks roundoff uncertainty, and checks how the one-sided slopes change as the step changes. An estimate that is too uncertain or appears non-differentiable is reported as a numerical failure rather than used for an update.

$$ h_i \approx \sqrt[3]{\varepsilon}\max(1,|z_i|), \quad\text{with several nearby scales checked for reliability.} $$

The estimated gradient norm is the Euclidean length of the estimated gradient:

$$ \|\widehat{\nabla f}(x,y)\|_2 = \sqrt{\widehat{\partial_x f}(x,y)^2 + \widehat{\partial_y f}(x,y)^2}. $$

A smaller gradient norm usually means the function is flatter at that point. That can indicate progress toward a minimum, but it is not a guarantee of a global minimum.

Plain Descent Update

The entered learning rate $\alpha$ is an initial maximum trial step. In plain descent mode, the calculator uses Armijo backtracking: it tries that cap, repeatedly halves an insufficiently descending trial, and records the accepted value $\alpha_k\leq\alpha$.

$$ x_{k+1} = x_k - \alpha_k\widehat{\partial_x f}(x_k,y_k), $$

$$ y_{k+1} = y_k - \alpha_k\widehat{\partial_y f}(x_k,y_k). $$

Momentum Update

When momentum is enabled, the calculator keeps a velocity vector. Starting with zero velocity, it uses the same accepted trial rate in the velocity update. If the momentum direction is not downhill, it resets the velocity and retries a plain descent direction.

$$ v_{k+1} = \beta v_k - \alpha_k\widehat{\nabla f}(z_k), $$

$$ z_{k+1} = z_k + v_{k+1}. $$

Here, $\beta$ controls how much of the previous velocity carries forward. Momentum can help a path move more steadily through a long, narrow valley, but it can also overshoot or oscillate when the settings are not suitable.

Examples of Gradient Descent in Practice

Example 1: A Simple Bowl-Shaped Function

Consider

$$ f(x,y) = (x-2)^2 + (y+1)^2. $$

This function has its minimum at $(2,-1)$, where $f(x,y)=0$. Suppose the starting point is $(-2,3)$ and the learning rate is $\alpha=0.18$.

The exact gradient for this example is

$$ \nabla f(x,y) = \begin{bmatrix}2(x-2) \\ 2(y+1)\end{bmatrix}. $$

At $(-2,3)$, the gradient is

$$ \nabla f(-2,3) = \begin{bmatrix}-8 \\ 8\end{bmatrix}. $$

One gradient descent step gives

$$ x_1 = -2 - 0.18(-8) = -0.56, $$

$$ y_1 = 3 - 0.18(8) = 1.56. $$

The function value falls from

$$ f(-2,3)=32 $$

$$ f(-0.56,1.56)=13.1072. $$

This is the basic goal of gradient descent: each reasonable step should move the point toward a lower value.

Example 2: Armijo Backtracking for a Large Trial Rate

Now consider

$$ f(x,y)=x^2+4y^2. $$

The $y$ direction is much steeper than the $x$ direction because of the coefficient $4$. The gradient is

$$ \nabla f(x,y)=\begin{bmatrix}2x \\ 8y\end{bmatrix}. $$

Starting from $(2,2)$ with an initial cap $\alpha=0.6$, the full trial would give

$$ x_1 = 2 - 0.6(4) = -0.4, $$

$$ y_1 = 2 - 0.6(16) = -7.6. $$

That trial raises the function value from

$$ f(2,2)=20 $$

$$ f(-0.4,-7.6)=231.2. $$

so Armijo backtracking rejects it. The calculator next tries $0.3$, then accepts $\alpha_1=0.15$. Its recorded first step is $(1.4,-0.4)$, where $f(1.4,-0.4)=2.6$. The rejected full trial is useful evidence that the initial cap was too large, but it is not the calculator's reported step.

Example 3: A Sharp Corner Can Mislead a Numerical Gradient

Finite differences work best when the function is smooth near the current point. A nonsmooth function can produce misleading information.

For example, consider

$$ f(x,y)=|x|+y^2. $$

At $x=0$, the derivative of $|x|$ is not defined. A central finite difference in the $x$ direction gives

$$ \frac{|0+h|-|0-h|}{2h}=\frac{h-h}{2h}=0. $$

That value looks flat, even though the function has a sharp corner. This is not an error in arithmetic; it is a limitation of using symmetric numerical samples near a nonsmooth point.

Example 4: The Rosenbrock Valley

The Rosenbrock function is often used to test optimization methods:

$$ f(x,y)=100(y-x^2)^2+(1-x)^2. $$

Its minimum is at $(1,1)$, but the path to that point lies through a long, narrow curved valley. Gradient descent may find the valley but then make slow progress along it even with backtracking. This example is useful because it shows that a function can have a simple formula but still be challenging for an iterative optimizer.

How to Interpret the Result

The selected step result shows the coordinates and function value at the currently selected iteration. This may be the final step, or it may be an earlier step selected with the slider or table.

The summary distinguishes the final iterate from the best objective value encountered. A nonconverged path can end at a point that should not be described as a minimum, even when it improved substantially from the start.

The final estimated gradient norm tells you how flat the function appears at the final point. A very small value often suggests that the path is near a stationary point. A large value suggests that the function is still sloped and more iterations, a different learning rate, or a different starting point may be needed.

The termination status states whether the gradient tolerance was reached, the path stagnated, the line search failed, a numerical estimate became unreliable, or the iteration limit was reached. Only the first is a converged stationary point, and its Hessian classification is still a local numerical assessment.

The backtracking diagnostics show how often the maximum learning rate had to be reduced and whether momentum was reset. They describe this run only and do not prove that the chosen maximum rate is globally optimal.

The graph shows the path through the $x$-$y$ coordinate plane. It does not show contour lines or a three-dimensional surface, so a straight-looking path on the graph does not necessarily mean the objective value changed smoothly.

Common Mistakes and Misconceptions

Mistake 1: Thinking the learning rate is just a minor detail.

The learning rate controls the step size. A small learning rate can make progress painfully slow, while a large learning rate can cause oscillation, overshooting, or divergence.

Mistake 2: Expecting exact derivatives.

This calculator estimates the gradient numerically. For smooth functions, central finite differences are often useful. Near discontinuities, sharp corners, or domain boundaries, the estimate may be inaccurate or undefined.

Mistake 3: Using implicit multiplication.

The expression parser accepts implicit multiplication, so 2x and x(y+1) are valid. You may instead write 2*x and x*(y+1) when explicit operators make a longer expression easier to read.

Mistake 4: Entering unsupported variables.

The function should use the variables $x$ and $y$. Other variable names are not part of the two-variable optimization problem.

Mistake 5: Treating a small gradient norm as a global-minimum guarantee.

A small gradient norm can indicate a stationary point, but a stationary point may be a local minimum, a saddle point, or a flat region. The starting point and function shape still matter.

Mistake 6: Assuming the graph is a contour plot.

The displayed path is a trace of the iterates in the $x$-$y$ plane. It does not by itself show the height of the objective function away from the path.

Mistake 7: Treating the best iterate as a proven minimum.

The best iterate is simply the lowest objective value seen on this path. It may be local, nonstationary, or limited by the starting point and iteration budget.

When to Use Gradient Descent

Use gradient descent when you want to explore how an iterative optimizer behaves on a two-variable function. It is especially helpful for:

visualizing how a starting point moves across an $x$-$y$ plane;
studying the effect of the learning rate;
comparing plain descent with momentum;
checking whether a function appears to decrease over iterations;
teaching or learning numerical optimization concepts;
experimenting with smooth objective functions such as quadratics or the Rosenbrock function.

For high-dimensional, constrained, noisy, or production-level optimization problems, use a dedicated optimization library or a method designed for that problem class.

Limitations and Things to Keep in Mind

This calculator is designed for finite real-valued functions of exactly two variables, $x$ and $y$. It does not solve higher-dimensional optimization problems, constrained optimization problems, stochastic gradient descent problems, adaptive optimizer problems, or complex-valued problems.

The gradient is estimated with adaptive centered finite differences, Richardson comparisons, forward/backward slope checks, and floating-point uncertainty estimates. If nearby samples are invalid, non-finite, non-differentiable, or below numeric resolution, the calculator keeps the valid path and reports a numerical failure.

The entered learning rate is an initial maximum. Each iteration uses Armijo backtracking and may accept a smaller value. The accepted learning rate is shown in the path table.

Momentum starts with zero velocity, and beta must be from $0$ through $0.99$. If momentum points uphill, the velocity is reset before the line search retries a plain descent direction.

Iteration $0$ records the starting point. A run that completes all $N$ requested updates contains $N+1$ points; convergence, stagnation, a failed line search, or a numerical failure can stop earlier and leave a shorter valid path.

A run can stop because the gradient tolerance was reached, the path stagnated at floating-point resolution, the line search could not find a reliable downhill step, a numerical estimate failed, or the iteration limit was reached. Nonconverged runs retain their valid path and are warnings, not reported minima.

When the gradient tolerance is reached, the calculator estimates a local Hessian and reports a likely local minimum, saddle point, likely local maximum, or an inconclusive flat point. This is an uncertainty-aware local classification, not proof of a global minimum.

Displayed numbers are rounded for readability. Very small values may display as $0$, and very large or very small finite values may display in exponential notation. Use the result as a numerical approximation, not as an exact symbolic solution.

Iterations must be a whole number from $1$ through $300$. Long paths may show only a window of table rows around the selected iteration.

For coursework, research, engineering, or other important decisions, verify the result with analytical derivatives, independent software, or a qualified instructor or professional when appropriate.

How to Use This Calculator

Enter a finite two-variable function $f(x,y)$ using $x$, $y$, numbers, operators, parentheses, and supported math functions.
Enter the starting coordinates for $x$ and $y$.
Enter a positive maximum learning rate $\alpha$.
Enter a whole-number iteration limit from $1$ through $300$, and a positive gradient tolerance.
Optionally enable momentum and enter beta from $0$ through $0.99$.
Review the selected step, best and final objective values, gradient uncertainty, accepted learning rates, termination status, stationary classification, graph, and iteration table.
Use the step slider or iteration table to inspect earlier and later points in the descent path.
Adjust the learning rate, iteration count, starting point, or momentum setting if the path increases, oscillates, or changes too slowly.

Frequently Asked Questions

What does the learning rate do?

The entered learning rate $\alpha$ is the largest step the calculator will try. If that step does not produce sufficient descent, Armijo backtracking repeatedly halves it. A very small cap can still make progress slow.

What does the estimated gradient norm mean?

The estimated gradient norm is the length of the numerical gradient vector at a point. A small norm means the function appears locally flat under the finite-difference estimate. It can suggest convergence, but it does not prove that the point is the global minimum.

Why can the function value increase during gradient descent?

The function value can increase when the step is too large, the function is steep or curved, momentum carries the path too far, or the numerical gradient is unreliable. Occasional increases can happen in some methods, but frequent increases are usually a sign to reduce the learning rate or reconsider the function settings.

Why does the calculator use finite differences instead of exact derivatives?

Finite differences allow the calculator to estimate a gradient from function values alone. This makes it flexible for many entered expressions, but it also means the gradient is approximate. Exact symbolic derivatives can be more accurate when they are available and correct.

Does momentum always improve gradient descent?

No. Momentum can help when successive gradients point in similar directions, especially in narrow valleys. It can also cause overshooting or oscillation if beta or the learning rate is poorly chosen.

Can gradient descent find the global minimum?

Sometimes, but not always. For well-behaved convex functions, gradient methods have stronger guarantees under suitable conditions. For nonconvex functions, the result can depend on the starting point, learning rate, function shape, and stopping rule.

Sources and References

Books

Jorge Nocedal and Stephen J. Wright. Numerical Optimization. 2nd ed., Springer, 2006. Relevant chapters: “Fundamentals of Unconstrained Optimization,” “Line Search Methods,” and “Calculating Derivatives.” Springer book page.
Stephen Boyd and Lieven Vandenberghe. Convex Optimization. Cambridge University Press, 2004. Relevant chapter: “Unconstrained Minimization.” Stanford book page and Cambridge book page.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016. Relevant chapter: “Optimization for Training Deep Models,” especially the sections on stochastic gradient descent, learning rate behavior, and momentum. Online book chapter.

Online and Educational Sources

Stephen Boyd. “Unconstrained Minimization.” EE364A Lecture Slides, Stanford University. Used for descent method structure, gradient descent update direction, and gradient-norm stopping criteria. Lecture slides.
Autar Kaw and contributors. “2.02: Numerical Differentiation of Continuous Functions.” Mathematics LibreTexts, accessed June 28, 2026. Used for central finite-difference derivative approximation. LibreTexts page.
Eric W. Weisstein. “Rosenbrock Function.” MathWorld—A Wolfram Resource, accessed June 28, 2026. Used for the Rosenbrock function form, common parameters, and global-minimum context. MathWorld page.