From Differentials to Derivatives

Robert E. Bradley

Salvatore J. Petrilli Jr.

Communicated by Notices Associate Editor Adrian C. Rice

1. Introduction

Which came first, the total differential or the partial derivative?

This seems like a simple question, because the total differential of a function $z=f(x,y)$ is defined in all the textbooks as

$$\begin{equation} dz = f_x(x,y) \, dx + f_y(x,y) \, dy, \cssId{totdiffdef}{\tag{1}} \end{equation}$$

with the obvious modifications when $f$ is a function of three or more variables.

If we understand the question “Which came first?” in the historical sense, however, we get the opposite answer, because the total differential is as old as the calculus itself, whereas partial derivatives were only defined in the 18th century.

In the integral calculus, we learn how to find integrals of functions. However, the first part of calculus, in which we learn how to find derivatives, is called the differential calculus, not the derivative calculus. Similarly, the course where we learn to solve equations involving derivatives is called Differential Equations.

What’s a differential, then? Modern textbooks usually define it in the chapter on applications of the derivative as $dy = f'(x) \, dx$, analogously to 1. The definition is often motivated as giving a local linear approximation to a function. This is particularly unsatisfying, given that the section on differentials frequently follows the section on Newton’s method for finding roots. We only appreciate the true value of these differentials in a later chapter, when we encounter integration by substitution.

In 1696, however, in the first calculus textbook⁠Footnote¹ BPS15, 2, the Marquis de l’Hôpital (1661–1704) defined the differential as follows:

Definition II. The infinitely small portion by which a variable quantity continually increases or decreases is called the Differential.

That is, a differential is an infinitely small increment in a variable. In the 17th century, some mathematicians were quite comfortable making arguments about infinitely small quantities, or infinitesimals, even though their reasoning was far from rigorous by current standards. Arguments involving infinitesimals date back at least as far as the determination of the area of a circle by Johannes Kepler (1571–1630). In fact, Archimedes (ca. 287–ca. 212 BCE) used the closely related method of indivisibles as a method of discovery (as opposed to proof) in his treatise The Method⁠Footnote² Kat09, 103–110, 514–516.

A positive quantity was considered infinitely small if it was not zero, but smaller than any given positive quantity. It’s a fairly straightforward exercise in undergraduate analysis to show that no real number has this property. However, 17th-century mathematicians considered quantities in an informal sense, without precise definitions, as though numbers were given a priori. Infinitesimals could be conceived as being related to finite quantities in analogy to how finite quantities are related to infinity—roughly speaking, if $\alpha$ is infinitesimal, then $\alpha : 1\Colon 1 : \infty$. For example, even though $\infty$ wasn’t rigorously defined, it was clear to them that $\infty + 1 = \infty$. So it must be that $1 + \mathrm{infinitesimal} = 1$. More generally, if $x$ is a finite quantity (a real number) and $dx$ is an infinitesimal increment in $x$, then $x + dx = x$, even though $dx \ne 0$. This strange state of affairs was turned into an axiom by Johann Bernoulli (1667–1748); see Postulate I in Section 2.2.

2. Leibniz, Bernoulli, and L’Hôpital

2.1. Birth of the differential calculus

Isaac Newton (1642–1727) and Gottfried Wilhelm Leibniz (1646–1716) are usually credited with the discovery of the calculus. There was an ugly battle for the priority of this discovery around the turn of the 18th century, but we won’t dwell on that here. Interested readers can consult A. Rupert Hall’s book Hal80 for more details. Hall argued that although their discoveries were mathematically equivalent, they were, however, discovered independently; this is the prevailing view today among historians of mathematics.

Newton crafted his calculus of fluxions in the 1660s but did not publish it. Nevertheless, his discoveries were known to a small circle of his colleagues and correspondents. Leibniz worked out his version of the calculus in the 1670s and published his differential calculus in 1684 in a paper called “A New Method for Maxima and Minima as well as Tangents …” Lei84. In the opening paragraph of this paper, he introduced the variable $x$ for the abscissa and stated that “some straight line selected arbitrarily is called $dx$,” which he used to define other quantities $dv$, $dw$, and $dy$, based on proportion involving the variables $v$, $w$, and $y$, as well as the proportion of $dx$ to $x$. Leibniz was one of the great philosophers of his age and understood the metaphysical problems associated with infinitely small quantities. So he did not refer to the differential $dx$ as infinitely small but finessed the point entirely by making it an arbitrary quantity.

Figure 1.

Johann Bernoulli (1667–1748).

The Bernoulli brothers Jakob (1654–1705) and Johann soon mastered Leibniz’s differential and integral calculus—the latter published in 1686—to solve problems in both pure and applied mathematics, especially differential equations Kat09, 584–588. Leaving Basel, Switzerland, where Jakob held the chair in mathematics at the University, the underemployed Johann traveled to Paris in the autumn of 1691, where he met the Marquis de l’Hôpital. The Marquis had been an officer in the French army but had recently retired due to poor eyesight. As a young man, the Marquis had displayed a talent for mathematics, and in his retirement, he devoted himself to mastering the mathematics of Descartes and Leibniz. The Marquis immediately appreciated the depth of Johann’s knowledge and hired him as a tutor. Bernoulli gave him lessons, first in his Paris apartment through the winter and spring of 1691–1692 and then at his summer residence in Oucques, on the Loire River. The most important result of this partnership was l’Hôpital’s publication of Analyse des Infiniment Petits⁠Footnote³ in 1696, the first textbook on the differential calculus. It was based on the course notes that Johann Bernoulli had provided him during the winter of 1691–1692. For more on the relationship between the mathematician and the aristocrat, as well as the priority issue for the Analyse, see BPS15, vii-xvi.

2.2. Differentials in the Analyse

Although L’Hôpital’s definition of the differential (Definition II ) seems rather opaque on the first reading, the situation was clarified to some extent by Johann Bernoulli’s two postulates for the differential calculus BPS15, 3:

Figure 2.

Postulate II illustrated in the Analyse.

Neither these postulates nor the definition of the differential itself give us any clues about how to calculate differentials, so l’Hôpital instructed us on the process by means of an example:

Let $a + x + y -z$ be the expression whose differential we are to take. If we suppose that $x$ is increased by an infinitely small quantity, i.e., that it becomes $x + dx$, then $y$ becomes $y + dy$ and $z$ becomes $z + dz$. As for the constant $a$, it remains $a$. Thus, the given quantity $a + x + y -z$ becomes $a + x + dx + y + dy - z - dz$, and the differential, which is found by subtracting the former from the latter, is $dx + dy - dz$. It is similar for other expressions, giving rise to the following rule.

Rule I. For quantities added or subtracted. We take the differential of each term in the given quantity and, keeping the signs the same, we add them together in a new quantity which is the differential that we wish to find.

Next up was the differential of a product. If $z=xy$ and we increment each variable by its differential, we get

$$\begin{equation*} z + dz = (x + dx)(y+dy) = xy + y\, dx + x\, dy +dx\, dy. \end{equation*}$$

Figure 3.

Guillaume François Antoine de l’Hôpital (1661–1704).

Subtracting the original equation, we see that $dz = y\, dx + x\, dy +dx\, dy$. The crucial step is the application of Postulate I: Because $dx\, dy$ is infinitely smaller than both $y\, dx$ and $x \, dy$, it is infinitely smaller than their sum. Hence, $dz = d(xy) = y\, dx + x\, dy$. Modern readers will immediately recognize the form of the Product Rule. L’Hôpital also considered the products of three factors and four factors. By an informal induction, he therefore concluded:

Rule II. For multiplied quantities. The differential of a product of several quantities multiplied together is equal to the sum of the products of the differential of each of the quantities multiplied by the product of the others.

Not surprisingly, l’Hôpital tackled the Quotient Rule next. The fact that Leibniz’s calculus applies to all equations—not just equations of functions—makes it particularly easy. If

$$\begin{equation*} z = \frac{x}{y}, \end{equation*}$$

then $yz=x$. Applying the Product Rule, we have $y\,dz + z\,dy = dx$. Solving for $dz$, we have

$$\begin{align*} dz &= \frac{1}{y}\left(dx - z\,dy\right) \\ &= \frac{1}{y}\left(dx - \frac{x}{y}\,dy\right) \\ &= \frac{y\,dx - x\,dy}{y^2}. \end{align*}$$

Once again, modern readers immediately recognize this as equivalent to today’s formulation of the Quotient Rule. L’Hôpital expressed it this way:

Rule III. For divided quantities, or fractions. The differential of any fraction is equal to the product of the differential of the numerator by the denominator minus the product of the differential of the denominator by the numerator all divided by the square of the denominator.

The Power Law was next. L’Hôpital’s goal was to show that if $y=x^n$, then $dy = n x^{n-1} \, dx$. He did this in stages: first for natural numbers, then for negative integers, and finally for rational numbers. He referred to integer powers as perfect and fractional powers as imperfect. He had no need for the case of irrational exponents. The use of negative and fractional exponents was still quite novel in 1695, so l’Hôpital first reviewed the meaning and manipulation of such exponents.

The Power Law for natural numbers is a simple consequence of Rule II. For negative integers, he applied Rule III to $y=x^{-m}=\frac{1}{x^m}$ for a natural number $m$. Finally, if $y=x^{\frac{m}{n}}$ for integers $m$ and $n$, then we have $y^n=x^m$. Applying the Power Law for integers to both sides, we have $ny^{n-1}\,dy = mx^{m-1}\,dx$, so that

$$\begin{align*} dy &= \frac{m}{n} \frac{x^{m-1}}{\left( x^{\frac{m}{n}}\right)^{n-1}} \, dx\\ &= \frac{m}{n} x^{m - 1 - m(1 - \frac{1}{n})} \, dx\\ &= \frac{m}{n} x^{\frac{m}{n} -1} \, dx. \end{align*}$$

As with the other rules of differentials, l’Hôpital gave the Power Rule in a didactic form.

Rule IV. For powers, both perfect and imperfect. The differential of any power, whether perfect or imperfect, of a variable quantity is equal to the product of the exponent of this power by the same quantity raised to a power diminished by unity, and multiplied by its differential.

As we have noted, Leibniz’s differential calculus is an algorithm that may be applied to any algebraic equation. The function concept was only developed in the 18th century; see Thi05.

Using modern notation, let’s suppose that we apply Rules I–IV to an equation of the form $y=f(x)$, for some algebraic function $f(x)$. It’s clear from the rules of this differential calculus that we will get a result of the form $dy = g(x) \, dx$ for some function $g(x)$. Furthermore, because of the correspondence between Rules I–IV and the rules of the modern calculus of derivatives, an induction on the composition of the function $f(x)$ will yield the result $g(x)=f'(x)$. That is, $dy=f'(x) \, dx$. This justifies the use of Leibniz’s notation in modern textbooks: A formal division of both sides by $dx$ yields

$$\begin{equation*} \frac{dy}{dx} = f'(x). \end{equation*}$$

Leibniz’s algorithm survives to a limited extent in modern textbooks in the sections on implicit differentiation and related rates. For example, if an equation in two variables is differentiated implicitly and then formally multiplied by $dx$, the result is precisely the same as applying Leibniz’s algorithm.

2.3. Finding tangent lines

Curiously, the expression $\frac{dy}{dx}$ did not appear anywhere in l’Hôpital’s Analyse. This was largely because 17th-century mathematicians, still strongly influenced by the Classical Greek tradition, would have considered the solution to a problem of finding a line as a question involving a geometrical construction, not one of finding a linear equation in $x$ and $y$. When l’Hôpital used the differential calculus to find tangent lines to curves, which is the topic of Chapter 2 of his Analyse, he instead found the length of the subtangent of the tangent line. In modern terms, the subtangent is the portion of the $x$-axis between the abscissa—the point $P$ in Figure 4, with coordinate $(x,0)$—and the point $T$ where the tangent line intersects the axis. The tangent triangle $\Delta TPM$ in Figure 4, consisting of the subtangent $TP$, the ordinate $PM$, and the tangent $MT$, is similar to the differential triangle $\Delta MRm$. Using the similarity of the triangles, we find that

$$\begin{equation} \frac{TP}{PM} = \frac{MR}{Rm}. \cssId{difftriangle}{\tag{2}} \end{equation}$$

Figure 4.

The tangent triangle and the differential triangle.

Of course, in the finite triangle $\Delta RmM$ depicted in Figure 4, the point $m$ lies on the curve $AMm$ and not on the tangent line $TM$. However, when $Pp=dx$ is infinitely small, then by Postulate II, the portion $Mm$ of the curve is a straight line, so that $m$ lies on both the curve and the tangent line. Because $MR=Pp=dx$, $Rm=dy$, and $PM=y$, equation 2 gives the length of the subtangent as

$$\begin{equation*} PT = \frac{y \, dx}{dy}. \end{equation*}$$

When $PT > 0$, as in Figure 4, then $T$ lies on the same side of $P$ as the origin $A$ does. When $PT < 0$, then $T$ lies on the opposite side. When $PT = 0$, then the tangent line is vertical. Finally, when $PT = \infty$, the tangent line is horizontal. Therefore, when l’Hôpital treated max/min problems in Chapter 3 of his Analyse, he solved $dy=0$.

3. Higher Orders, Many Variables

3.1. Higher-order differentials

Second derivatives play an important role in the derivative calculus, so it’s little surprise that the differential calculus considers second- (and higher-) order differentials.

Once again, let’s consider an equation of the form $y=f(x)$. We’ve seen that in this case $dy=f'(x)\,dx$. If we now take differentials of this equation, we must use the Product Rule:

$$\begin{align} d(dy) &= d(f'(x)) \,dx + f'(x) d(dx) \\ &= f''(x) \, (dx)^2 + f'(x) d(dx). \cssId{second-d}{\tag{3}} \end{align}$$

In order to simplify the term $d(dx)=d^2 x$, Leibniz (and Bernoulli and l’Hôpital) reasoned as follows. The differential triangle has three sides: $dx$ ($Pp=MR$ in Figure 4), $dy$ ($Rm$), and $ds$ ($Mm$). The actual values of $dx$, $dy$, and $ds$ are not important, only the ratios $dx\Colon dy\Colon ds$. Thus, we are free to choose one of them arbitrarily. By convention, we will choose $dx$ to be constant—i.e., $dx$ is the same for every value of $x$. As a consequence, $d(dx)=0$. Therefore, equation 3 simplifies to

$$\begin{equation*} d^2y = f''(x) \, dx^2. \end{equation*}$$

Once again, we see that this justifies the modern notation $\frac{d^2y}{dx^2}$ for $f''(x)$.

L’Hôpital occasionally had need of third-order differentials in the Analyse, which of course satisfy the relation

$$\begin{equation*} d^3y = f'''(x) \, dx^3. \end{equation*}$$

L’Hôpital introduced second-order differentials in Chapter 4 of the Analyse and used them to find inflection points and cusps. They also figured prominently in Chapter 5, where he found the radius of curvature to be

$$\begin{equation*} \frac{\left(dx^2+dy^2\right)^{\frac{3}{2}}}{-dx\,d^2y}. \end{equation*}$$

This is the formula that Johann Bernoulli shared with l’Hôpital at Father Malebranche’s salon in Paris in November 1691, which prompted the Marquis to hire the 24-year-old Bernoulli as his tutor.

3.2. Many variables

So far, we have only considered the differential calculus in equations involving two variables. However, the calculus presented by l’Hôpital was in no way limited in the number of variables in the equation. Indeed, l’Hôpital’s first example, used to justify the Sum and Difference Rules, involves an expression in three variables.

An equation in three variables can take many forms but, with an eye to multivariate calculus as we currently practice it, let’s consider an equation of the form $z=f(x,y)$. As a motivation for partial derivatives, one may ask Calculus 3 students to consider a simple but representative example by manipulaing an equation like

$$\begin{equation*} z = x^2 - 4xy + y^3. \end{equation*}$$

We determine $\Delta z$ by incrementing $x$ by $\Delta x$ and $y$ by $\Delta y$. L’Hôpital would have used $x + dx$ and $y + dy$ instead, and he would have found that

$$\begin{align*} dz =& (2x - 4y) dx + dx^2 - 4dx\,dy \\ & + (-4x + 3y^2)dy+ (3y + dy) dy^2. \end{align*}$$

By Postulate I, the second, third, and fifth terms on the right side may be considered as 0. Meanwhile, we recognize the coefficients of $dx$ and $dy$ to be the partial derivatives. This justifies the modern definition of the total differential, equation 1, but it turns the process on its head: The differential calculus provides the expression for $dz$ and the coefficients therein can be used to define partial derivatives.

In the 1740s, Jean d’Alembert (1717–1783) and Leonhard Euler (1707–1783) considered the one-dimensional wave equation. In modern terms, if $y(x,t)$ expresses the ordinate corresponding to $x$ of a point on the string at time $t$, then

$$\begin{equation} \frac{\partial ^2 y}{\partial x^2} = \frac{1}{c^2} \frac{\partial ^2 y}{\partial t^2} \cssId{wave_equation}{\tag{4}} \end{equation}$$

for some constant $c$. How is this expressed in the differential calculus? Following Euler, we let $dy = p\,dx+q\,dt$ be the total differential of $y$. With this notation established, we consider the total differentials of $p$ and $q$:

$$\begin{align*} dp &= r \, dx + s \, dt \quad {\text{and}}\\ dq &= s \, dx + u \, dt, \end{align*}$$

by the equality of mixed partials or, as Euler expressed it Eul24, 139:

Because it is known by the nature of differentials, that the element $dt$ in $dp$ and the element $dx$ in $dq$, must have a common coefficient.

We note that both Euler and d’Alembert used the equality of mixed partial derivatives in solving the wave equation. The term “partial derivative” had not yet been coined, so Euler referred to it as a differential coefficient. Although this is called Clairaut’s theorem today in all the textbooks, priority for the result is actually due to Euler, and d’Alembert acknowledged this in his solution. (Having reached this point, the PDE that we denote by 4 is derived by showing that $r = ku$ for some positive constant $k$.)

4. Functions and Limits

It has often been observed that the calculus of derivatives depends almost entirely on two important concepts, the function and the limit. As we have seen, these concepts don’t appear anywhere in the differential calculus nor in Newton’s calculus of fluxions.

Leibniz’s differential calculus was a calculus of algebraic equations. As described in l’Hôpital’s Analyse, it could only be used for the analysis of transcendental equations or curves with additional tools—usually arguments involving similar triangles with infinitesimal sides. However, at about the same time that l’Hôpital published his textbook, Johann Bernoulli was able to assert that

$$\begin{equation} d\ln x = \frac{1}{x} \, dx \cssId{difflog}{\tag{5}} \end{equation}$$

and used his “logarithmic calculus” to integrate rational functions by partial fractions. Newton was able to do the equivalent thing even earlier. The identity 5 was not the result of analyzing the logarithmic curve, but it came rather from the inverse relation between integration and differentiation. In 1647, Gregory of St. Vincent (1587–1667) had demonstrated that the area under the hyperbola was given by the natural logarithm; see Kat09, 529–531.

How about trigonometric equations? Any real progress on this front had to wait until the middle of the 18th century, when Euler gave the modern definition of trigonometric quantities as functions of arcs on a unit circle.

4.1. A brief history of the function concept

Leibniz was the first to introduce the term “function” to mathematics in a letter of 1673 and he used it to describe a geometric property of a curve, such as the relationship between a point on the curve and its subtangent. From 1694 to 1716, Leibniz and Johann Bernoulli corresponded about mathematical problems and adopted the term function. In 1718, Johann Bernoulli, in an article on the Isoperimetric Problem defined it as follows:

I call a function of a variable magnitude a quantity composed in any matter whatsoever from this variable magnitude and from constants.

For the next 170 years, a variety of mathematicians, including Euler, Lacroix (1765–1843), and Fourier (1768–1830), would add their own contributions to the evolving concept of a function. The goal was to define a function in a way that was completely arbitrary and free of geometrical considerations or reference to a particular formula.

Ultimately, in 1888, Dedekind (1831–1916) published Was sind und was sollen die Zahlen? in which he gave us the formal definition of a function in modern form:

A function $\phi$ on a set $S$ is a law according to which to every determinate element $s$ of $S$ there belongs a determinate thing which is called the transform of $s$ and denoted by $\phi (s)$.

Readers interested in a more in-depth historical exposition on the function concept can refer to Thi05.

4.2. Functions in Euler’s Introductio

Figure 5.

Leonhard Euler (1707–1783).

Euler’s Introductio in analysin infinitorum (1748) was one of the most influential mathematical textbooks of all time. The title means “Introduction to the analysis of infinities” although it is almost always translated into English as Introduction to the Analysis of the Infinite. It is sometimes called Euler’s precalculus textbook, because he meant for it to bridge the gap between elementary algebra and the study of the differential and integral calculus. It is available in English translation Eul88. On page 2, Euler gave his definition of a function:

A function of a variable quantity is an analytic expression composed in any way whatsoever of the variable quantity and numbers or constant quantities.

The definition is quite similar to Bernoulli’s and very different from Dedekind’s. By modern standards the reliance on an “analytic expression” makes it much less general than a completely arbitrary mapping between sets. Later, in Chapter 5, Euler extended the definition to a function of two or more variables.

To modern readers, one of the most remarkable things about the Introductio is that Euler derived power series expansions for functions of one variable without the explicit use of the differential calculus. With rational functions, for example, this can be done by means of the geometric series. Similarly, functions involving rational exponents can be expanded by using the generalized binomial theorem. But for the transcendental functions, some additional trickery is needed, usually in the form of infinitesimals. Here is Euler’s method for deriving the series for the exponential function Eul88, 92–94.

Suppose $a > 0$. Because $a^0=1$, Euler reasoned that for an infinitely small quantity $\omega$, $a^{\omega }$ must be infinitely close to 1. Therefore, there is another infinitesimal $\psi$ such that $a^{\omega } = 1 + \psi$. Furthermore, because both $\omega$ and $\psi$ are infinitely small, there must be a quantity $k$ such that $\psi = k \omega$. Thus,

$$\begin{equation} a^{\omega } = 1 + k\omega . \cssId{atothetiny}{\tag{6}} \end{equation}$$

It also follows that $w = \log _a(1 + k\omega )$.

Next, Euler considered $a^{j \omega } = (1 + k\omega )^j$. By the binomial theorem, we have

$$\begin{align} a^{j \omega } = 1 &+ \frac{j}{1} k\omega + \frac{j(j-1)}{1\cdot 2} (k\omega )^2 \\ &+ \frac{j(j-1)(j-2)}{1 \cdot 2 \cdot 3} (k\omega )^3 + \cdots . \cssId{Euler_magic}{\tag{7}} \end{align}$$

Now take $j = \frac{z}{\omega }$, where $z$ is any (finite) real number. Because $\omega$ is infinitely small, it follows that $j$ must be infinitely large. Now because $\omega = \frac{z}{j}$, equation 7 becomes

$$\begin{align*} a^{z} = 1 &+ \frac{1}{1} kz + \frac{1(j-1)}{1\cdot 2j} k^2z^2 \\ &+ \frac{1(j-1)(j-2)}{1 \cdot 2j \cdot 3j} k^3z^3 + \cdots . \end{align*}$$

Euler asserted that for every natural number $n$,

$$\begin{equation} \frac{j-n}{j} = 1, \cssId{inflarge}{\tag{8}} \end{equation}$$

because $j$ is infinitely large. Hence, we have

$$\begin{equation*} a^z = 1 + \frac{kz}{1} + \frac{k^2z^2}{1 \cdot 2} + \frac{k^3z^3}{1 \cdot 2 \cdot 3} + \cdots . \end{equation*}$$

With a little reflection, modern readers will conclude that $k = \ln a$, and so

$$\begin{equation*} e^x = 1 + \frac{x}{1} + \frac{x^2}{1 \cdot 2} + \frac{x^3}{1 \cdot 2 \cdot 3} + \cdots . \end{equation*}$$

We note in passing that, as with l’Hôpital, we are remaining true to the notation that Euler used. He did not make use of the factorial symbol, nor the symbol $i$ for the imaginary unit, nor did he reserve the letter $z$ for complex numbers.

For more details on Euler’s treatment of exponential, logarithmic, and trigonometric functions, including the derivation of Euler’s famous identity

$$\begin{equation*} e^{\sqrt {-1}\theta } = \cos \theta + \sqrt {-1} \sin \theta , \end{equation*}$$

see Eul88, 75–115.

The arguments given above are breathtaking. They are far from rigorous by modern standards, but most of the details can easily be justified by translating them into the language of limits. Equation 8, for example, is essentially equivalent to

$$\begin{equation*} \lim _{ j \rightarrow \infty } \frac{j-n}{j} = 1 \end{equation*}$$

for a fixed natural number $n$. Likewise, the existence of the (finite) real number $k$ in 6 may be justified by considering

$$\begin{equation*} \lim _{h \rightarrow 0} \frac{a^h - 1}{h} = k. \end{equation*}$$

It comes as no surprise, then, that the last big step between Euler’s differential calculus and the calculus of derivatives came with the introduction of the limit concept.

4.3. A calculus of limits

The new mathematics of infinitesimals (or of fluxions in Great Britain) supercharged the study of classical physics at the turn of the 18th century. It was a tool that was well suited to the discovery of the differential equations of moving bodies, for example. In the 1730s, Euler systematized the field of differential equations. At the same time, though, many were concerned with the lack of rigorous foundations for the new analysis.

Michel Rolle (1652–1719), for example, criticized the new calculus in his role as a member of the Academy of Sciences in Paris. It’s widely reported in the secondary literature that he dismissed the differential calculus as “a collection of ingenious fallacies,” although it’s not clear that he ever said precisely that. Nevertheless, Itard wrote that in 1699 Rolle “vigorously attacked infinitesimal analysis and strove to demonstrate that it was not based on solid reasoning and led to errors” Ita70.

Of greater significance was the criticism of the philosopher George Berkeley (1685–1753), who wrote the polemical essay “The Analyst” in 1734. The tract was a critique of both Newtonian fluxions and Continental infinitesimals. He argued that mathematicians on both sides of the English Channel were in error because they manipulated increments that they took to be nonzero and then ultimately set them equal to zero. This is made explicit in l’Hôpital’s Analyse, where Postulate I allows for higher-order infinitesimals to be neglected. Berkeley described how the British mathematicians argued in similar ways and he maintained that “the minutest errors are not to be neglected in mathematics” Kat09, 628–629.

Newton was very much aware of the foundational difficulties associated with his method of fluxions. Therefore, when he published his monumental Philosophiae Naturalis Principia Mathematica New87, in which he stated his three laws of motion and derived the laws of planetary orbits, he gave all of his arguments in the language of classical geometry, which was the standard for rigorous mathematical reasoning in the 17th and 18th centuries. In doing so, he introduced what he called his “method of first and ultimate ratios” to abbreviate the sort of long-winded double reductio ad absurdum arguments which had been used by Archimedes and other classical mathematicians Kat09, 556–560.

In an attempt to address Berkeley’s objections, Colin Maclaurin (1698–1746) wrote A Treatise of Fluxions, a two-volume work published in 1742 Mac42. Maclaurin gave a systematic exposition of Newton’s method of fluxions, a collection of applications, and foundational arguments to address the concerns raised by Berkeley and others. The second volume began with a chapter “On the method of infinitesimals, of the limits of ratios, and of the general theorems that are derived from this doctrine.” After having given what he considered to be rigorous foundations for the method of fluxions in the first volume, he addressed here both “the harmony betwixt the method of fluxions and of infinitesimals” and “Sir Isaac Newton’s method by the limit of ratios.” This is where limits—introduced informally and didactically—first make their appearance in the evolution of the calculus of derivatives. Maclaurin makes no pretense to priority for the method of limits. Rather, he considers the method to be implicit in Newton’s Principia Mac42, 495–509.

We have already encountered d’Alembert as a pioneer of partial differential equations. He was also an early proponent of limits. François-Joseph Servois (1768–1847) wrote that “dAlembert distinguished himself among the geometers who applied the method of limits to the differential calculus” Ser14b, 144. In addition to being a mathematician and a member of the Paris Academy of Sciences, d’Alembert was also a man of letters. Along with Denis Diderot (1713–1784), he edited the groundbreaking Encyclopédie and authored most of its articles related to mathematics. In the entry “Limit,” d’Alembert wrote that “The theory of limits is the basis of the true metaphysics of differential calculus.” He gave the geometrical example of a circle being the limit of inscribed or circumscribed polygons, but he also referred the reader to his article “Differential.” There he stated that Newton “never considered the differential calculus as the study of infinitely small quantities, but as … the method of finding limits of ratios” Cal95, 482–485. He considered the ordinary parabola and found the secant line given at two points, a finite distance apart. Although he did not use the language of functions and derivatives, his argument is essentially equivalent to considering the difference quotient $\frac{\Delta y}{\Delta x}$ and then taking the limit where $\Delta x$ is positive and decreasing to 0.

4.4. Cauchy and limits

It is often said that Augustin-Louis Cauchy (1789–1857) introduced the limit concept into the foundations of the differential calculus. This is a vast oversimplification of the situation. Indeed, we can see that the idea of justifying calculus with limits goes back to d’Alembert and Maclaurin—arguably even as far back as Newton. The limit notion caught traction with d’Alembert’s support. Silvestre-François Lacroix (1765–1843), for example, wrote an elementary calculus textbook in 1802 that described the calculus as being based on limits of difference quotients. Conversely, Cauchy’s foundations of calculus are still quite deficient by modern standards. However, Cauchy’s textbooks, written for the students in the École Polytechnique, represented a pivotal step in the development of a rigorous calculus based on the method of limits.

In his Cours d’analyse de l’Ecole Royale Polytechnique BS09, Cauchy defined limits didactically, as follows:

When the values successively attributed to a particular variable approach a fixed value in such a way as to end up by differing from it by as little as we wish, this fixed value is called the limit of all the other values.

We immediately recognize this informal definition; it’s the sort of thing we might say today in teaching first-semester calculus. There are no epsilons or deltas in the definition, although Cauchy did use epsilons and deltas in his proofs included in subsequent textbooks. Cauchy also finessed the distinction between a sequential limit and a functional limit. The one definition is intended to cover both cases.

Figure 6.

Augustin Louis Cauchy (1789–1857).

Having defined the limit, Cauchy now turned his attention to infinitesimals.

When the successive numerical values of such a variable decrease indefinitely, in such a way as to fall below any given number, this variable becomes what we call infinitesimal, or an infinitely small quantity. A variable of this kind has zero as its limit.

Cauchy’s term “numerical value” coincides precisely with the modern term “absolute value.”

Cauchy’s Cours d’Analyse was an important milestone in the history of analysis, but it was never actually used as a textbook at the École Polytechnique. Two years later in 1823 Cauchy published Résumé des Leo̧ns … sur le calcul infinitésimal, or “Summary of the Lessons Given at the École Royale Polytechnique on the Infinitesimal Calculus” Cau12. This was a textbook for a complete year of study of the differential and integral calculus. Whatever it might be lacking by modern standards, it was a reasonably rigorous and very lucid development of a freshman calculus course based on limits. It consists of 40 lectures, or chapters, with the first 20 covering the calculus of derivatives and the second half covering the integral calculus, beginning with the definition of the definite integral.

In Lecture 1, Cauchy copied his definitions of “limit” and “infinitesimal” word for word from the Cours d’analyse. In Lecture 2, he discussed finite differences and then considered the case where $\Delta x = i$, an infinitesimal. Then in Lecture 3 he defined the derivative:

…if we then set $\Delta x = i$ [an infinitesimal], the two terms of the ratio of differences

$$\begin{equation*} \frac{\Delta y}{\Delta x} = \frac{f(x+i)-f(x)}{i} \end{equation*}$$

will be infinitely small quantities. But while these two terms indefinitely and simultaneously will approach the limit of zero, the ratio itself may be able to converge toward another limit. …the limit of the ratio $\frac{\Delta y}{\Delta x} = \frac{f(x+i)-f(x)}{i}$ will depend on the form of the proposed function $y = f(x)$. To indicate this dependence we give the new function the name derived function and we represent it by the notation $y'$ or $f'(x)$.

We note that both the name “derived function”—which is rendered in English as “derivative”—and the notation $f'(x)$ are due to Joseph-Louis Lagrange (1736–1813).

5. Lagrangian Formalism

At the dawn of the 19th century, many mathematicians were working on the foundational problem and there was no general consensus as to which scheme might eventually lead to a satisfactory account of the calculus. Differentials were widely used, limits were gaining popularity, but a solid foundation using limits was still decades away. In Great Britain, fluxions still held sway—although Maclaurin spoke of the harmony between fluxions and differentials, the conceptual foundation for fluxions was quite different from that of Leibniz’s differentials. On top of all that, there was a fourth path that had recently been blazed.

5.1. Another competing foundation for the calculus

This fourth foundational scheme—power series expansions—was championed by Lagrange. He believed he could use algebraic analysis to derive a power series expansion of any function, without recourse to derivatives, limits, or differentials. He was not trying to discover new results, but he was seeking an algebraic foundation for all previous results in calculus. His attempt to accomplish this came with the publication of the Théorie des Fonctions Analytiques in 1797 Lag97. Lagrange began as follows Lag97, 5:

In a memoir printed among those of the Academy of Berlin for 1772, I proposed that the theory of the expansion of functions into series contained the true principles of the differential calculus, freed from any consideration of the infinitely small, or of limits, and I proved the theorem of Taylor using this theory, which we may regard as the fundamental principle of this calculus and which had previously never been proven except with the assistance of this same calculus, or by consideration of infinitely small differences.

Lagrange stated that a function of $x$, which he called $f(x)$, is an analytic expression of $x$. To find $f(x + h)$, he formed an algebraic series,

$$\begin{equation*} f(x + h) = f(x) + ah + bh^2 + ch^3 + \cdots , \end{equation*}$$

where $a, b, c, \ldots$ are new functions of $x$, which are related to $f(x)$ and are independent of $h$. The first coefficient in the power series, $a$, was determined by the first derivative of $f(x)$, denoted by $f'(x)$.

Through an impressive manipulation of power series, Lagrange was able to show that the formal expansion of a function $f(x+h)$ is

$$\begin{align*} f(x) + f'(x)h &+ \frac{f''(x)}{2!}h^2 + \cdots + \frac{f^n (x)}{n!}h^n \\ &+ \frac{f^{(n+1)}(x+i)}{(n+1)!}h^{n+1}, \end{align*}$$

where $i$ is between 0 and $h$. Lagrange considered the term

$$\begin{equation*} \frac{f^{(n+1)}(x+i)}{(n+1)!}h^{n+1}, \end{equation*}$$

the remainder term, or error term, for a Taylor series. He believed that, with the remainder term included in a power series, all functions could be expanded, since all the error was accounted for. For additional details of Lagrange’s derivation, see Kat09, 633–636.

Two observations should be made about Lagrange and his work in algebraic analysis. First, Lagrange, like Newton and Taylor, did not consider the issue of convergence when expanding functions into series. When Lagrange made the claim that $h$ can be taken as small as one desires, he was essentially stating that $h$ is approaching zero in a difference quotient, which at that point was the missing component of the modern definition of the derivative. Finally, Lagrange had set the stage for other mathematicians to see the necessity for rigor in calculus. “By now teaching and publishing were complementary for Lagrange. He further explicated and developed that theme [rigor] in his courses at the École Polytechnique from 1795 through 1798, which he then published as Nouvelles Leçons sur le calcul des fonctions” Gil85, 508.

5.2. François-Joseph Servois, a disciple of Lagrange

Servois pursued several careers during his life: He was a priest, artillery officer, professor of mathematics, and curator of the Artillery Museum, located in the 7th Arrondissement of Paris. Readers interested in a more in-depth biography of Servois can refer to Pet17. In a list of mathematicians, Servois would stand to be lesser known. To the extent that his name is known at all, it is for introducing the words distributive, commutative, and pole into mathematics. However, Servois made significant contributions to mathematics, in particular to the field of linear operator theory.

The intellectual climate brought on by the French Revolution fostered a firm rejection of the categorical thinking from the enlightenment in favor of more formal ideas of rigor Gil85. Inspired by this, Servois believed that the only way to make the foundation of the differential calculus truly rigorous was to base it on the method of series that he had devised, following Lagrange. Servois stated that the following questions could not be answered rigorously using the method of infinitesimals:

Furthermore, he warned Ser14b, 148:

In a word, I am convinced that the infinitesimal method does not nor cannot have a theory, that in practice it is a dangerous instrument in the hands of beginners, that it necessarily imprints a long-lasting character of awkwardness and pusillanimity upon their work in the course of applications. Finally, anticipating, for my own part, the judgment of posterity, I dare to predict that this matter will one day be accused of having slowed the progress of the mathematical sciences, and with good reason.

5.3. Servois’s Essai

Servois published his mathematical results in “Essai sur un nouveau mode d’exposition des principes du calcul différentiel” (Essay on a New Method of Exposition of the Principles of Differential Calculus) Ser14a and his philosophical reflections on the foundations for calculus in “Réflexions sur les divers systèmes d’exposition des principes du calcul différentiel, et, en particulier, sur la doctrine des infiniment petits” (Reflections on the Various Systems of Exposition of the Principles of the Differential Calculus and, in particular, on the Doctrine of the Infinitely Small) Ser14b.

In the Essai, he proposed using power series expansion as fundamental to the differential calculus. He defined the differential operator as an infinite series, which he called an infinitinôme, in powers of the difference operator $\Delta$, given by $\Delta f(x) = f(x + h) - f(x)$:

$$\begin{equation*} dz = \Delta z - \frac{1}{2} \Delta ^2 z + \frac{1}{3} \Delta ^3 z - \frac{1}{4} \Delta ^4 z + \cdots . \end{equation*}$$

Using this operator, he demonstrated how to find differentials of functions and was able to discover the basic laws of the differential calculus, such as the power and product rules. Additionally, he derived the differentials of the “elementary simple functions,” such as $\ln (x)$ and $\sin (x)$. The latter required a unique function that Servois defined, which was a function of complex numbers. The majority of his results are derived using his notions of distributivity and commutativity. Using modern notation, he defined a distributive function to be one satisfying $F(x + y + z + \cdots ) = F(x) + F(y) + F(z) + \cdots$. Furthermore, he considered two functions $F$ and $G$ to be commutative if $F(G(x)) = G(F(x))$. If we take $G$ to be a constant function, then a modern reader would recognize that a distributive function that commutes with $G$ is a linear operator. Servois never used the term “operator,” but rather, he used the name function in both the ordinary way, and as a “function applied to functions.”

The commissioners of the Institut de France praised Servois for having “done something that is very useful for the science [of analysis]” Ser14a, 140. However, the Institut, whose name reverted to the Académie des Sciences following the reign of Napoleon, did not publish his two memoirs on the foundations of calculus. Instead, they appeared in Joseph Diez Gergonne’s (1771–1859) Annales de mathémathiques pures et appliqués, which was the first academic journal devoted entirely to mathematics. This gave his foundational ideas a broad audience. In particular, Servois was noticed by several British mathematicians who in the 1820s were interested in reforming the calculus as it was taught and practiced in Great Britain. Like Servois, they initially backed Lagrange’s formalism as the proper route to rigorous foundation for calculus. Perhaps the greatest impact of this British analytic school was its influence on the study of abstract algebraic structures later in the 19th century. These abstract stuctures were defined in terms of properties such as distributivity and commutativity. For more on Servois’ influence, see DLP12.

6. Nonstandard Analysis

A foundation for a calculus of infinitesimals proved elusive in the 19th century. Many mathematicians and physicists continued to use differentials profitably for decades following the publication of Cauchy’s textbooks, but a calculus of derivatives based on limits gradually found ascendancy, thanks to the work of Georg Bernhard Riemann (1826–1866), Karl Weierstrass (1815–1897), Richard Dedekind (1831–1916), and others Kat09, 765–789.

Before this time, mathematicians treated real numbers as though they were given a priori. Once the reals were axiomatized as a complete ordered field, one could show that there were no differentials in the field of real numbers. But this doesn’t mean that they did not exist in any realm.

The problem with differentials is that they have a dual nature—Johann Bernoulli’s Postulate I (see Section 2.2) seems to say that a differential is zero. And yet we can calculate ratios with differentials, such as the subtangent. Because of the latter point, most 18th-century mathematicians would have said that differentials are nonzero. Euler, on the other hand, said that differentials are, in fact, zero. However, arguing by analogy with his notion of infinities, he said there were different orders of zeros and some could have finite ratios with others Eul00, 47–61. Perhaps 20th-century mathematics could come up with a resolution.

Figure 7.

Abraham Robinson (1918–1974).

Indeed, in 1961, Abraham Robinson (1918–1974) used model theory to create a rigorous foundation for a calculus based on infinitesimals in an article entitled “Non-Standard Analysis,” Rob61 which is now the accepted term used for this branch of mathematics. Robinson’s work is grounded in set theory and abstract algebra.

To give a rigorous definition of an infinitesimal, Robinson needed to extend the field of real numbers to what he called the hyper-real numbers, which he denoted as $\mathbf{R}^*$. Intuitively, $\mathbf{R}^*$ contains the ordinary reals, as well as infinite hyper-reals, and infinitesimals. The construction of the hyper-reals is similar to the procedure for constructing the reals from the rational numbers using equivalence classes of Cauchy sequences.

We begin with $\mathbf{R}$, the set of real numbers, and define two binary operations—“addition” and “multiplication”—elementwise on the set of all sequences of real numbers. From there, we use the concepts of a filter and ultra-filter.

Let $I$ be a nonempty set. A filter on I is a nonempty collection $\mathfrak{U}$ of subsets of $I$ having the following properties:

Furthermore, $\mathfrak{U}$ is an ultra-filter if, for any subset $A$ of $I$, either $A \in \mathfrak{U}$ or its complement $A' \in \mathfrak{U}$ (and not both, by conditions 1 and 2). Using the Axiom of Choice, in the form of Zorn’s Lemma, we can show that if there is a filter, then there must exist an ultra-filter. This gives us the ability to form an equivalence relation on sequences of real numbers, where the equivalence classes are the hyper-real numbers HL85, 2–4.

With the hyper-reals so constructed, we define addition and multiplication on the equivalence classes by adding and multiplying representative sequences elementwise. Finally, we define an order relation as follows: $\mathbf{r} < \mathbf{s}$ if and only if, for representative sequences $r$ and $s$, we have $\{i \in \mathbf{N}: r_i < s_i\} \in \mathfrak{U}$. With these definitions, the structure $\mathbf{R}^*$ is a linearly ordered field. The mapping from $\mathbf{R}$ into $\mathbf{R}^*$ given by $*r = [\left<r,r,r,\ldots \right>]$ is an order-preserving injective homomorphism from $\mathbf{R}$ into $\mathbf{R}^*$ HL85, 5.

If $x=\left<x_1, x_2, x_3, \ldots \right>$ is a representative of $\mathbf{x} \in \mathbf{R}^*$, then $|\mathbf{x}|$ is the equivalence class containing the sequence $\left<|x_1|, |x_2|, |x_3|, \ldots \right>$. We are now in a position to define an infinitesimal. An element $\mathbf{x} \in \mathbf{R}^*$ is:

Two elements $\mathbf{x}$ and $\mathbf{y} \in \mathbf{R}^*$ are said to be infinitely close, $\mathbf{x} \approx \mathbf{y}$, if $\mathbf{x}-\mathbf{y}$ is infinitesimal. Thus, $\mathbf{x}$ is infinitesimal if and only if $\mathbf{x} \approx *0$.

Nonstandard analysis was a popular topic of study in the 1960s. This was an exciting period in the development of set theory. For example, Cohen (1934–2007) proved the independence of the Continuum Hypothesis in 1963 Coh63. Martin Davis suggests there was more to the popularity of nonstandard analysis than just an interest in set theory Dav77:

It is a great historical irony that the very methods of mathematical logic that developed (at least in part) out of the drive toward absolute rigor in analysis have provided what is necessary to justify the once disreputable method of infinitesimals. Perhaps indeed, enthusiasm for the nonstandard methods is not unrelated to the well-known pleasures of the illicit. But far more, this enthusiasm is the result of the mathematical simplicity, elegance, and beauty of these methods and their far reaching application.

Some authors even tried to reform the teaching to calculus by reintroducing the study of infinitesimals. However, nonstandard analysis never took off as a popular alternative way of learning the theory of calculus—even though a number of papers have been written about its presumed pedagogical advantages—and the reason why is mainly an unanswered question. Dauben Dau88 makes the claim that this version of analysis failed to catch on because all the results that can be proven in $\mathbf{R}^*$ can be proven in $\mathbf{R}$. So why would one go through the process of constructing a new number system? Others claim that the $\epsilon$-$\delta$ method has all the intuitive aspects of nonstandard analysis without the need of additional formalism. But perhaps the real reason was that by the time the problem of infinitesimals had been solved, mathematics had long since moved on.

Illustration Credits

The portraits of l’Hôpital, Bernoulli, Euler, and Cauchy are bas relief sculptures. They were carved by Susan Petry, an artist, experimental psychologist, and professor emerita at Adelphi University. L’Hôpital and Euler are carved in cherry. Bernoulli and Cauchy are carved in tulip poplar. All portraits are approximately 9 by 12 inches. The photo of Robinson was taken in 1970 by an unknown photographer. It is in the public domain.

References

[BPS15][BS09][Cal95][Cau12][Coh63][Dau88][Dav77][DLP12][Eul00][Eul88][Eul24][Gil85][Hal80][HL85][Ita70][Kat09][Lag97][Lei84][Mac42][New87][Pet17][Rob61][Ser14a][Ser14b][Thi05]

Robert E. Bradley is a full professor of mathematics at Adelphi University. His email address is [email protected].

Salvatore J. Petrilli Jr. is a full professor of mathematics and associate dean at Adelphi University. His email address is [email protected].

Article DOI: 10.1090/noti3145

Credits

Figures 1, 3, 5, and 6 are courtesy of Susan Petry.

Figure 2, Figure 4, and photo of Robert E. Bradley are courtesy of Robert E. Bradley.

Figure 7 is courtesy of Konrad Jacobs (https://opc.mfo.de/detail?photo_id=3540, CC BY-SA 2.0 de, https://commons.wikimedia.org/w/index.php?curid=24920550).

Photo of Salvatore J. Petrilli Jr. is courtesy of Salvatore J. Petrilli Jr.