Lagrange and probability theory

Only two memoirs by Lagrange on the calculus of probability are known, “Mémoire sur l’utilité de la méthode de prendre le milieu entre les résultats de plusieurs observations” [10] published in 1776 and “Recherches sur les suites récurrentes dont les termes varient de plusieurs manières différents” [11], published in 1777. This might not appear to be much, but because both of these memoirs are truly exceptional, as are the majority of Lagrange’s works, what they contain is too rich for a treatment that is necessarily brief. Lagrange’s work forms a sum of parts. In order to be fully understood, each of the memoirs must be seen in the context of the entirety of his work. This is particularly truly of the second memoir [11], which is actually the development of a project conceived in 1759 regarding the history of differential calculus and the calculus of probability. We know from Pascal that the majority of the problems in game theory can be reduced to the study of the difference equations that describe the evolution of the fortune of a player during an indefinite sequence of games. The idea of the young Lagrange was to treat these equations as analogous to the theory of differential equations already well developed by Euler and d’Alembert. Lagrange intended to transport the results of the theory of the equations to infinitely small differences, which he had developed during the same period, to the theory of finite difference equations, or better, he wanted to develop two parallel theories. For example, in the first article on this topic [9], he proposed an original method of varying the constants in the case of non-homogenous linear differential equations, and transposed it to the case of finite differences. He would take this idea up again around 1775 (for example, see [[15], vol. XIV, pp. 62–63, 74] and apply it brilliantly in his memoirs on mechanics, of which the last dates to 1808–1809.^{Footnote 1}

The applications of probability announced in 1759 were not published until 1777. Lagrange presented a new and innovative approach that extended to the case of the equations of partial differences, introduced by Laplace [17], which are analogous to partial differential equations, one of the predominant topics in analysis in those years. Lagrange’s method was different from that of the generating functions of de Moivre and Laplace, and allowed him to indicate the first complete analytical solution to the problem of the duration of a game, considered the most difficult one of the classical theory of probability; even de Moivre had given a general expression of this one, without knowing precisely how he had obtained it. Lagrange’s 1777 memoir [11] must be considered not only in the context of the long, rich history of probability, but also in the even longer and richer history of differential and integral calculus. It would be too long to reprise it here, above all because there already exist important works in this area, including the recent paper by Maria Teresa Borgato [5].

Here, therefore, we will limit our discussion to the 1776 memoir [10], the first by Lagrange on the calculus of observations. We will see that this memoir is sufficiently rich to give us an idea of Lagrange’s incomparable virtuosity in analytical and algebraic calculation, as well as his unquestionable genius of invention and his hesitancy in thorny questions of statistics, in a moment of history in which everything, or almost, had yet to be invented.

The memoir [10] by Lagrange published in 1776 was presented at the Berlin Academy on 2 and 16 November 1769 and 21 August 1771, and the text bears the marks of this double presentation, given at a distance of 2 years, above all because the final one (presented in the Miscellanea Taurinensia) appears to have been revised yet again in the course of the year 1773. It is in effect possible that the 1776 memoir is one of two submitted to the director of the Miscellanea on 27 January 1774.

Whatever the case, the existence of Lagrange’s memoir seems to have been known before its publication. As Johann III Bernoulli wrote in his Recueil pour les astronomes:

Le Problème de prendre entre un certain nombre d’observations le vrai milieu, qui est rarement le milieu arithmétique, intéresse beaucoup les Astronomes; il est à souhaiter qu’on leur présente l’esprit rapproché des différentes méthodes données pour cet effet par le P. Boscovich, par M. Lambert dans l’ouvrage cité page 157 de mon premier volume par M. Daniel Bernoulli, dans un mémoire qui n’est pas encore imprimé, enfin par M. de la Grange dans une belle Théorie qui a fait dernièrement le sujet de quelques lectures dans nos assemblées académiques [ [3]: vol. 2, p. 249, footnote].^{Footnote 2}

(The problem of choosing the true mean between a certain number of observations, which rarely coincides with the arithmetic means, is of great interest to the astronomers; it is to be hoped that their current thinking is close to the different methods given for this purpose by Father Boscovich, by Mr. Lambert in the work cited page 157 of my first volume, by Daniel Bernoulli, in a memoir that is not yet printed, and finally by M. de la Grange in a beautiful theory which has recently been the subject of some lectures in our academic meetings).

We can also see the interesting paper “Milieu…” by Johann III Bernoulli [4], in which he describes the first version of the memoir by Daniel Bernoulli [2], saying that he had received a copy of it in 1769, and the “discrete” part of the memoir [10] by Lagrange, which showed, in any case, the interest of the European academies in the search for a mean derived from different observations, a problem that had also inspired the early research of Laplace on the calculus of probability [18, 22].

We will divide our present discussion into two parts: the first concerning probability; the second relating to statistics.

As the title aptly indicates, the 1776 memoir [10] asserts, with the aid of the calculus of probability, the interesting truth that, in taking the average of several measurements error is inevitable, but that one can hope to “compensate” for them. In so doing, is the accuracy of the measurements taken separately improved, and if so, how?

The problem was stated very clearly by Thomas Simpson in “A Letter to the Right Honorable George Earl” [24], published in 1755 in the Transactions of the Royal Society. Lagrange never cites Simpson, but it is difficult to believe that he had no knowledge of the memoir. He knew and appreciated Simpson’s mathematical work, which he had sometimes discussed with d’Alembert.^{Footnote 3} There is therefore no reason why he would not have read this discussion, especially because it had been republished in 1757 in the Miscellanea Taurinensia, which was often cited by the scholars of the day. Simpson was the first, or one of the first, to propose the use of the calculus of probability in the theory of observations, notably in physical astronomy, where each measurement is subject to errors and is only partially reproducible. As Lagrange wrote in the introduction to his memoir, in the probabilistic theory of observations, one presumes that “les erreurs qui peuvent se glisser dans chaque observation sont données et qu’on connoisse aussi le nombre de cas qui peuvent donner ces erreurs, c’est-à-dire la facilité de chaque erreur” (the errors that can insinuate themselves in every observations are given, and thus we know the number of cases that can give these errors, that is, the probability of each error). Lagrange quite frequently used the noun facilité (facility) which generally designated the number of cases or the probability of a constant taken in the discrete case, which is today called the density in the continuous case.

One thus takes as known “les limites entre lesquelles toutes les erreurs possibles doivent être renfermées avec la loi de leur facilité” (the limits within which all the possible errors must be contained according to the law of their facility). He goes on to state, “Je chercherai dans l’une et dans l’autre de ces hypothèses, quelle est la probabilité que l’erreur du résultat moyen soit nulle ou égale à une quantité donnée” (I will seek in both of these hypotheses, what the probability is that the error of the resulting mean is zero or equal to a given quantity).

For example, Problem I, no. 1 presumes that in every observation the errors can only be 0, +1 or –1, and that there are a cases for 0, and b cases for both +1 and –1; what is the probability of having an exact result in taking the mean of the individual results of a number n of observations?

This problem, Lagrange tells is, reduces to the following. One throws n times a die that has a + 2b faces, of which a are marked 0, b are marked 1 and b are marked –1. “Trouver la probabilité qu’il y a d’amener zéro” (Find the probability of coming up with zero), that is, that the sum of the faces is zero. This problem is a classic from the early eighteenth century, when it was treated notably by Montmort, Nicolas I Bernoulli and de Moivre. This is precisely the problem, sometimes called the problem of points, that generalises that of a die of f faces, each of which is marked by a whole number e ₁, e ₂, …, e _f , thrown n times, which de Moivre had treated in the first part of his theory of generating functions (the second part is constituted of recurring series, which we will not discuss here). To a die made in this way is associated the to polynomial $x^{e_1}+x^{e_2}+\dots +x^{e_f}$. The number of ways to obtain a sum s in n throws is equal to the coefficient of the power x ^s in the polynomial expansion:

The method of de Moivre [6] of 1756 (which was published several times, in 1730, 1738 and 1756), became classic in the second half of the eighteenth century. Lagrange naturally knew it and used it in his memoir, particularly in Problem I, where he resorted to it to calculate the constant term of the power

This is not very difficult, for example, Lagrange proposed writing a + b(x + x ⁻¹) in the form

with α and β such that α ² + β ² = a and αβ = b.

One raises the product (p) to the power n: the constant sought is equal to the sum of the coefficients of the binomial development (α + βx)ⁿ.

For example, presuming a = 2b, and $\alpha =\beta =\sqrt{b}$, the constant sought equals:

Lagrange observes a little further on, in no. 6, that the sum of the square of the coefficients of the binomial (1 + 1)ⁿ has a simple form, $\sum _{k=0}^n(C_n^k{)}^2=C_{2n}^n,$ a combinatorial formula that results, for example, from the fact that$\sum _{k=1}^n(C_n^k{)}^2=\sum _{k=1}^n{C}_n^k{C}_n^{n-k}$ and that the second member is a way to count the combinations of 2n objects of class n. Laplace also immediately proved this formula anew in a letter to Lagrange on 11 August 1780 [[15], vol. XIV, pp. 95–96].

Lagrange was obviously happy to range about this topic, and to detour from one formula to generate one or two algebraic or combinatorial nuggets. Let us recall one that is particularly elegant.

At No. 5, Remark I, Lagrange sets out to find the laws that govern the facilities and the way to calculate them when n varies, in the general case where a and b are arbitrary.

Lagrange likewise treated the case where the errors assume the three values 0, –1 and r, where r is a positive integer, and although we will not go into this for the sake of brevity, we need not doubt its ingenuity. Instead we will look at the case where the facilities of the errors are unknowns, and must be determined starting from the observations. Given its importance, we will undertake this study in the next section; here let us briefly look at the final part of Lagrange’s memoir, which concerns the facility of the sum or mean of n errors likely to assume arbitrary values. This study is obviously based on the results of Simpson’s, mentioned earlier, but with the addition of the elements that are quite fundamental and would make a remarkable impression on the young–Laplace, who in his turn would take them up in a very important memoir [22], a Laplacian response to Lagrange.

The first problem dealt with by Lagrange is the classic problem of points. One takes a die of f regular faces, each of which has the same possibility of coming up. This is thrown n times, and determines the law of the sum of the points obtained in the different throws. The problem dates back at least as far as Galileo, and is one that all scientists of the seventeenth and eighteenth centuries grappled with to greater or lesser effect. It is actually one of the most sophisticated problems of the classic calculus of probability. It was on the occasion of this problem that Nicolaus Bernoulli and Montmort articulated the sieve formula (or the principle of inclusion–exclusion), one of the founding theories of combinatorics.

As we said, the general problem of points was completely resolved by the geometers at the beginning of the 1700 s, and was then taken up again by Simpson, who adapted it to the early probabilistic theory of observations. In the final part of his memoir, Lagrange begins by finding anew the results of de Moivre and Simpson by using the same method, that of generating functions, but his exceptional algebraic power enabled him to go further and faster. We will say nothing here about what Lagrange does with particularly elegance thanks to an algebraic lemma (no. 23), except that it led to the alternate formulas of Montmort-Nicolas Bernoulli-de Moivre-Simpson-Lagrange-Laplace, etc.

We have already remarked that on this occasion Lagrange introduced a theory of observations, that is to say, in stochastic calculus, the “Laplace transforms”, a notion already used, notably by Euler, in the theory of differential equations. Here, however, it finds an important field of application and new properties, as we will see.

With the aid of this notion, Lagrange set out to deal with Problem X (no. 40):

On suppose que chaque observation soit sujette à toutes les erreurs possibles comprises entre ces deux limites p et –q, et que la facilité de chaque erreur x, c’est-à-dire le nombre des cas où elle peut avoir lieu divisé par le nombre total des cas, soit représentée par une fonction quelconque de x désignée par y; on demande la probabilité que l’erreur moyenne de n observations soit comprise entre les limites r et - s.

(Suppose that each observation is subject to all the possible errors comprised within the two limits p and –q, and that the facility of each error x, that is to say the number of case where it can have divided by the total number of cases, is represented by an arbitrary function of x designated by y. What is the probability that the mean error of n observations is comprised between two values r and – s?).

Let y be the facility of x, its probability density. Lagrange associated it to the transform ∫ya ^x dx, which he never names, but which Laplace called in general terms his “generating function” [[21], Book I].

Maintenant pour avoir la probabilité que l’erreur moyenne de n observations soit z, il faudra considérer le polynome qui est représenter par l’intégrale de ya ^x dx, en supposant cette intégrale prise de manière qu’elle s’étende depuis x = p jusqu’à x = −q, l’on élèvera ce polynôme à la puissance n, et l’on cherchera le coefficient de puissance z de a, par les règles données dans les corollaires du lemme précédent (n° 33); ce coefficient, qui sera une fonction de z exprimera la probabilité que l’erreur moyenne soit z, comme il est facile de voir, d’après ce qui a été démontré plus haut.

(Now, to have the probability that the mean error of n observations is z, it is necessary to consider the polynomial that is represented by the integral of ya ^x dx, and to suppose that integral taken such that it extends from x = p to x = −q; we raise this polynomial to the power n, and seek the coefficient z of a, by the rules given in the corollary to the previous lemma (no. 33); that coefficient, which will be a function of z, will express the probability that the mean error is z, as is easily seen after what was shown above).

This quote, taken word for word from the text of no. 40, states a fundamental property of generating functions, the “Laplace transforms”, remarkably by Lagrange. It transforms convolutions into products. No proof is given by either Lagrange or by Laplace, who would use it in his turn in the case of the transforms called “Fourier”. It is “easy” to see, and that suffices. It is evidently tied to the method of de Moivre, which gives it the true dimension.Written thus

the coefficient Y(z) of the power a ^z is the facility in z of the sum of n observations of facility y. In all cases, it is about writing the function of a, f(a) = (∫ya ^x dx)ⁿ, in the form of a Laplace transform ∫Ya ^z dz.

This dazzlingly calculus of Lagrange made a huge impression on the young–Laplace, who, though calling Lagrange’s method ‘beautiful’, set himself at once to re-prove it by his method of convolution.^{Footnote 4}

Lagrange dealt with other cases. It would be too long to go into it here, especially as the alternate formulas obtained become intractable quite quickly when, for example, n exceeds 10, and the search for an equivalent when n is very large is impossible in this formulation.

The problem left open by Lagrange occupied Laplace for almost forty years. His solution was published in 1810, and its applications, notably the least-squares method of Legendre and Gauss, constituted the culmination of the “analytical theory of probability”. Laplace’s idea is nevertheless very simple; it suffices to place a = e ^it, that is to say, to replace the Lagrange (Laplace) transform of the facility y

with its Laplace (Fourier) transform

The inversion is made quite simply by analogy with the inversion of the Fourier series. So doing, Laplace surreptitiously reintroduces the passage from finite to infinitely small, banished by Lagrange, and his method would suffer from it for a long time, until Fourier’s theory of integrals became a fundamental chapter of the modern theory of functions (without the passage from finite to infinite, but with the generalised introduction of the concept of almost everywhere, which would have undoubtedly displeased Lagrange, but which completed Laplace). But this is another story.

Let us now go to the second part of Lagrange’s memoir, the only contribution of our subject to mathematical statistics, that is to say, the statistics governed by the calculus of probability.

We will quickly examine the purely statistical part of the 1776 memoir [10], Lagrange’s only known contribution to probabilistic statistics. This concerns from nos. 16 to 22 of the memoir in question.

Lagrange begins by established a preliminary result, problem V, no. 16, that we will give without proof, with the notations slightly modified.

Suppose that a measuring instrument is erroneous by an amount e _i , with a facility a _i , for i comprised between 1 and m. To simplify, let us suppose that the sum of the m facilities is equal to 1. One takes n measurements with this instrument. One knows that the facility of the sum of the errors committed in the course of measuring, equal to μ, is equal to the coefficient of the power μ in the polynomial development ${\left({\sum }_i{a}_i{x}^{e_i}\right)}^n$. Lagrange shows that the maximum value of this facility is obtained when the number of times α _i that the instrument errs by e _i in the course of n measurements, is the integer closest to na _i , for all values of i between 1 and m. From this it follows that “the mean error for which the probability will be the largest will be expressed by $\frac{m}{n}={\sum }_i{a}_i{e}_i$”, which is the mathematical expression of the law of error of the instrument in question. And thus, adds Lagrange, “cette quantité représentera la correction qu’il faudra faire au résultat moyen de plusieurs observations” (this quantity will represent the correction that must be made to the resulting mean of the several observations). The error that occurs, and which must be corrected, is that which has, a priori, the largest probability of occurring. Further on we will discuss the principle of maximum probability as it was known in Berlin from at least 1760.

Let us also recall that, if the probability of obtaining the error e _i is equal to a _i , the facility to be obtained in the course of n measurements α _i , times the error e _i , for i comprised between 1 and m, is given by the polynomial law:

This being said, Lagrange set out to solve the following problem, Problem VI, no. 19:

Je suppose qu’on ait vérifié un instrument quelconque, et qu’ayant réitéré plusieurs fois la même vérification on ait trouvé différentes erreurs, dont chacune se trouve répétée un certain nombre de fois; on demande quelle est l’erreur qu’il faudra prendre pour la correction de l’instrument.

(I suppose that an arbitrary instrument has been verified, and that the same verification has been repeated several times and has found different errors, each of which are repeated a certain number of times; which is the error which must be taken for the correction of the instrument?)

This time the e _i are the unknowns, and must be determined. This is what is now called the problem of parameter estimation of a polynomial law, after having observed n instances of that law. Once that estimation has been made, one deduces the required “correction” by the mean formula obtained in the preceding Problem V.

Inverting Bernoulli’s theorem, going back from the frequencies to the probabilities, was a problem that had been generally well-known to European scholars for a very long time. Johann Bernoulli appears to be the first to have stated it, but in spite of his efforts and those of his commentators—de Moivre in particular—by the first half of the eighteenth century still no satisfactory solution had been found. It is known that Bayes, in a justly famous memoir published posthumously [1] gave a quite remarkable solution, but it seems to have passed unnoticed by scholars on the continent. It does not appear in any case that Lagrange knew of it, even though he read English well enough to read British scientists such as Thomas Simpson, mentioned earlier.

Towards 1770, at the same time and in different places in Europe where science was being studied, different, very ingenious solutions to the same problem were found, most of which were not published, notably by Daniel Bernoulli in 1769, Condorcet in 1770, Lagrange between 1769 and 1773, Laplace in 1773, etc. The first published solutions appeared very quickly: Laplace [18], Lagrange [10] Daniel Bernoulli [2], Euler [7], and others.

Lagrange had to intervene in this terrain like the others, and distinguish himself. We shall see that he did so without excessive enthusiasm, but with that mathematical genius that distinguishes all his papers on any subject he dealt with.

Lagrange’s analysis is very elliptical. His statistical hypotheses, his methodologies, his principles are never explicated and can be interpreted in different ways. Lagrange details only the calculations, always superb. However, the very elegant result of Lagrange is hardly applied to the theoretical instruments that allow themselves to be reduced by the polynomial law, when no hypothesis is made on the facility of the errors. This is a non-parametric point of view a priori. The law of error is not known and is itself the unknown.

This is a delicate point that has been very well analysed by Stephen M. Stigler in his survey paper [25]. Lagrange estimated directly the unknowns a _i and maximised the probability of the observations made, expressed by the polynomial law, without formulating any hypothesis about the law of facility of the errors, which can be known by other means or postulated by hypothesis. The result is always the same. The “most probable” estimation of a _i is the observed frequencies A _i , such that the best “correction” is always the observed mean ∑ _i A _i e _i , which Lagrange, as we have seen, calculated the facility under various hypotheses using the de Moivre-Lagrange method, hypotheses made afterwards, arrived at here and thus after the battle.

On the other hand, Lambert [16], Daniel Bernoulli [2] and the statistics of today adopt the opposite approach, that is, formulating the hypotheses on the facility of the error before determining the parameters. The method of estimation by maximum likelihood leads then to very different results.

Did Lagrange know that his study was a little short? Had he discussed this with Lambert? In any case, he didn’t pursue this path, perhaps because he felt he had better to do elsewhere, because Laplace soon argued against the systematic use of averages in the theory of observations [22]. There are no known responses from Lagrange to Laplace’s first major statistical memoirs of 1774 and 1781, or to those late, very extraordinary ones of the years 1810–1811. Conversely, as we said, Laplace took the greatest account of the memoir by Lagrange, which he used to criticise and refine his own writing, in several important memoirs [19, 20] and in his Théorie analytique [21].

In any case, we can see that Lagrange gave here the first known proof of the normal approximation to a polynomial distribution, a result rediscovered notably by Karl Pearson, in his great paper of 1900 [23].

Although we might deplore the Lagrange’s abandonment of statistics after the 1776 memoir [10], it is a fact. The leading geometer of Europe never again after this date published anything on this subject. It is not that he turned away completely from the study of statistics tables and their applications in astronomy, population theory or insurances [12, 13], but his works, an undoubtedly small part of his oeuvre, were not probabilistic. Nevertheless, these subjects were fashionable in Europe, especially in France but also in Italy, with beautiful works by Italian economists and mathematicians, particularly Gaeta and Fontana, who undertook the translation of the treatise by de Moivre, at least the actuarial part [8]. Steve Stigler sent me a letter from Father Gregorio Fontana to Lambert, dated 20 August 1776, when he was at the University of Basel, in which the mathematician from Pavia wrote:

A présent je traduis de différentes langues en Italien tous les Opuscules qu’on écrit sur la Question de la régularité avec laquelle les Garçons qui naissent chaque année dans les grandes Villes excèdent toujours les Filles. Ces opuscules sont les Mém. du dr Arbuthnot, les lettres de Nicolas Bernoulli, la Dissert. de ‘sGravesande, ce qu’en dit Moivre dans sa Theorie of Chances, les deux Mém. de Mr. Dan. Bernoulli, et le 20me chap. de l’ouvrage de Süssmilch. Si vous, Monsieur, en connaissez d’autres, faites-moi la grâce de me les indiquer.

J’attends même de Mr. De la Place un Mémoire manuscrit pour l’insérer dans mon Recueil, et j’écris maintenant à Mr. La Grange pour le même objet.

(At present I am translating from different languages into Italian all of the book that have been written on the question of the regularity with which the [number of] boys that are born each year in the large cities exceed [that of] the girls. These books are the memoirs of Dr Arbuthnot, the letters of Nicolaus Bernoulli, the dissertation of de’sGravesande, of which de Moivre speaks in his Doctrine of Chances, the two memoirs by Daniel Bernoulli, and the 20th chapter of the work of Süssmilch. If you, Sir, know of others, please have the goodness to tell me of them.

I am waiting to receive from the same Mr Laplace a manuscript memoir to insert in my collection, and I am now writing to Mr Lagrange with the same object.)

Fontana then requested of Lambert a memoir “qui sera fort goûté en Italie, où ce genre d’études est fort en vogue” (that would be much enjoyed in Italy, where this kind of studies are powerful and in vogue).

In another letter of Fontana to Lambert, dated 5 May 1775, which Stigler also very kindly sent me, the Italian scholar wrote:

Mr. Jean Bernoulli m’a dit que le grand géomètre Mr. La Grange, à qui je vous prie de faire mes compliments, pense traduire le beau livre de Moivre intitulé The Doctrine of Chances avec des remarques de sa façon

(Mr Johann Bernoulli told me that the great geometer Mr Lagrange, to whom I beg you to present my compliments, is thinking of translating the beautiful book of de Moivre entitled The Doctrine of Chances with remarks of his own).

We know that this project was quite soon abandoned [[15], vol. XIV, p. 66], but it can be hypothesised that the notes and additions were already composed and formed the basis for the probabilistic parts of the 1777 memoir [11].

Lagrange thus intended to continue after 1775–1776. He didn’t, and no one really knows why. Was it because of a fundamental opposition to probabilistic methods in the theory of observations, his lack of desire and time for applications, or simply his desire to preserve the peace he had made with Lambert, Daniel Bernoulli, and above all Laplace, who had multiplied the first-rate work on this theme and who Lagrange was reluctant to confront directly? Already competing with him, by necessity, in physical astronomy, the most perfect of sciences, Lagrange might have had little desire to add the doctrine of probability, which, after all, remained subordinate in the European theoretical science of the late eighteenth century. It was probably better to test new ground where the director of the mathematical class of the Berlin Academy could freely express his mathematical genius to its full potentia—number theory, for example—without fear of being questioned or challenged on uncertain matters that lent themselves willingly to the academic battles that Lagrange abhorred. In short, it is unclear why, but we can see that the 1776 memoir on probability, extraordinary as it is, remains isolated in the work of Lagrange.

(Translated from the French by Kim Williams)