Probability of having leading digit 1 to 9 assuming exponential growth

Several possible explanations include:

1. Upper bound: How would you obtain data with leading digits evenly distributed among 1 to 9? One way is to use a random number generator from 1 to 99. This requires us to specify an upper bound but naturally occurring data does not have such a bound (the lower bound will always be 0 since we are concerned with positive numbers only). Also the upper bound has to be 9 or 99 or 999 … for 1 to 9 to have the same probability; if the bounds is 1 and 19, then the leading digit will more likely be 1.
   With this explanation, certain examples, such as numbers in the newspaper, following this distribution seem to make more sense. To understand how to get the exact probability try the following Exercise.
   
   
2. Exponential: population over time is roughly exponential. Suppose it starts at 1 and doubles every year (), then it takes 1 year to go from 1 to 2. To reach 3, solve using a calculator, we get , so it takes only 0.585 years to go from 2 to 3. This will continue to decrease because of exponential growth. But it will jump up when you calculate the time spent at 10 to 20 because we increased the increment when we go from unit digit to tens.
   Exercise 2: calculate the time spent from 10 to 20. Using properties of logarithm, what can you conclude? From this you can kind of see the fraction of time spent from 1 to 2, 10-20, 100-200, … versus the total time is just the fraction of time spent from 1 to 2 versus 1 to 10. Using properties of log, the answer is . This is exactly the one appearing in the above graph.With this calculation we obtain the log distribution: probability that the leading digit is d .
   Definition: A sequence of numbers is Benford if the leading digits approaching the log distribution in the limit as n approches infinity.
   It is sometimes simpler to work with the probability that the leading digit is d or less because it is .
   To understand why many phenomena satisfy Benford’s law without being exponential require us to explore exercise 2 more rigorously.
3. Multiplicative (Geometric): It turns out that lots of phenomena that are multiplicative in nature can be shown to satisfy Benford’s law. The way it is proved is similar to exponential growth. From , we can reduce multiplicative processes to linear ones, which is usually easier. For example, the stock market can be modeled by multiplying it by 2 and 1/2 with probability .5 and .5 , respectively, every year (where the values given are arbitrary and certainly inaccurate). If we take log, this becomes adding 1 and -1 with probability .5 and .5, which is like flipping a coin and trying to figure out the total number of heads after some time (the latter is called random walk and can be modeled by a bell-shaped curve by the Central Limit Theorem; the former is called geometric Brownian motion and satisfies Benford’s law)
   

   Geometric Brownian Motion Simulation

   
   
   

   Random Walk Simulation

4. Universal: Forget about the previous explanations. If there is a law about leading digit of physical data, then it shouldn’t depend on what unit; if it works for stock prices in US dollar, it should also work for Euro or British Pound. This is called scale-invariance. With this idea in mind, we will see that log distribution satisfies this property. More importantly, it is the only distribution that satisfies the probability.
   To give a definition of scale-invariance, we start with a sequence with the probability that a number from the sequence has leading digit less than d denoted by D(d), then this probability will be the same as that of the sequence , for any c>0.
   Similarly we can change the number system we use: instead of base 10, we use base 8. We hope this will again give us log is the only distribution that work (this won’t work though because the sequence {1, 1, 1, …} is the same no matter what base we are in. That means the distribution with 1 being 1 and everything else 0 will screw things up. But that’s the only problem.)

You may want to try other range such as 1 to 199. Then you see the probability keeps oscillating (between what?), never approaching a number and so you need some averaging process to get a single number. The key is to try to make the answer as independent of the bound as possible.

Answer: If we want to get rid of having to specify a bound, we can try looking at what happens as the bound gets bigger. The example above has probability 11/19, 11/29, 11/39, and we see it will oscillate between 1/9 and 5/9 but never approaching a single number. The next try is to average the probability for various upper bound: (1/1+1/2+…+1/9+2/10+3/11+…+10/18+11/19+11/20+…+11/99)/99=0.253 (this is quite difficult to calculate with a calculator, so for first try, I compute (11/19+11/29+…+11/99)/9=0.242. ). If we do this with more upper bounds, it will approach a single number around 24.1%. This is not the the desired answer(30.1%) but we are not far away. As explained in the literature, averaging once is not enough to get rid of the effects of having to specify the upper bound. If we take the data from the first average and average them again, it will become closer. To eliminate the upper bound effect completely, we have to repeatedly average the average and this will yield the answer, although we will not actually compute this limit here. For those interested, the result is due to Flehinger and one method of computing is presented here in one of the .ps file.

The iterated averaging scheme is only one of many methods because the main factor is how well we get rid of the effects of choosing an upper bound. For example, our first idea is to use a random number generator. Instead of choosing an upper bound, we will use a second random number generator with upper bound 900,000 to obtain the upper bound for the first generator. Or we can use a third generator to get the bound for the second one instead of specifying the upper bound of the second generator. Of course we will eventually need to make a choice of upper bound, this method works very well after chaining it a few times. In fact we can use other distributions (random number generator is uniform distribution), such as normal distribution (which has parameters to be specified as well). This explains why Benford’s Law seems to work especially well if you combine data from different sources (such as numbers in the newspaper).

For a very detailed exposition of everything, click here

Recall that we want to find the what proportion of a sequence satisfies for some integer m, with d being the leading digit we choose.

Let’s go over our example of population growth and modify things to give a new interpretation: the probability becomes

             (1)

Interpretation: Note that and lie between 0 and 1, so even though we specify m is whatever integer, it is forced to be the greatest integer that’s less than (for example if , m has to be 23 and ). Notation: will be denoted and geometrically the interval from 0 to 1 is glued to become a circle and 0.3 is the same as 1.3, 2.3, … and as n increases by 1, looks like rotation around a circle (with an irrational angle of 108.37… degree). Hence the name irrational rotation, meaning is irrational (Exercise: why is log 2 irrational? hint).

Notice the answer we seek () is precisely the difference between the left and right bound in . So what we need to see is that the sequence of points generated by irrational rotation is evenly distributed around the circle:

Definition: is equidistributed in [0,1] if for any interval (a, b) in [0,1], the proportion of in (a, b) is b-a.

So we are done if we show is equidistributed.

By the way, if we repeat everything above replacing by , we get the following:

Sufficient condition for Benford: if is equidistributed in [0,1], then is Benford.

In fact this is how many sequences are shown to be Benford.
Example 1: showing geometric Brownian motion is Benford is reduced to showing bell-shape probability distribution mod 1 is equidistributed in [0,1], which is more tractable although still messy. Therefore we won’t do it (showing as N approaches infinity).
Example 2: Fibonacci sequence is Benford. It is defined by recurrence relation and the first two values are 0 and 1. In fact, most sequences defined by recurrence relation is Benford. The reason is that they are roughly exponential (for Fibonacci, where b, c>1), so that taking log gives something that looks like irrational rotation for large n and can be shown to be equidistributed.
(Details for these examples can be found here)

Let’s get back to showing
Theorem: Irrational rotation, , is equidistributed in [0,1], where n=1, 2, … and b is irrational (Exercise: what happens if b is rational, say 3/4?)

One thing special about irrational is the following

Property: For any x in [0,1], there exist some number in as close to x as we want (but not equal to x).
Proof by example: let’s say x=0, b=0.31 and we want to find n so that so that we get closer to x by half or .31…/2. From the sequence {0.31…, 0.62…, 0.93…=0.07…, …}, it is not hard as the third number is good enough . It turns out we can keep on repeating, this time getting closer than .07…/2: {.07…, 0.14…, …, .98…=0.01…}. So why does it work? Because as you get close to x, the two numbers in the sequence that sandwich x are distance b away from each other so one of the two numbers has to be at most b/2 away from x. Picking x to be 0, we have find a way to increment be less than b/2, and so we have find a way to continue, this time finding two numbers in the sequence separated by less than b/2 that sandwich x (why doesn’t this work for rational, say b=1/4? It fails during the second step using x=0, because some points land on 0 and so we cannot cut the distance from x by half.)

Going back to the theorem, we want to see why it works for the case [0, 0.5]. We need to show the proportion*

as n approaches infinity.

Proof (Sketchy): It looks nasty to compute so let’s be crafty. We can ask about the proportion for [0.5, 1] instead. The answer should be the same intuitively and the above property confirms our intuition: by skipping an initial segment of , it will be for any x arbitrarily close to what we want, which is 0.5. Although skipping an initial segment will affect the proportion, but letting n go to infinity eliminates the effect.
Finally, since the proportion in [0, 0.5] and [0.5, 1] must add up to 1, each must be 0.5 and we are done. This argument can be repeated for other intervals and we are finally done.

The above concept is called translation invariance, characterizing uniform probability uniquely and is thus useful in proofs. It will be seen later because it is a disguise for scale invariance of log distribution.

Exercise a (generalized Benford’s law): If instead of the leading digit, we specify the first two digits that we want the probability, say what’s the proportion of with starting with 14, how would you compute it? Can you come up with a general formula? How about if we use base 7 instead of base 10.

Answer: You want to repeat the procedure to get (1). The first one is log 15 – log 14. If you change the base, then again repeat and you will see that the formula only changes from log base 10 to log base 7.

Different View: Dynamical System

The proportion* above is called the time average as you think of the irrational rotation as happening every second. There is also a space average, which is just the length of the interval under consideration (for above it was [0, 0.5]). Equidistribution can be obtained without using the sketchy argument above: it is a consequence of the fact that time average equals space average:

where Tx=x+b, and f(x)=1 if x mod 1 is in the interval ([0, 0.5] for above) and f(x)=0 otherwise.

Exercise: how does this give equidistribution?

For those who know fourier series, the proof of the formula with our specific map T involves showing the formula is true for (or sine) and then claim that we are done because f is periodic.

For more detail, click here.

(eq 1)

After we change the unit, becomes , and denote the new D and F with a bar on top, so , and scale invariance means . Repeatedly using eq 1,

Summarizing,

, for , and (eq. 2)

satisfies eq. 2, but does eq. 2 implies (for )?

First, we need to assume D(10)=1, which is reasonable. Second, we extend D to accept positive real numbers as input, not just positive integers, so that we can talk about D being continuous, which we will also assume (This is still reasonable if we take eq. 1 as the definition of D.). Then the answer is yes.

Exercise: D(1)=?

Answer: Let c=1, then D(1)=1 by eq. 2.

Exercise: Using D(1) and D(10), can you figure out other values of D using eq. 2?

For example, D(1/10)=-D(10)=-1 or more generally . That’s about it without being clever.

Trick for D(2): we only need to figure out for some integer q, because . The only thing we know is , so try to find integers q and p so that

(eq. 3)

where a is close to 1 (impossible for a to be 1, why?). After taking log, this means we want to approximate by p/q such that is as close to an integer as we want, which is possible by Lemma 1 as log 2 is irrational. Finally applying D to eq. 3 (and simplifying using eq. 2 many times) gives

, and in the limit as a approaches 1, p/q approaches log 2 and D(a) approaches 0 by continuity of D , so . Similar arguments give

where d>0.

What we can compare is …, base 0.1, base 10, base 100, base 1000, … (or base b^n, where b>1), because leading digit 1 to 2 in base 10 is leading digit 1 to 2 or 10 to 20 in base 100. Rewriting it as 10^0 to 10^(log 2) in base 10 and 100^0 to 100^(log 2 /2) or 100^(1/2) to 100^[(1+log 2)/2, we see the right definition of a base-invariant probability would be something like

where b is the base.

Using base-invariance will not give Benford’s law because the number 1 is special as changing the base will leave 1 unchanged. Removing 1, we will get Benford’s law (after some analysis which we won’t do. For the interested reader, see “Base-Invariance Implies Benford’s Law“).

Also, log can take in non-integer (positive) values as well, so it works as long as d>1. It helps to define the base b Mantissa function of a positive number p, as if where k is an integer such that . For example , , .

With Mantissa, the notation above is actually .

Using Mantissa, we can define invariant-sum sequence which is another characterization of Benford sequence just like scale and base-invariance. Using base 10, it means taking the numbers from a sequence whose leading digit is d, and summing up the Mantissa of the numbers, on average will be the same for d=1, 2, …, 9. For details, see this.