Missing information for large $N$¶

This worksheet is designed to see how large $N$ must be for the approximate expression for $I(N)$ be close to the (more) exact expression. That is, after using Stirling’s approximation, we have

$$ I(N) = k \left[- \ln N – \sum_{r=1}^n(NP_r)\ln P_r + n\ln N + \sum_{r=1}^n\ln P_r \right], $$

and for large $N$, the second term dominates, and we can approximate

$$ I(N) = -Nk \sum_{r=1}^n P_r\ln P_r . $$

For this code, we set $k = 1$ because we are only interested in the percent difference between these two.

In this worksheet, you can create your own probability distribution and plot this percent difference as a function of $N$.

Skip down to the “Run calculations” section if you just wish to test the code. In that section you will see

probs = createprobs(np.array([1./2,1./4,1./8]))

comparison(probs,100,2000,1)

In the first of these lines, the function createprobs has three fractions separated by commas. This is the example in the textbook (and this code is how I made the figure in the book). You can change these, and even add more terms (you don’t need just three options!). The function itself creates an array of probabilities from this, by adding an addition fraction (in this case a fourth) so that the sum of probs will equal 1.

Then comparison takes in the probability probs, and three integers. The first is the smallest value of $N$ to include, the second is the largest value of $N$ to include, and the final number (optional) is the percent you would like to see. (This is just for a visual to easily see where your plot reaches this value).

First click “Run All” in the Cells menu (in CoLab, it is in the Runtime menu). After you have run the notebook once, you can then change the probabilities on new lines below and hit “shift-enter” to make other plots.

Things to try out:

• See how large $N$ must be for the default probabilities for the difference to reach 0.1%, 0.01%, or 0.001%.

• See how $N$ changes in these cases when you have different probabilities, or even more than are included here.

• Go to the Functions section and see how the code works and try to modify it to do other things.

Functions¶

In this section, we have imported libraries that we need in order to simulate these outcomes. If you’re only interested in playing around with the results, just “run” the entire notebook and move to the next section. If you’re interested in coding and wish to modify this, have at it!

Run calculations¶