Tommy Ogden2017-11-17T21:37:14+00:00http://tommyogden.com/Tommy Ogdent@ogden.euSieving for Primes2017-08-03T00:00:00+00:00http://tommyogden.com/sieving-for-primes<aside>This post is also available as a <a href="http://nbviewer.ipython.org/url/tommyogden.com/assets/notes/sieving-for-primes/sieving-for-primes.ipynb">Jupyter notebook</a>.</aside>
<p>A common task in number theory problems is to find all the primes up to a number
$n$. <a href="https://projecteuler.net/">Project Euler</a> is a great resource for problems
like this. A simple method of finding primes, the <a href="https://en.wikipedia.org/wiki/Sieve_of_Eratosthenes">Sieve of
Eratosthenes</a>, was known in
Ancient Greece. You start with 2 and discard as non-prime all multiples of 2 up
to $n$. You then move onto the next number you haven’t discarded, 3, and mark as
non-prime all multiples of 3 up to $n$. We’ve discarded 4, so we move on to 5
and continue. We can stop when we get to $\sqrt n$ as we’ll have discarded all
the multiples by then.</p>
<p>Here’s what the algorithm looks like for $n = 400$.</p>
<script src="http://d3js.org/d3.v3.min.js"></script>
<style>
.svg-canvas {
background-color: white;
border: solid 1px rgba(208,199,198,1);
}
.box rect {
fill: rgba(208,199,198,1);
/*fill: rgba(113,107,105,1);*/
shape-rendering: crispEdges;
}
.box text {
fill: rgba(245,243,242,1);
font: 9px sans-serif;
text-anchor: middle;
alignment-baseline: middle;
}
.play path {
stroke: rgba(255,255,255,1);
stroke-width: 16px;
fill: rgba(113,107,105,1);
}
.play:hover path {
fill: rgba(189,54,19,1);
}
.play rect {
fill: none;
pointer-events: all;
cursor: pointer;
}
</style>
<aside>The numbers remaining in red at the end are determined to be
prime.</aside>
<aside><a href="http://bl.ocks.org/tommyogden/e63a893e60ba0c96badbb98050a2dedc">
See this example at bl.ocks.org</a></aside>
<script src="/assets/notes/sieving-for-primes/eratosthenes.js"></script>
<p>When should we use the Sieve? In this notebook I’ll compare it to a naive
iterative search by writing example algorithms in Python.</p>
<h2 id="solution-a-iterative-search">Solution A: Iterative Search</h2>
<p>Our first approach is to iteratively build a list of primes up to a limit $n$.</p>
<p>When checking a number $i$ for primality, we only need to check prime factors,
so we can check the list as we’re building it. Also, in checking if a number $i$
is prime, we only need to check for factors up to $\sqrt{i}$.</p>
<p>We’ll write a method that appends the next prime to an existing list of primes.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">primes_next</span><span class="p">(</span><span class="n">primes</span><span class="p">):</span>
<span class="s">"""Take an ordered list of prime numbers and append the next
prime.
Args:
primes: list of ordered primes, with the last prime odd,
e.g. [2,3].
Returns:
The list primes with the next prime appended,
e.g. [2,3,5].
"""</span>
<span class="n">i</span> <span class="o">=</span> <span class="n">primes</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="k">while</span> <span class="bp">True</span><span class="p">:</span>
<span class="n">i</span> <span class="o">+=</span> <span class="mi">2</span>
<span class="k">for</span> <span class="n">p</span> <span class="ow">in</span> <span class="n">primes</span><span class="p">:</span>
<span class="k">if</span> <span class="n">p</span><span class="o">**</span><span class="mi">2</span> <span class="o">></span> <span class="n">i</span><span class="p">:</span> <span class="c"># No factors found, i prime</span>
<span class="n">primes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">i</span><span class="p">)</span>
<span class="k">return</span> <span class="n">primes</span>
<span class="k">if</span> <span class="n">i</span><span class="o">%</span><span class="n">p</span> <span class="o">==</span> <span class="mi">0</span><span class="p">:</span> <span class="c"># Factor found, try next i</span>
<span class="k">break</span></code></pre></figure>
<p>So for example, if we have the list <code class="highlighter-rouge">[2, 3, 5]</code> we expect to return <code class="highlighter-rouge">[2, 3, 5,
7]</code>.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">print</span><span class="p">(</span><span class="n">primes_next</span><span class="p">([</span><span class="mi">2</span><span class="p">,</span> <span class="mi">3</span><span class="p">,</span> <span class="mi">5</span><span class="p">]))</span></code></pre></figure>
<aside>
[2, 3, 5, 7]
</aside>
<p>Now we repeat this until we get to $n$.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">primes_iterative</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="s">""" Build a list of the primes up to n iteratively. """</span>
<span class="n">p</span> <span class="o">=</span> <span class="p">[</span><span class="mi">2</span><span class="p">,</span><span class="mi">3</span><span class="p">]</span>
<span class="k">while</span> <span class="n">p</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span> <span class="o"><=</span> <span class="n">n</span><span class="p">:</span>
<span class="n">primes_next</span><span class="p">(</span><span class="n">p</span><span class="p">)</span>
<span class="n">p</span><span class="o">.</span><span class="n">pop</span><span class="p">()</span> <span class="c"># Discard the last one.</span>
<span class="k">return</span> <span class="n">p</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">print</span><span class="p">(</span><span class="n">primes_iterative</span><span class="p">(</span><span class="n">n</span><span class="o">=</span><span class="mi">20</span><span class="p">))</span></code></pre></figure>
<aside>
[2, 3, 5, 7, 11, 13, 17, 19]
</aside>
<p>We’ll compare the methods later by counting the number of primes up to $n$ each
returns.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">sol_a</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="s">""" Count the number of primes up to n iteratively."""</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">primes_iterative</span><span class="p">(</span><span class="n">n</span><span class="p">))</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">print</span><span class="p">(</span><span class="s">"The number of primes up to 1000 is {0}."</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">sol_a</span><span class="p">(</span><span class="n">n</span><span class="o">=</span><span class="mi">1000</span><span class="p">)))</span></code></pre></figure>
<aside>
The number of primes up to 1000 is 168.
</aside>
<h2 id="solution-b-sieve-of-eratosthenes">Solution B: Sieve of Eratosthenes</h2>
<p>For the Sieve of Eratosthenes algorithm, we’ll use some helper functions. First
we want a method to sieve out the multiples of a factor $f$.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">sieve_multiples</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">f</span><span class="p">):</span>
<span class="s">"""Set the primality of multiples of f to False.
Args:
A: List of booleans representing primality of each index.
f: Factor to find multiples of.
Notes:
- A is indexed such that A[0] represents 1, A[1] represents 2, etc.
- Only sieves factors greater than f^2.
"""</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">f</span><span class="o">**</span><span class="mi">2</span><span class="o">-</span><span class="mi">1</span><span class="p">,</span> <span class="nb">len</span><span class="p">(</span><span class="n">A</span><span class="p">),</span> <span class="n">f</span><span class="p">):</span>
<span class="n">A</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="bp">False</span></code></pre></figure>
<p>Note that we only need to sieve for factors greater than $f^2$, as the smaller
multiples will already have been discarded.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">A</span> <span class="o">=</span> <span class="p">[</span><span class="bp">True</span><span class="p">]</span><span class="o">*</span><span class="mi">10</span>
<span class="k">print</span><span class="p">(</span><span class="n">A</span><span class="p">)</span>
<span class="n">sieve_multiples</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="mi">2</span><span class="p">)</span>
<span class="n">sieve_multiples</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="mi">3</span><span class="p">)</span>
<span class="n">sieve_multiples</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="mi">5</span><span class="p">)</span>
<span class="k">print</span><span class="p">(</span><span class="n">A</span><span class="p">)</span></code></pre></figure>
<aside>
[True, True, True, True, True, True, True, True, True, True]<br />
[True, True, True, False, True, False, True, False, False, False]
</aside>
<p>Next a couple of simple helper methods. One to get us the next <code class="highlighter-rouge">True</code> value in a
boolean list.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">next_factor</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">start</span><span class="o">=</span><span class="mi">0</span><span class="p">):</span>
<span class="s">"""Returns the next True index in A, after start."""</span>
<span class="k">return</span> <span class="nb">next</span><span class="p">((</span><span class="n">i</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">a</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">A</span><span class="p">[</span><span class="n">start</span><span class="p">:],</span> <span class="n">start</span><span class="o">=</span><span class="n">start</span><span class="p">)</span> <span class="k">if</span> <span class="n">a</span><span class="p">),</span> <span class="bp">None</span><span class="p">)</span></code></pre></figure>
<p>Another to return all the indexes of all remaining <code class="highlighter-rouge">True</code> values, shifted by 1.
Once we’ve performed the sieve, this will represent the reminaing primes.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">remaining</span><span class="p">(</span><span class="n">A</span><span class="p">):</span>
<span class="s">"""Returns the indexes of all remaining True values, shifted by 1."""</span>
<span class="k">return</span> <span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">a</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">A</span><span class="p">)</span> <span class="k">if</span> <span class="n">a</span><span class="p">]</span></code></pre></figure>
<p>Now we’re ready to perform the sieve.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">primes_eratosthenes</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="s">""" Build a list of the primes below n by the Sieve of Eratosthenes. """</span>
<span class="n">A</span> <span class="o">=</span> <span class="p">[</span><span class="bp">True</span><span class="p">]</span><span class="o">*</span><span class="n">n</span>
<span class="n">A</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="bp">False</span> <span class="c"># 1 is not prime</span>
<span class="n">primes</span> <span class="o">=</span> <span class="p">[]</span>
<span class="n">p</span> <span class="o">=</span> <span class="mi">1</span> <span class="c"># 2 is prime</span>
<span class="k">while</span> <span class="n">p</span><span class="o">**</span><span class="mi">2</span> <span class="o"><</span> <span class="n">n</span><span class="p">:</span> <span class="c"># Only need to check up to sqrt(n)</span>
<span class="n">A</span><span class="p">[</span><span class="n">p</span><span class="p">]</span> <span class="o">=</span> <span class="bp">False</span>
<span class="n">primes</span><span class="o">.</span><span class="n">append</span><span class="p">(</span><span class="n">p</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="n">sieve_multiples</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">p</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="n">p</span> <span class="o">=</span> <span class="n">next_factor</span><span class="p">(</span><span class="n">A</span><span class="p">,</span> <span class="n">p</span><span class="p">)</span>
<span class="n">primes</span><span class="o">.</span><span class="n">extend</span><span class="p">(</span><span class="n">remaining</span><span class="p">(</span><span class="n">A</span><span class="p">))</span> <span class="c"># All remaining must be prime.</span>
<span class="k">return</span> <span class="n">primes</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">print</span><span class="p">(</span><span class="n">primes_eratosthenes</span><span class="p">(</span><span class="n">n</span><span class="o">=</span><span class="mi">20</span><span class="p">))</span></code></pre></figure>
<aside>
[2, 3, 5, 7, 11, 13, 17, 19]
</aside>
<p>Again we’ll write a count method to compare.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">sol_b</span><span class="p">(</span><span class="n">n</span><span class="p">):</span>
<span class="s">""" Count the number of primes up to n by the Sieve of Eratosthenes. """</span>
<span class="k">return</span> <span class="nb">len</span><span class="p">(</span><span class="n">primes_eratosthenes</span><span class="p">(</span><span class="n">n</span><span class="p">))</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">print</span><span class="p">(</span><span class="s">"The number of primes up to 1000 is {0}."</span><span class="o">.</span><span class="n">format</span><span class="p">(</span><span class="n">sol_b</span><span class="p">(</span><span class="n">n</span><span class="o">=</span><span class="mi">1000</span><span class="p">)))</span></code></pre></figure>
<aside>
The number of primes up to 1000 is 168.
</aside>
<h2 id="compare-counts">Compare Counts</h2>
<p>To check the result of the methods against each other, we’ll try a few values of
$n$. The methods could both be wrong of course.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">N</span> <span class="o">=</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">17</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">100</span><span class="p">,</span> <span class="mi">1000</span><span class="p">,</span> <span class="mi">10000</span><span class="p">,</span> <span class="mi">100000</span><span class="p">,</span> <span class="mi">1000000</span><span class="p">]</span>
<span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">N</span><span class="p">):</span>
<span class="k">assert</span><span class="p">(</span><span class="n">sol_a</span><span class="p">(</span><span class="n">n</span><span class="p">)</span> <span class="o">==</span> <span class="n">sol_b</span><span class="p">(</span><span class="n">n</span><span class="p">))</span></code></pre></figure>
<h2 id="complexity-analysis">Complexity Analysis</h2>
<p>Finally, we’ll compare timings by solving the problem over a range of $n$
values.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">N</span> <span class="o">=</span> <span class="p">[</span><span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">,</span> <span class="mi">50</span><span class="p">,</span> <span class="mi">100</span><span class="p">,</span> <span class="mi">1000</span><span class="p">,</span> <span class="mi">10000</span><span class="p">,</span> <span class="mi">100000</span><span class="p">,</span> <span class="mi">1000000</span><span class="p">,</span> <span class="mi">10000000</span><span class="p">]</span>
<span class="n">sol_times_a</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.0</span><span class="p">]</span><span class="o">*</span><span class="nb">len</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>
<span class="n">sol_times_b</span> <span class="o">=</span> <span class="p">[</span><span class="mf">0.0</span><span class="p">]</span><span class="o">*</span><span class="nb">len</span><span class="p">(</span><span class="n">N</span><span class="p">)</span>
<span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">n</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">N</span><span class="p">):</span>
<span class="n">result</span> <span class="o">=</span> <span class="o">%</span><span class="n">timeit</span> <span class="o">-</span><span class="n">o</span> <span class="n">sol_a</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="n">sol_times_a</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="n">best</span>
<span class="n">result</span> <span class="o">=</span> <span class="o">%</span><span class="n">timeit</span> <span class="o">-</span><span class="n">o</span> <span class="n">sol_b</span><span class="p">(</span><span class="n">n</span><span class="p">)</span>
<span class="n">sol_times_b</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">result</span><span class="o">.</span><span class="n">best</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="o">%</span><span class="n">matplotlib</span> <span class="n">inline</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="n">plt</span>
<span class="kn">import</span> <span class="nn">seaborn</span> <span class="k">as</span> <span class="n">sns</span>
<span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">loglog</span><span class="p">(</span><span class="n">N</span><span class="p">,</span> <span class="n">sol_times_a</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'A: Iterative'</span><span class="p">,</span> <span class="n">marker</span><span class="o">=</span><span class="s">'o'</span><span class="p">,</span> <span class="n">clip_on</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">loglog</span><span class="p">(</span><span class="n">N</span><span class="p">,</span> <span class="n">sol_times_b</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'B: Eratosthenes'</span><span class="p">,</span> <span class="n">marker</span><span class="o">=</span><span class="s">'o'</span><span class="p">,</span> <span class="n">clip_on</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">0</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s">'n'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s">'time (s)'</span><span class="p">);</span></code></pre></figure>
<figure>
<img class="text-framed" src="/assets/notes/sieving-for-primes/output_33_0.png" />
<figcaption>Fig. 1: Best-of-three timings.</figcaption>
</figure>
<p>In figure 1 we compare the best-of-three timings on my laptop of the two methods
over a range of $n$ values. We see that the iterative search is quicker for $n =
10$, but for $n = 20$ and beyond, the Eratosthenes method is quicker. Over this
range, both are polynomial below quadratic.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">speedup</span> <span class="o">=</span> <span class="p">[</span><span class="n">sol_times_a</span><span class="p">[</span><span class="n">i</span><span class="p">]</span><span class="o">/</span><span class="n">sol_times_b</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">x</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">sol_times_b</span><span class="p">)]</span>
<span class="n">plt</span><span class="o">.</span><span class="n">figure</span><span class="p">(</span><span class="n">figsize</span><span class="o">=</span><span class="p">(</span><span class="mi">10</span><span class="p">,</span> <span class="mi">5</span><span class="p">))</span>
<span class="n">plt</span><span class="o">.</span><span class="n">semilogx</span><span class="p">(</span><span class="n">N</span><span class="p">,</span> <span class="n">speedup</span><span class="p">,</span> <span class="n">marker</span><span class="o">=</span><span class="s">'o'</span><span class="p">,</span> <span class="n">clip_on</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">axhline</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">c</span><span class="o">=</span><span class="s">'grey'</span><span class="p">,</span> <span class="n">ls</span><span class="o">=</span><span class="s">'--'</span><span class="p">,</span> <span class="n">lw</span><span class="o">=</span><span class="mi">1</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s">'n'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s">'Eratosthenes Speedup'</span><span class="p">);</span></code></pre></figure>
<figure>
<img class="text-framed" src="/assets/notes/sieving-for-primes/output_35_0.png" />
<figcaption>Fig. 2: Speedup of the Eratosthenes method.</figcaption>
</figure>
<p>In figure 2 we show the speedup of the Eratosthenes method over the iterative
search. The speedup peaks at about 7$\times$ faster at $n = 10^4$. Beyond this
Eratosthenes method is still significantly faster, but the speedup is coming
down. This is likely due to the fact that the Sieve of Eratosthenes for high $n$
becomes memory intensive, as we need to store all of the numbers up to $n$,
whereas with the more computationally intensive iterative search, we only need
to store the primes we’ve already calculated.</p>
<p>The runtimes at $n = 10^7$ are in the order of minutes and already long enough
that we’d want to look at further tuning of the algorithm beyond this.</p>
January to March 20172017-06-20T00:00:00+00:00http://tommyogden.com/bookshelf/january-to-march-2017<h2 id="spqr-by-mary-beard">SPQR by Mary Beard</h2>
<p><em>A History of Ancient Rome</em>. You can only learn things by attaching them to
what you’ve already learned, and I hadn’t learned much about a thousand years
of Roman history. That said, I got more out of it than expected because Beard’s
writing is fun and the book is rarely dry.</p>
<p>One thing I do know about is the wood tablets at the fort of Vindolanda at
Hadrian’s Wall, because I’ve been to see them at the museum there. When the
first tablets were peeled apart and the ink writing discovered, they were
rushed to the university at Durham but oxidised before they could be
transcribed. The message was thought lost but at the medical school they found
they could still read it in the infrared. And they’ve been able to protect many
of the rest dug up so you can see them yourself. The tablets include the
oldest surviving writing in Latin by a woman, an invitation to a birthday
party. I recommend a visit.</p>
<h2 id="the-hunt-for-red-october-by-tom-clancy">The Hunt for Red October by Tom Clancy</h2>
<p>I always picture Alec Baldwin as Jack Ryan.</p>
<aside><span class="rating" title="The best books are indicated with ★.">★</span></aside>
<h2 id="the-lowland-by-jhumpa-lahiri">The Lowland by Jhumpa Lahiri</h2>
<p>A sweeping but intimate novel that starts in 1960s Calcutta with two young
brothers, Subhash and Udayan. Their lives fork, an impulsive Udayan caught
up in the Naxalite movement at home and a studious Subhash setting out for
university and a life in New England. It’s about staying and leaving, about
parenthood, loss, guilt and responsibility, through four generations of a
family.</p>
<aside><span class="rating" title="The best books are indicated with ★.">★</span></aside>
<h2 id="endurance-by-alfred-lansing">Endurance by Alfred Lansing</h2>
<p><em>Shackleton’s Incredible Voyage</em>. Lansing wrote this authoritative account of
the Endurance expedition, having got hold of most of the diaries kept on the
voyage and then interviewed the surviving crew. The mission of the great
explorer to make the first land crossing of the Antarctic went awry when the
tallship got stuck in the icepack of the Weddell Sea, was crushed and sank
leaving the crew without a hope.</p>
<p>There’s no way to condense the ordeals and suffering the book details. It is
unfathomable that they survived. After all that they’d endured over a year,
Shackleton took a few men in the tiny open boat <em>James Caird</em> and navigated 800
miles over the most deadly ocean on the planet to hit their only hope of
rescue, the speck of South Georgia island. Once there, they had to climb over
the sawtooth mountains and glaciers of the interior to get to the whaling
station, that itself a route nobody had survived attempting. Shackleton
then went back to rescue every one of his crew.</p>
<p>Frank Hurley shot early <a href="https://www.flickr.com/photos/statelibraryofnsw/albums/72157618020442474">colour photographs</a> and got the film home. They are
beautiful.</p>
<figure>
<a href="https://www.flickr.com/photos/statelibraryofnsw/3534618959/in/album-72157618020442474/">
<img class="text" src="/assets/photos/2017/hurley-endurance.jpg" alt="" /></a>
<figcaption>← ‘A mid-winter glow, Weddell Sea, 1915’ by Frank Hurley.</figcaption>
</figure>
<figure>
<a href="https://www.flickr.com/photos/statelibraryofnsw/3534591491/in/album-72157618020442474/">
<img class="text" src="/assets/photos/2017/hurley-shackleton.jpg" alt="" /></a>
<figcaption>← ‘Sir Ernest Shackleton watching a lead forming’ by Frank Hurley.</figcaption>
</figure>
<h2 id="ragtime-by-el-doctorow">Ragtime by E.L. Doctorow</h2>
<blockquote>
<p>‘Do not play this piece fast. It is never right to play Ragtime fast.’</p>
</blockquote>
<h2 id="the-carter-of-la-providence-by-georges-simenon">The Carter of La Providence by Georges Simenon</h2>
<p><em>Inspector Maigret #4.</em> Simenon is great at canals, as most other things.</p>
<h2 id="the-innovators-by-walter-isaacson">The Innovators by Walter Isaacson</h2>
<p><em>How a Group of Hackers, Geniuses, and Geeks Created the Digital Revolution</em>. A
history of the people who invented computers, microprocessors, software and the
web, from Ada Lovelace to Larry Page. Each character in the story is met only
briefly, because many people contributed.</p>
<p>And that’s really the theme of the book. We still have this notion of looking
for lone genius inventors when in fact innovation since the last century has
driven by brilliant thinkers in collaboration and many incremental
advances.</p>
October to December 20162017-01-03T00:00:00+00:00http://tommyogden.com/bookshelf/october-to-december-2016<aside><span class="rating" title="The best books are indicated with ★.">★</span></aside>
<h2 id="prisoners-of-geography-by-tim-marshall">Prisoners of Geography by Tim Marshall</h2>
<p><em>Ten Maps That Tell You Everything You Need to Know About Global Politics</em>. With
our century’s aircraft, internet and supranational institutions, we may be
tempted to think we’ve overcome the bounds of physical geography in global
affairs. Our history is told in ideologies and leaders, good or bad.</p>
<aside>
'There is on the globe one single spot, the possessor of which is our natural
and habitual enemy. It is New Orleans.' —Thomas Jefferson
</aside>
<p>Marshall shows us that plains, rivers and mountains are still predominant in ten
examples from around the world with corresponding maps. He explains why Tibet is
important to China, why Russia cares so much about the north European plain, and
why with the Louisiana Purchase complete, the United States was destined to be a
superpower.</p>
<p>He might overstate the case a bit but as I never see this kind of analysis
— only discussion of what Putin or Trump has said today — it’s a
refreshing way of looking at things. It’s a shallow dive but comes with a
decent bibliography to dig in further, which I intend to do.</p>
<h2 id="open-city-by-teju-cole">Open City by Teju Cole</h2>
<p>Open City follows the wandering thoughts of Julius on long walks around
Manhattan. I put it on a reading list about New York I researched before
visiting last October, then bought it from the geographically arranged shelves
of McNally Jackson. So I expect the book will always remind me of that city,
though the story resists ties of place.</p>
<aside>
'And so when I began to go on evening walks last fall, I found Morningside
Heights an easy place from which to set out into the city.'
</aside>
<p>I put it down after a few pages. Nobody thinks like Julius, joining dots between
Bach, Nietzsche, Auden, Mahler. I picked it up again and realised that the
connection of ideas, people, places was the point. We might be at Ground Zero
one moment and the Nigeria of his childhood next, or talking to a Moroccan
writer in a Brussels internet café. We’re in Julius’ head. That’s all novels,
but it’s the fluency here that makes it work.</p>
<p>I heard the citizen of the world is a ‘citizen of nowhere’. Cole reminds us that
art and people transcend nations. ‘I’m on a road to nowhere. <a href="https://www.youtube.com/watch?v=AWtCittJyr0">There’s a city in
my mind</a>.’</p>
<h2 id="mindset-by-carol-dweck">Mindset by Carol Dweck</h2>
<p><em>How You Can Fulfill Your Potential.</em> The most obvious criticism you can give
Mindset is that Dweck takes 250 pages to explain a thesis that can be given in
a couple of sentences: We should stop with the old debates of nuture versus
nature, genes versus environment, as research shows a greater indicator of
success is our approach to abilities. In broad strokes: we may have a <em>fixed
mindset</em> and consider our qualities to be set, or a <em>growth mindset</em> and
believe that our qualities can be developed through effort. Guess which works
best.</p>
<p>Importantly, the growth midset itself can be developed. Dweck is a research
psychologist, and cites convincing studies to back up the argument, though none
look at long-term effect. The rest of the book is given to examples from sports
and business and sweeping, mostly obvious advice for applying mindsets to work,
relationships and parenting.</p>
<aside>
'I've missed more than 9,000 shots in my career. I've lost almost 300 games. 26
times, I've been trusted to take the game-winning shot and missed. I've
failed over and over and over again in my life. And that is why I succeed.'
—Michael Jordan
</aside>
<p>I’ll note an example I like: Michael Jordan was no obvious natural talent. He didn’t
make his varsity team. He just worked harder than anyone to improve, even when
he’d been the best for so long.</p>
<p>It’s such a broad thesis that I suspect it’s easy to find examples to back it
and hard to disprove. I’d say the most valuable advice in the book is this:
don’t praise children for ability or talent, ‘you’re so clever’ and so on.
You’ll leave them fearing eventual failure, that you’ll see them turn out not
to be no genius after all and be forever disappointed. Instead praise their
successes for the effort and work that went into them. They’ll grow to treat
setbacks like Jordan’s missed shots.</p>
<h2 id="travels-with-charley-by-john-steinbeck">Travels with Charley by John Steinbeck</h2>
<p>In 1960, at 58, Steinbeck makes a months-long tour around America with his
French poodle Charley in a modified camper truck. He wants to see what’s changed
in the years he’s been in Europe and New York. What he finds isn’t always
positive: radio’s the same everywhere, some hitchhikers are racist. But for the
most part his account of the open road and the the characters he finds on it is
warm.</p>
<aside><span class="rating" title="The best books are indicated with ★.">★</span></aside>
<h2 id="my-ántonia-by-willa-carther">My Ántonia by Willa Carther</h2>
<aside>
'There seemed to be nothing to see; no fences, no creeks or trees, no hills or
fields. If there was a road, I could not make it out in the faint starlight.
There was nothing but land: not a country at all, but the material out of
which countries are made.'
</aside>
<p>A tough, romantic tale of the immigrant life of early settlers on the Nebraska
plain in the age of the Homestead Act.</p>
<p>Ántonia is a truly well-written character, and Carther is a master at
both people and landscapes.</p>
<blockquote>
<p>‘I was something that lay under the sun and felt it, like the pumpkins, and I
did not want to be anything more. I was entirely happy. Perhaps we feel like
that when we die and become a part of something entire, whether it is sun and
air, or goodness and knowledge. At any rate, that is happiness; to be
dissolved into something complete and great. When it comes to one, it comes as
naturally as sleep.’</p>
</blockquote>
<h2 id="turings-cathedral-by-george-dyson">Turing’s Cathedral by George Dyson</h2>
<p><em>The Origins of the Digital Universe.</em> An early history of the Institute for
Advanced Study at Princeton. Dyson has a good perspective from which to tell
this story, as he grew up there while his father Freeman Dyson was a fellow.</p>
<aside> 'In some sort of crude sense which no vulgarity, no humor, no
overstatement can quite extinguish, the physicists have known sin; and this is
a knowledge which they cannot lose.' —Robert Oppenheimer</aside>
<p>Though it is Turing’s name in the title, the book centres on the work of the
Hungarian émigré and polymath John von Neumann to build the general-purpose
ENIAC computer in the years up to 1946. Von Neumann, Teller and other hawks
thought the best route to world peace was a ‘preventive’ hydrogen bomb strike
on the Soviet Union, and the computer was intended to use their newly-invented
Monte Carlo methods to simulate thermonuclear explosion. Thankfully that bomb
was never dropped, peaceful uses for the computer were found in biology and
meteorology, and the result of the von Neumann architecture is the digital
world we now live in.</p>
July to September 20162016-09-21T00:00:00+00:00http://tommyogden.com/bookshelf/july-to-september-2016<h2 id="capitalism-by-james-fulcher">Capitalism by James Fulcher</h2>
<p><em>A Very Short Introduction.</em> A brief review of capitalism’s history, its emergence
in Europe and current global reach. The most interesting chapter for me is the
comparison between different models taken in the United States, Japan and
Sweden. On the latter:</p>
<blockquote>
<p>‘The labour movement’s leadership recognized that welfare depended not just on
the strength of socialist ideas and working class organization but also on
the operation of a dynamic capitalist economy that could compete
internationally and increase the size of the national economic cake. It was
one of the central principles of Swedish economic policy that unprofitable
companies should be allowed to go bankrupt, so that their resources could be
transferred to profitable sectors of the economy. Swedish workers were in
this respect rather less protected than British workers by government
interventions to bail out failing companies. Furthermore, the union-controlled
labour market policy did not protect jobs but assisted workers to
become mobile and retrain.’</p>
</blockquote>
<aside><span class="rating" title="The best books are indicated with ★.">★</span></aside>
<h2 id="the-crossing-by-cormac-mccarthy">The Crossing by Cormac McCarthy</h2>
<p>The second book in the Border Trilogy is a story of three crossings of the Texas
border by the teenaged Billy Parham. The first is a dreamlike journey to return
a wild, pregnant wolf to its Mexican mountain home, the second and third follow
in fate and circumstance.</p>
<aside>'The horse raising its head above the skyline to listen among the
constellations and then bending to graze again. He studied those worlds
sprawled in their pale ignitions upon the nameless night.'</aside>
<p>I can never get enough of McCarthy’s runaway sentences and the made-up words
that sound right as soon as you read them: ‘the godmade sun’, ‘the mute and
footsore dog’, ‘the needlethin sweepsecond hand sectoring the dial’. The best
are often saved for his keen-eyed attention to the animals.</p>
<aside><span class="rating" title="The best books are indicated with ★.">★</span></aside>
<h2 id="the-secret-history-by-donna-tartt">The Secret History by Donna Tartt</h2>
<p>Pretty sure if I’d read this novel before 2012 I would not have spent the next
three years living with classicists.</p>
April to June 20162016-06-30T00:00:00+00:00http://tommyogden.com/bookshelf/april-to-june-2016<h2 id="night-at-the-crossroads-by-georges-simenon">Night at the Crossroads by Georges Simenon</h2>
<p><em>Inspector Maigret #6.</em> Maigret investigates a murder at the remote Three Widows
crossroads.</p>
<h2 id="the-hanged-man-of-saint-pholien-by-georges-simenon">The Hanged Man of Saint-Pholien by Georges Simenon</h2>
<p><em>Inspector Maigret #3.</em> Maigret travels from Paris to Leige to solve the mystery
of a suicide which seems only to be about a suitcase containing an old suit.</p>
<h2 id="advanced-marathoning-by-pete-pfitzinger">Advanced Marathoning by Pete Pfitzinger</h2>
<p>Read for the second time as I prepared to run Manchester in April, the
Pfitzinger plan is now a key part of my yearly training. For now I’ve only done
the lowest mileage (80 km per week) of the brutal training regimes, but even
that moved my distance running on massively.</p>
<p>Along with the weekly plans, the chapters on physiology at the start helped me
to understand how each of the many track sessions, intervals and long weekend
runs develop $\mathrm{max(\dot{V}O_2)}$, lactate threshold and glycogen storage
on the way to race fitness.</p>
<h2 id="high-quality-software-engineering-by-david-drysdale">High Quality Software Engineering by David Drysdale</h2>
<p><em>Lessons from the Six-Nines World.</em> This free book available from <a href="http://lurklurk.org/">Drysdale’s
homepage</a> drew me in with the perspective offered in that subhead —
six-nines meaning 99.9999% uptime, or 31 seconds downtime per year. It covers
the main areas of software engineering: design, coding, reviews and testing, and
declares three themes: ‘maintainability’, ‘knowing reasons why’ and ‘developing
developers’.</p>
<p>For me the most useful sections were on designing for scale and reliability and
on testing strategies. Not so useful were those on project management, which
felt dated. The list of technical fundamentals near the back will serve as a
good to-do list for me.</p>
Menai Strait2016-04-03T00:00:00+00:00http://tommyogden.com/photos/2016/menai-strait<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26211355236/in/dateposted-public/" title="Early Light on the Strait">
<img class="full" src="https://farm2.staticflickr.com/1451/26211355236_5a4d0a0edd_k.jpg" alt="Early Light on the Strait"></a>
<!-- <script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> -->
<figcaption></figcaption>
</figure>
<p>Early light on the Strait</p>
Washington, D.C.2015-10-29T00:00:00+00:00http://tommyogden.com/photos/2015/washington-dc <!-- <script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> -->
<p>The United States is a welcoming place. This I knew from previous visits to
the New World. Apart from the time they served me something called
‘Biscuits and Gravy’ for breakfast, which turned out to <a
href="https://en.wikipedia.org/wiki/Biscuits_and_gravy"> contain neither
biscuits nor gravy</a>, I’ve always found
Americans to be perfect hosts.
What I wasn’t expecting was that, on our first trip to Washington,
me and Livy would get to step into the Oval Office. The actual one. With the
desk and the rug and the president and everything. Well, we had to wait until
Barack Obama had finished work for the day until we could nose about his place
of work, but fair enough.</p>
<p>I don’t think I’ve been as excited about being somewhere in my
life. We were asked not to take photos in the West Wing so I don’t have anything
from there to show, but at the time that just made me more determined to get
every photon I could into my eyeballs as we pottered around to gaze at the Navy
Mess, the Cabinet Room, the Roosevelt Room. And then that room with no
corners.</p>
<p>The Marine stationed at the door kindly offered that we take one step inside,
and I knew better than to ask for a second. From there I tried my best to
imprint the image on my retinas — the busts of Martin Luther King and
Lincoln, the two pastoral Edward Hoppers behind that hunk of wood from the HMS
Resolute saved from the Arctic ice.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25720567814/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1676/25720567814_9868d47fd5_k.jpg" alt="The White House from outside the West Wing."></a>
<figcaption>↑ The White House seen from outside the West Wing.</figcaption>
</figure>
<p>We were allowed to take photos in the Press Briefing Room (obviously I <a
href="https://www.instagram.com/p/9YV8IdxoCH/">pretended I was C.J. Cregg</a>)
and outside the building, so here’s a view of the Residence and North
Portico from outside the West Colonnade.</p>
<p>
During our time in what must be eighteen of the highest-security acres on the
planet, we were also invited to walk around the Eisenhower Executive Office
Building which sits next door and houses most of the administration staff
in its 500-odd offices.
</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26299453856/in/album-72157666837003091" title="">
<img class="half-left" src="https://farm2.staticflickr.com/1615/26299453856_fb8eff785a_k.jpg" alt=""></a>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26299450766/in/album-72157666837003091" title="">
<img class="half-right" src="https://farm2.staticflickr.com/1651/26299450766_fe4811a8ae_k.jpg" alt=""></a>
<figcaption>↑ Inside the <a href="https://www.whitehouse.gov/1600/eeob">Eisenhower Executive Office Building</a>.</figcaption>
</figure>
<p> The corridors and staircases of the Eisenhower building are superior to
those of the West Wing, which have low ceilings and functional carpets.
It’s hard to comprehend that the Eisenhower building was long unloved.
Mark Twain declared it ‘the ugliest building in America.’ I guess
he hadn’t seen Trump Tower. Having swerved a planned demolition in 1957 to
save on maintenance costs, it’s now a National Historic Landmark, so
should be safe.
</p>
<p>That's all I’ve got from the White House. If the people who made that
possible happen to read this — thank you.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25720552614/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1607/25720552614_6a2511f554_k.jpg" alt=""></a>
<figcaption>↑ National Gallery of Art, East Building</figcaption>
</figure>
<p>Of course we also popped into some of the amazing Smithsonian National
Museums around the Mall: of American History, of Natural History, of the
American Indian, of Air and Space, and the National Gallery of Art.</p>
<p>There are too many wonderful exhibits to list, but I would say
Charles Lindbergh’s <em>Spirit of St. Louis</em>, the Apollo 11 Command
Module, the Star-Spangled Banner, Doug Engelbart’s first prototype mouse
and Vincent van Gogh’s <a
href="https://en.wikipedia.org/wiki/Portraits_of_Vincent_van_Gogh#/media/File
:Vincent_van_Gogh_-_Self-Portrait_-_Google_Art_Project_(719161).jpg">blue, blue
self-portrait</a> stand out as memorable things to stand close to.</p>
<!-- National Museum of American History, . National Museum of Natural History
National Air and Space Museum, National Museum of the American Indian
National Gallery of Art -->
<!-- <p>Apart from plodding around 1600 Penn,
visited the amazing museums, including the various Smithsonians, Air and
Space, Native American.</p> -->
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26232992292/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1536/26232992292_987e246913_k.jpg" alt=""></a>
<figcaption>↑ Pennsylvania Avenue from the Newseum.</figcaption>
</figure>
<p>Both Livy and me particularly enjoyed the Newseum, which features exhibits on
the history of journalism, the First Amendment and the technological history of
media. A poignant memorial lists the many names of correspondents and
photographers killed in the course of their reporting, updated every year.</p>
<p>The balcony offers great views along Pennsylvania Avenue from the Capitol to
the White House.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26299438516/in/album-72157666837003091" title="">
<img class="half-left" src="https://farm2.staticflickr.com/1586/26299438516_769d42e95d_k.jpg" alt=""></a>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26299432336/in/album-72157666837003091" title="">
<img class="half-right" src="https://farm2.staticflickr.com/1663/26299432336_0c3d534f93_k.jpg" alt=""></a>
<figcaption>↑ Buzz Aldrin’s Apollo 11 Spacesuit.<br/>
← Rowhouses in Bloomingdale.</figcaption>
</figure>
<p>We stayed with friends in the Bloomingdale neighbourhood to the
north of the Capitol Building (which usefully marks the origin of the
city’s grid system). The street presents these beautiful
Victorian-style rowhouses, which feature on the <em>House of Cards</em> intro if
you look close.
<p>And that was the District of Columbia. From Union Station we took the Amtrak
back north through Baltimore to New York.</p>
<p><em>It's not the greenery turning gold in fall,
the scenery circling the Mall…</em></p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25720544774/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1444/25720544774_6530dad8fb_k.jpg" alt=""></a>
<!-- <figcaption>Lorem ipsum dolor sit amet</figcaption> -->
</figure>
Manhattan2015-10-29T00:00:00+00:00http://tommyogden.com/photos/2015/manhattan<!-- 'He was as tough and romantic as the city he loved.' -->
<!-- <script async src="//embedr.flickr.com/assets/client-code.js" charset="utf-8"></script> -->
<p>In October me and Liv took a trip to what I think collectively are called the
mid-Atlantic cities of the United States: New York City, Washington, D.C. and
Philadelphia. First we spent a week staying in an apartment on the Lower East
Side. These photos are from Manhattan.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26052314690/in/album-72157666837003091" title="">
<img class="full" src="https://farm2.staticflickr.com/1718/26052314690_7f273a846d_k.jpg" alt="">
</a>
<figcaption>↑ La Liberté éclairant le monde.</figcaption>
</figure>
<p>A walk along the High Line from Chelsea Market to the Hudson Yards.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26232791572/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1447/26232791572_32308d0462_k.jpg" alt=""></a>
<figcaption>↑ The construction site at Hudson Yards.</figcaption>
</figure>
<p>The construction site here is the largest development ever in the United States.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25720395164/in/album-72157666837003091" title="">
<img class="half-left" src="https://farm2.staticflickr.com/1719/25720395164_4d0ce4c2d9_k.jpg" alt=""></a>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25720302644/in/album-72157666837003091" title="">
<img class="half-right" src="https://farm2.staticflickr.com/1527/25720302644_ce90d32ffd_k.jpg" alt=""></a>
<figcaption>↑ The Brooklyn Bridge<br/>← The Guggenheim</figcaption>
</figure>
<p>We went to a few of the museums of course: the Guggenheim, MoMA and The Metropolitan. I
took early runs along the East River under the bridges.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25722479453/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1574/25722479453_8bc4d94ab3_k.jpg" alt=""></a>
<figcaption>↑ Fall turns in Central Park.</figcaption>
</figure>
<p>Another day we rode bikes around the big loop of Central Park.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26232725062/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1625/26232725062_6afe765865_k.jpg" alt=""></a>
<figcaption></figcaption>
</figure>
<!-- <p>Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod
tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam,
quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo
consequat.</p> -->
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25722363883/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1646/25722363883_14ab7bb226_k.jpg" alt=""></a>
<figcaption>↑ McNally Jackson books.</figcaption>
</figure>
<p>We got to the Housing Works Bookstore and to McNally Jackson. Didn't make
it to Strand but hopefully next time.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/25720289784/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1668/25720289784_e8890a34fb_k.jpg" alt=""></a>
<figcaption>↑ Looking downtown from the Rockefeller Centre.</figcaption>
</figure>
<p>Your standard tourist skyscraper view of Gotham, of course. I think the viewing point from
the balcony bar at the Met is better, perched above Central Park.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26052452400/in/album-72157666837003091" title="">
<img class="half-left" src="https://farm2.staticflickr.com/1697/26052452400_37047fd457_k.jpg" alt=""></a>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26299125176/in/album-72157666837003091/" title="">
<img class="half-right" src="https://farm2.staticflickr.com/1619/26299125176_f88ca606b5_k.jpg" alt=""></a>
<figcaption>↑ Orchard Street on the Lower East Side.<br/>
← Brownstones in Harlem.</figcaption>
</figure>
<p>I would recommend a tour of the building maintained at the Tenement Museum on
Orchard Street on the Lower East Side. Each floor is preserved at a different
decade in its history, and tells the story of the migrant families living there
at each point. It's a time capsule, boarded up in the 1930s and
untouched until the museum bought the building in 1988.</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26325331755/in/album-72157666837003091" title="">
<img class="full" src="https://farm2.staticflickr.com/1476/26325331755_783f42e829_k.jpg" alt=""></a>
<figcaption>↑ The Library of Columbia University.</figcaption>
</figure>
<p>Morningside Heights and the campus at Columbia.</p>
<figure>
<a ata-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26052399340/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1571/26052399340_5592f0a930_k.jpg" alt=""></a>
<figcaption>↑ Looking out to the Upper Bay and Verrazano-Narrows Bridge
from the crown of the Statue of Liberty.</figcaption>
</figure>
<p>Livy was smart enough to book us tickets to go up to the crown in advance,
so we got to go up there.</p>
</p>
<figure>
<a data-flickr-embed="true" href="https://www.flickr.com/photos/pantone292/26299265826/in/album-72157666837003091/" title="">
<img class="full" src="https://farm2.staticflickr.com/1645/26299265826_e96de94463_k.jpg" alt=""></a>
<figcaption></figcaption>
</figure>
The Two-Step Adams-Bashforth Method with Different Stepsizes2014-09-25T00:00:00+00:00http://tommyogden.com/the-two-step-adams-bashforth-method-with-different-stepsizes<aside>This post is also available as a <a href="http://nbviewer.ipython.org/url/tommyogden.com//assets/notes/the-two-step-adams-bashforth-method-with-different-stepsizes/the-two-step-adams-bashforth-method-with-different-stepsizes.ipynb">Jupyter
notebook</a>.</aside>
<p>The Adams-Bashforth family of numerical methods has a well-known derivation
but I couldn’t find a source which gave the two-step method in the case that the
two stepsizes are different. Why do I want that? The two-step method requires
two initial points. The second point is often calculated using a Euler step, but
as the Euler method is $\mathcal{O(h^1)}$ I want to make this first step small
to avoid introducing a large global error. The third point is then calculated
with the Adams-Bashforth method with different step sizes. From then on the
Adams-Bashforth method can be used as usual. Another use might be in an
<a href="http://en.wikipedia.org/wiki/Adaptive_stepsize">adaptive stepsize</a> method, where we want to adjust the stepsizes as
we go.</p>
<h2 id="an-initial-condition-ode-problem">An Initial Condition ODE Problem</h2>
<p>Say we have an ordinary differential equation $ y’ = f(t,y(t)) $ with an initial
condition $y(t_0) = y_0$ and we want to solve it numerically. If we know $y(t)$
at a time $t_n$ and want to know what $y$ is at a later time $t_{n+1}$, the
fundamental theorem of calculus tells us that we find it by integrating $y’$
over the time interval,</p>
<script type="math/tex; mode=display">y(t_{n+1}) = y(t_n) + \int_{t_n}^{t_{n+1}} \! y'(t) \, \mathrm{d}t = y(t_n) +
\int_{t_n}^{t_{n+1}} \! f(t,y(t)) \, \mathrm{d}t.</script>
<p>The idea behind any ODE solver is to compute the right-hand-side integral for
some numerical approximation of $f$. The problem is then computed over a series
of steps $n = 1, 2, \dots N$ to give a sequence of points $y_n$ which
approximate $y(t)$ to some order of accuracy as a function of the stepsize. The
method is <em>consistent</em> if the local error (i.e. the error from step $n$ to step
$n+1$) goes to zero faster than the stepsize $(t_{n+1} - t_n)$ goes to zero.</p>
<h2 id="polynomial-interpolation">Polynomial Interpolation</h2>
<p>Where the Euler method takes the slope $f$ to be a constant on the interval
$[t_n, t_{n+1}]$, the idea behind <em>Adams-Bashforth</em> methods is to approxmiate
$f$ by a <a href="lagrange">Lagrange interpolating polynomial</a>:</p>
<script type="math/tex; mode=display">P(t) = \sum_{j=1}^{m}{P_j(t)}</script>
<p>where</p>
<script type="math/tex; mode=display">P_j(t) = y_j \prod_{\substack{k=1 \\ k \ne j}}^{m}{ \frac{t - t_k}{t_j - t_k}
}.</script>
<p>Here $P(t)$ is the polynomial of degree $\le (m-1)$ that passes through the $m$
points $(t_1, y_1 = f(t_1))$, $(t_2, y_2 = f(t_2))$ $\dots$ $(t_m, y_m =
f(t_m))$. We’ll take the linear $(m = 2)$ interpolant on the point $t_{n}$ and
an earlier point $t_{n-1}$, so we have</p>
<script type="math/tex; mode=display">P(t) = f(t_n, y_n)\frac{t - t_{n-1}}{t_n - t_{n-1}} + f(t_{n-1},
y_{n-1})\frac{t - t_{n}}{t_{n-1} - t_n}.</script>
<p>Now if we put this approximating polynomial into the integral above, we
find</p>
<script type="math/tex; mode=display">% <![CDATA[
\begin{align}
\int_{t_n}^{t_{n+1}} \! f(t,y(t)) \, \mathrm{d}t &\approx \int_{t_n}^{t_{n+1}}
\! P(t) \, \mathrm{d}t \\
&= \int_{t_n}^{t_{n+1}} \! \left[ f(t_n, y_n)\frac{t - t_{n-1}}{t_n - t_{n-1}} +
f(t_{n-1}, y_{n-1})\frac{t - t_{n}}{t_{n-1} - t_n} \right] \mathrm{d}t \\
&= \frac{(t_n - t_{n+1})}{2(t_{n-1}-t_n)} \left[ f(t_n,y_n)(t_n + t_{n+1} -
2t_{n-1}) - f(t_{n-1},y_{n-1})(t_n - t_{n+1}) \right]
\end{align} %]]></script>
<h2 id="step-sizes">Step Sizes</h2>
<p>If we let $h_1 := t_n - t_{n-1}$ and $h_2 := t_{n+1} - t_n$ then</p>
<script type="math/tex; mode=display">\int_{t_n}^{t_{n+1}} \! P(t) \, \mathrm{d}t = \frac{h_2}{2 h_1} \left[ (2 h_1
+ h_2) f(t_n,y_n) - h_2 f(t_{n-1},y_{n-1}) \right].</script>
<p>Putting this back into the approximation, we get</p>
<script type="math/tex; mode=display">y(t_{n+1}) \approx y(t_{n}) + \frac{h_2}{2 h_1} \left[ (2 h_1 + h_2)
f(t_n,y_n) - h_2 f(t_{n-1},y_{n-1}) \right]</script>
<p>and our sequence of approximation points $y_n$ is calculated as</p>
<aside>This is the key result of this post: the formula when the two steps $h_1$ and
$h_2$ are not equal.</aside>
<script type="math/tex; mode=display">y_{n+1} = y_n + \frac{h_2}{2 h_1} \left[ (2 h_1 + h_2) f(t_n,y_n) - h_2
f(t_{n-1},y_{n-1}) \right]</script>
<p>for $n = 1, 2, \dots N$. If the steps are of <em>equal size</em>, i.e. $h := h_1 = h_2$
we find</p>
<script type="math/tex; mode=display">y_{n+1} = y_n + \frac{3}{2} h f(t_n,y_n) - \frac{1}{2} h f(t_{n-1},
t_{n-1}),</script>
<p>which is the <a href="http://en.wikipedia.org/wiki/Linear_multistep_method#Families_of_multistep_methods">standard two-step Adams-Bashforth method</a>.</p>
<h2 id="accuracy">Accuracy</h2>
<p>Replacing $f(t,y(t))$ with the interpolant $P(t)$ <a href="http://en.wikipedia.org/wiki/Linear_multistep_method#CITEREFIserles1996">incurs a global
error</a> of order $\mathcal{O}(h^m)$, so in the case of the two-
step method we have $\mathcal{O}(h^2)$.</p>
<p>Note that if you follow the same derivation with $m = 1$ you get the Euler
method — so the Euler method is also in fact the one-step Adams-Bashforth
method.</p>
<h2 id="python-implementation-of-the-method">Python Implementation of the Method</h2>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">import</span> <span class="nn">numpy</span> <span class="k">as</span> <span class="n">np</span>
<span class="kn">import</span> <span class="nn">matplotlib.pyplot</span> <span class="k">as</span> <span class="n">plt</span>
<span class="o">%</span><span class="n">matplotlib</span> <span class="n">inline</span></code></pre></figure>
<p>We’ll now define a function to implement the two-step Adams-Bashforth method for
a system of first-order ODEs. Below we’ll try it out on a test equation.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">ode_int_ab</span><span class="p">(</span><span class="n">func</span><span class="p">,</span> <span class="n">y_0</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="n">args</span><span class="o">=</span><span class="p">{}):</span>
<span class="s">""" Two-Step Adams-Bashforth approximation to a first-order ODE system
with initial conditions.
Args:
func: (callable) The first-order ODE system to be approximated.
y_0: (array) The initial condition.
t: (array) A sequence of time points for which to solve for y.
args: (dict) Extra arguments to pass to function.
Out:
y: (array) the approximated solution of the system at each time in t,
with the initial value y_0 in the first row.
"""</span>
<span class="c"># Initialise the approximation array</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">([</span><span class="nb">len</span><span class="p">(</span><span class="n">t</span><span class="p">),</span> <span class="nb">len</span><span class="p">(</span><span class="n">y_0</span><span class="p">)])</span>
<span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">=</span> <span class="n">y_0</span>
<span class="c">### Step 0: Euler</span>
<span class="n">h</span> <span class="o">=</span> <span class="n">t</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="n">t</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">y</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span> <span class="o">+</span> <span class="n">h</span><span class="o">*</span><span class="n">func</span><span class="p">(</span><span class="n">t</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">args</span><span class="p">)</span> <span class="c"># Euler step</span>
<span class="c">### Step 1: Adams-Bashforth, Different Stepsizes</span>
<span class="n">h_1</span> <span class="o">=</span> <span class="n">t</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="n">t</span><span class="p">[</span><span class="mi">0</span><span class="p">]</span>
<span class="n">h_2</span> <span class="o">=</span> <span class="n">t</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">-</span> <span class="n">t</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span>
<span class="n">y</span><span class="p">[</span><span class="mi">2</span><span class="p">]</span> <span class="o">=</span> <span class="n">y</span><span class="p">[</span><span class="mi">1</span><span class="p">]</span> <span class="o">+</span> <span class="mf">0.5</span><span class="o">*</span><span class="n">h_2</span><span class="o">/</span><span class="n">h_1</span><span class="o">*</span><span class="p">((</span><span class="mf">2.</span><span class="o">*</span><span class="n">h_1</span> <span class="o">+</span> <span class="n">h_2</span><span class="p">)</span><span class="o">*</span><span class="n">func</span><span class="p">(</span><span class="n">t</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="mi">1</span><span class="p">],</span> <span class="n">args</span><span class="p">)</span> <span class="o">-</span>
<span class="n">h_2</span><span class="o">*</span><span class="n">func</span><span class="p">(</span><span class="n">t</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="mi">0</span><span class="p">],</span> <span class="n">args</span><span class="p">))</span>
<span class="c">### Steps 2 to N-1: Adams-Bashforth</span>
<span class="c"># Loop through the time steps</span>
<span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">t_i</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">t</span><span class="p">[</span><span class="mi">2</span><span class="p">:</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">start</span><span class="o">=</span><span class="mi">2</span><span class="p">):</span>
<span class="n">h</span> <span class="o">=</span> <span class="n">t</span><span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">-</span> <span class="n">t_i</span> <span class="c"># size of the step</span>
<span class="n">y</span><span class="p">[</span><span class="n">i</span><span class="o">+</span><span class="mi">1</span><span class="p">]</span> <span class="o">=</span> <span class="n">y</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">+</span> <span class="p">(</span><span class="mf">1.5</span><span class="o">*</span><span class="n">h</span><span class="o">*</span><span class="n">func</span><span class="p">(</span><span class="n">t_i</span><span class="p">,</span> <span class="n">y</span><span class="p">[</span><span class="n">i</span><span class="p">],</span> <span class="n">args</span><span class="p">)</span> <span class="o">-</span>
<span class="mf">0.5</span><span class="o">*</span><span class="n">h</span><span class="o">*</span><span class="n">func</span><span class="p">(</span><span class="n">t</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">y</span><span class="p">[</span><span class="n">i</span><span class="o">-</span><span class="mi">1</span><span class="p">],</span> <span class="n">args</span><span class="p">))</span> <span class="c"># Adams-Bashforth</span>
<span class="k">return</span> <span class="n">y</span></code></pre></figure>
<h2 id="a-test-problem-the-exponential">A Test Problem: The Exponential</h2>
<p>To test our solver, let’s take a simple ODE: $y’ = ay$ with intial value $y(0) =
1$ and $a \in \mathbb{C}$. We know the analytic solution of this equation is $y
= \mathrm{e}^{at}$ so we can check the accuracy of the method against this.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">def</span> <span class="nf">exp</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="n">args</span><span class="p">):</span>
<span class="s">""" An exponential function described as a first-order ODE. """</span>
<span class="n">dydt</span> <span class="o">=</span> <span class="n">args</span><span class="p">[</span><span class="s">'a'</span><span class="p">]</span><span class="o">*</span><span class="n">y</span>
<span class="k">return</span> <span class="n">dydt</span></code></pre></figure>
<aside>I've gone for a one-line docstring in this case as the ODE function is short
and hopefully transparent.</aside>
<p>Now we’ve defined our solver and a test method, we can check that the method
works for some example parameters. Here we’ll set $a = 1$ and solve over $t \in
[0, 5]$.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">y_0</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">array</span><span class="p">([</span><span class="mf">1.</span><span class="p">])</span> <span class="c"># Initial condition</span>
<span class="n">solve_args</span> <span class="o">=</span> <span class="p">{</span><span class="s">'a'</span><span class="p">:</span> <span class="mf">1.</span><span class="p">}</span>
<span class="n">t_max</span> <span class="o">=</span> <span class="mf">5.</span></code></pre></figure>
<p>Next we’ll solve the system for different numbers of steps: $N = 5, 10, 20$
(corresponding to $h = 1, \frac{1}{2}, \tfrac{1}{4}$). Then we’ll plot the
results alongside the known analytic result $y = \mathrm{e}^{t}$.</p>
<p>For now, note that we’re leaving the first Euler step the same size as all the
subsequent steps.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="k">for</span> <span class="n">num_steps</span> <span class="ow">in</span> <span class="p">[</span><span class="mi">5</span><span class="p">,</span> <span class="mi">10</span><span class="p">,</span> <span class="mi">20</span><span class="p">]:</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mf">0.</span><span class="p">,</span> <span class="n">t_max</span><span class="p">,</span> <span class="n">num_steps</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span> <span class="c"># Time steps</span>
<span class="n">y</span> <span class="o">=</span> <span class="n">ode_int_ab</span><span class="p">(</span><span class="n">exp</span><span class="p">,</span> <span class="n">y_0</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="n">solve_args</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">y</span><span class="p">,</span> <span class="s">'-o'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="nb">str</span><span class="p">(</span><span class="n">num_steps</span><span class="p">)</span> <span class="o">+</span> <span class="s">' steps'</span><span class="p">)</span>
<span class="n">y_known</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">solve_args</span><span class="p">[</span><span class="s">'a'</span><span class="p">]</span><span class="o">*</span><span class="n">t</span><span class="p">)</span> <span class="c"># Analytic result for comparison</span>
<span class="n">plt</span><span class="o">.</span><span class="n">plot</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="n">y_known</span><span class="p">,</span> <span class="s">'k--'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'known'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s">'$t$'</span><span class="p">),</span> <span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s">'$y$'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span></code></pre></figure>
<figure>
<img class="text-framed" src="/assets/notes/the-two-step-adams-bashforth-method-with-different-stepsizes/B_Deriving_the_Two-Step_Adams-Bashforth_Method_15_0.png" />
<figcaption>Fig. 1: Numerical approximation integrating over 5 (blue), 10 (green) and 20 (red) steps. The known solution is shown for comparison (black dotted).</figcaption>
</figure>
<p>From the plot above the method implementation looks good: the numerical solution
behaviour follows the known analytic (black dashed line) solution, and is
converging on it as we increase the number of steps. We can check that the order
of the approximation is indeed $\mathcal{O}(h^2)$ by plotting a function of the
global error at $t=5$, given by $| \, y_N - e^5 \, |$, over a large range of
stepsizes.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">max_N</span> <span class="o">=</span> <span class="mi">16</span>
<span class="n">N</span> <span class="o">=</span> <span class="mi">2</span><span class="o">**</span><span class="n">np</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">1</span><span class="p">,</span> <span class="n">max_N</span><span class="p">)</span> <span class="c"># N = 2, 4, 8, ..., 2^max_N</span>
<span class="n">order_check</span> <span class="o">=</span> <span class="mi">2</span> <span class="c"># for visual check of the order of accuracy</span>
<span class="n">y_end</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">N</span><span class="p">))</span> <span class="c"># array to fill with the final values</span>
<span class="n">stepsize</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">N</span><span class="p">))</span> <span class="c"># array to fill with the stepsizes</span>
<span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">N_i</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">N</span><span class="p">):</span> <span class="c"># Loop over different numbers of steps</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="mf">0.</span><span class="p">,</span> <span class="n">t_max</span><span class="p">,</span> <span class="n">N_i</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="n">y_end</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">ode_int_ab</span><span class="p">(</span><span class="n">exp</span><span class="p">,</span> <span class="n">y_0</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="n">solve_args</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="n">stepsize</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">t_max</span><span class="o">/</span><span class="n">N_i</span>
<span class="n">plt</span><span class="o">.</span><span class="n">loglog</span><span class="p">(</span><span class="n">stepsize</span><span class="p">,</span> <span class="nb">abs</span><span class="p">(</span><span class="n">y_end</span> <span class="o">-</span> <span class="n">y_known</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]),</span> <span class="s">'-o'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'Global error'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">loglog</span><span class="p">(</span><span class="n">stepsize</span><span class="p">,</span> <span class="n">stepsize</span><span class="o">**</span><span class="n">order_check</span><span class="p">,</span><span class="s">'k--'</span><span class="p">,</span> <span class="n">label</span><span class="o">=</span><span class="s">'$h^2$'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s">'$h$'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">legend</span><span class="p">(</span><span class="n">loc</span><span class="o">=</span><span class="mi">2</span><span class="p">)</span></code></pre></figure>
<figure>
<img class="text-framed" src="/assets/notes/the-two-step-adams-bashforth-method-with-different-stepsizes/B_Deriving_the_Two-Step_Adams-Bashforth_Method_17_0.png" />
<figcaption>Fig. 2: Global error (blue) against stepsize $h$. The function $h^2$ is shown for comparison (black dotted).</figcaption>
</figure>
<p>Plotting the global error (blue line) against stepsize on a logarithmic scale,
we see that the slope is constant below $h = 0.1$, which tells us that the order
is constant. We could find that order formally by fitting this logarithm, but
it’s good enough for now to compare the global error with the function $h^2$
(black dashed line). That the slopes are the same indicates visually that the
method is $\mathcal{O}(h^2)$. (Try comparing with $h^1$ or $h^3$ by changing
<code class="highlighter-rouge">order_check</code> in the code above to $1$ or $3$.)</p>
<h2 id="changing-the-first-step-size">Changing the First Step Size</h2>
<p>Finally, remember that the reason we derived the two-step Adams-Bashforth method
with different stepsizes was so we could make the first Euler
$\mathcal{O}(h^1)$ step smaller. This step will otherwise introduce a large
error which will carry through the subsequent Adams-Bashforth steps. How small
should we make it?</p>
<p>We’ll use the test ODE system <code class="highlighter-rouge">exp</code> as above but this time keep the main
stepsize fixed at $h = 10^{-2}$. Then we’ll compare taking the first Euler
stepsize $h_0$ from $10^{-2}$ down to $10^{-6}$.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">num_stepsizes</span> <span class="o">=</span> <span class="mi">40</span>
<span class="c"># logarithmically-spaced points from 10^-6 to 10^-2</span>
<span class="n">first_stepsizes</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">logspace</span><span class="p">(</span><span class="o">-</span><span class="mi">6</span><span class="p">,</span> <span class="o">-</span><span class="mi">2</span><span class="p">,</span> <span class="n">num_stepsizes</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="n">y_end</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">zeros</span><span class="p">(</span><span class="nb">len</span><span class="p">(</span><span class="n">first_stepsizes</span><span class="p">))</span>
<span class="n">num_steps</span> <span class="o">=</span> <span class="mf">500.</span> <span class="c"># i.e. h = 0.01</span>
<span class="k">for</span> <span class="n">i</span><span class="p">,</span> <span class="n">first_step</span> <span class="ow">in</span> <span class="nb">enumerate</span><span class="p">(</span><span class="n">first_stepsizes</span><span class="p">):</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">linspace</span><span class="p">(</span><span class="n">first_step</span><span class="p">,</span> <span class="n">t_max</span><span class="p">,</span> <span class="n">num_steps</span><span class="o">+</span><span class="mi">1</span><span class="p">)</span>
<span class="n">t</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">insert</span><span class="p">(</span><span class="n">t</span><span class="p">,</span> <span class="mi">0</span><span class="p">,</span> <span class="mf">0.</span><span class="p">)</span> <span class="c"># Put a first 'small' step first</span>
<span class="n">y_end</span><span class="p">[</span><span class="n">i</span><span class="p">]</span> <span class="o">=</span> <span class="n">ode_int_ab</span><span class="p">(</span><span class="n">exp</span><span class="p">,</span> <span class="n">y_0</span><span class="p">,</span> <span class="n">t</span><span class="p">,</span> <span class="n">solve_args</span><span class="p">)[</span><span class="o">-</span><span class="mi">1</span><span class="p">]</span>
<span class="n">y_known</span> <span class="o">=</span> <span class="n">np</span><span class="o">.</span><span class="n">exp</span><span class="p">(</span><span class="n">solve_args</span><span class="p">[</span><span class="s">'a'</span><span class="p">]</span><span class="o">*</span><span class="n">t</span><span class="p">)</span></code></pre></figure>
<p>We’ll plot the global error on a semilogarithmic axis.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">plt</span><span class="o">.</span><span class="n">semilogx</span><span class="p">(</span><span class="n">first_stepsizes</span><span class="p">,</span> <span class="nb">abs</span><span class="p">(</span><span class="n">y_end</span> <span class="o">-</span> <span class="n">y_known</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]),</span> <span class="s">'-o'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s">r'$h_0$'</span><span class="p">),</span> <span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s">'Global error'</span><span class="p">)</span></code></pre></figure>
<figure>
<img class="text-framed" src="/assets/notes/the-two-step-adams-bashforth-method-with-different-stepsizes/B_Deriving_the_Two-Step_Adams-Bashforth_Method_22_0.png" />
<figcaption>Fig. 3: Global error against first stepsize $h_0$.</figcaption>
</figure>
<p>As we’d expect, the global error decreases as the contribution from the first
error is brought down by reducing the first stepsize. But if we zoom in on the
y-axis, we see that we can only bring it down so far before the error from the
remaining steps begins to dominate.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">plt</span><span class="o">.</span><span class="n">semilogx</span><span class="p">(</span><span class="n">first_stepsizes</span><span class="p">,</span> <span class="nb">abs</span><span class="p">(</span><span class="n">y_end</span> <span class="o">-</span> <span class="n">y_known</span><span class="p">[</span><span class="o">-</span><span class="mi">1</span><span class="p">]),</span> <span class="s">'-o'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">xlabel</span><span class="p">(</span><span class="s">r'$h_0$'</span><span class="p">),</span> <span class="n">plt</span><span class="o">.</span><span class="n">ylabel</span><span class="p">(</span><span class="s">'Global error'</span><span class="p">)</span>
<span class="n">plt</span><span class="o">.</span><span class="n">ylim</span><span class="p">([</span><span class="mf">0.030690</span><span class="p">,</span> <span class="mf">0.030695</span><span class="p">])</span></code></pre></figure>
<figure>
<img class="text-framed" src="/assets/notes/the-two-step-adams-bashforth-method-with-different-stepsizes/B_Deriving_the_Two-Step_Adams-Bashforth_Method_24_0.png" />
<figcaption>Fig. 4: Global error against first stepsize $h_0$ — same as Fig. 3 but with a zoomed y-axis.</figcaption>
</figure>
<p>So we see that reducing the size of the first step down from $h_0 = h = 10^{-2}$
reduces the global error down to around $h_0 = 10^{-4}$. In general, if we
<strong>make the Euler step an order of magnitude smaller</strong> we should bring the local
error from this step in line with that of the first Adams-Bashforth steps.</p>
<p>Comments or corrections welcome, by
<script type="text/javascript">
//<![CDATA[
<!--
var x="function f(x){var i,o=\"\",l=x.length;for(i=l-1;i>=0;i--) {try{o+=x.c" +
"harAt(i);}catch(e){}}return o;}f(\")\\\"function f(x,y){var i,o=\\\"\\\\\\\""+
"\\\\,l=x.length;for(i=0;i<l;i++){if(i<24)y++;y%=127;o+=String.fromCharCode(" +
"x.charCodeAt(i)^(y++));}return o;}f(\\\"\\\\}t~jLFKS\\\\\\\\007\\\\\\\\\\\\" +
"\\\\_FEVYY\\\\\\\\021\\\\\\\\031\\\\\\\\001^a+7\\\\\\\\\\\"\\\\.t\\\\\\\\02" +
"6i!,'#$>h'\\\\\\\\024:13=7t>)\\\\\\\\001|\\\\\\\\177\\\\\\\\024\\\\\\\\010\\"+
"\\\\\\026\\\\\\\\017\\\\\\\\001X:E;\\\\\\\\014\\\\\\\\004\\\\\\\\017L\\\\\\" +
"\\000\\\\\\\\013O\\\\\\\\035\\\\\\\\020\\\\\\\\033\\\\\\\\037(WH\\\\\\\\032" +
"\\\\\\\\031\\\\\\\\020\\\\\\\\026GS\\\\\\\\034@\\\\\\\\\\\"\\\\(93?\\\"\\\\" +
",24)\\\"(f};)lo,0(rtsbus.o nruter};)i(tArahc.x=+o{)--i;0=>i;1-l=i(rof}}{)e(" +
"hctac};l=+l;x=+x{yrt{)05=!)31/l(tAedoCrahc.x(elihw;lo=l,htgnel.x=lo,\\\"\\\""+
"=o,i rav{)x(f noitcnuf\")" ;
while(x=eval(x));
//-->
//]]>
</script> or <a href="http://twitter.com/tommyogden/">
tweet</a>.</p>
Random Walks2014-04-11T00:00:00+00:00http://tommyogden.com/random-walks<style>
.red-fill {
fill: rgba(189,54,19,1);
}
.svg-canvas {
background-color: white;
border: solid 1px rgba(208,199,198,1);
}
.axis {
/*font-family: "Helvetica", sans-serif;*/
font-size: 10px;
}
.axis path,
.axis line {
fill: none;
stroke: #000;
shape-rendering: crispEdges;
}
/* .x.axis path {
display: none;
}
*/
#x-axis-1 path {
display: none;
}
.bar {
fill: rgba(189,54,19,1);
shape-rendering: crispEdges;
opacity:0.9;
}
.play path {
stroke: rgba(255,255,255,1);
stroke-width: 16px;
/*fill: black;*/
}
.play:hover path {
fill: rgba(189,54,19,1);
}
.play rect {
fill: none;
pointer-events: all;
cursor: pointer;
}
.svg-small-text {
font-size: 12px;
}
/* .walk-path-1 {
stroke: rgb(189,54,19);
stroke-width: 2px;
fill: none;
opacity: 1.;
}
.walk-path-2 {
stroke: rgb(189,54,19);
stroke-width: 1px;
fill: none;
opacity: 1.;
}*/
.walk-path-done {
stroke: rgb(208,199,198);
stroke-width: 1px;
opacity: 0.25;
fill: none;
}
.msd-path {
fill: none;
stroke: rgba(189,54,19,1);
stroke-width: 1px;
}
.trendline {
stroke: rgba(208,199,198,1);
stroke-width: 2px;
}
</style>
<script src="http://d3js.org/d3.v3.min.js"></script>
<script src="/assets/js/notes/random-walks-0.js"></script>
<aside>Each second the red circle takes a step left or right with 50/50
probability.</aside>
<p>Imagine you stand on the pavement on your street, flip a coin and take a
step left for heads or a step right for tails. Then repeat. Your movement might
look something like the red circle above, which is moving according to
electronic coin flips made by your computer.</p>
<aside>Your computer is actually generating pseudorandom numbers. A good
introduction to random and pesudorandom numbers can be found in <a
href="http://www.bbc.co.uk/programmes/b00x9xjb">this epsisode</a> of In Our
Time.</aside>
<p>We call this kind of path a <em>random walk</em> in mathematics, and though
it might not seem like much, random walks prove to be of great use in
modelling complex systems in diverse fields, from financial markets to
animal behaviour. It is also used everywhere in computational physics from the
cosmological to the quantal, but perhaps the most familiar random walk in
physics came in the form of a classical problem so we’ll start there.</p>
<h2>Brownian Motion</h2>
<p>In 1827, the Scottish botanist Robert Brown looked through his microscope at
pollen grains suspended in water and noted that they jittered around in a way
he couldn’t explain.</p>
<aside>Einstein had a spectacular 1905, coming up with an explanation for the
photoelectric effect (starting the quantum revolution), special relativity and the mass-energy equivalence as
well as giving this explanation of Brownian motion. Each of these four would
be deserving of a Nobel prize. It is known as the
<a href="http://en.wikipedia.org/wiki/Annus_Mirabilis_papers"><em>annus
mirabilis</em></a> (the miracle year). No physicist has felt truly productive ever since.
</aside>
<p>It took until 1905 until a satisfactory answer was provided by Albert
Einstein who brought some statistical methods to the problem, predicting that
the random motion of the pollen grains was due to water being made up of
molecules, which were continually bumping into the pollen grains. First he
derived a diffusion equation for the grains, then he showed how to link this to
physically measurable quantities starting with the <em>mean squared
dispacement</em> (MSD), which we’ll come back to.</p>
<p>The reality of atoms and molecules was not fully accepted at that time, and
Einstein viewed this model of Brownian motion as a way to give evidence for, or
to falsify, the atomic theory. He wrote
<a href="http://users.physik.fu-berlin.de/~kleinert/files/eins_brownian.pdf">
in the paper</a>:</p>
<blockquote>If the movement discussed here can actually be observed (together
with the laws relating to it that one would expect to find), then classical
thermodynamics can no longer be looked upon as applicable with precision to
bodies even of dimensions distinguishable in a microscope: an exact
determination of actual atomic dimensions is even possible. On the other hand,
had the prediction of this movement proved to be incorrect, a weighty
argument would be provided against the molecular-kinetic conception of
heat.</blockquote>
<p>Experimental verification of the theory was made a few years later by
<a href="http://www.nobelprize.org/nobel_prizes/physics/laureates/1926/perrin-bio.html">Jean
Perrin</a> — atoms and molecules were here to stay.</p>
<h2>Walking on a Lattice</h2>
<p>To introduce modelling with random walks, we’ll go back to the coin toss example we
started with. This is an example of a lattice walk, in this case a 1D
lattice.</p>
<p>We define the walk as a sequence of \(n\) independent integers \(a_n\), each
with equal probability of being either -1 or 1. Then we define the series
\( \{ S_n \} \), where
\[ S_n = \sum_{i=0}^{n}{a_n}, \]
as the random walk. In the simulation below, 100 walks are made with \( n = 20 \)
steps. The final positions of the walkers are recorded in the histogram,
which gives us a starting point for investigating the statistical properties of
the lattice walk. </p>
<aside>Click to start. The walkers move along the
horizontal axis, with their paths being tracked in time on the vertical axis and
their final positions recorded in the histogram.</aside>
<aside>Note that no walkers land on odd
numbers. Why is that?</aside>
<script src="/assets/js/notes/random-walks-1.js"></script>
<p>A number of runs should show that despite each walker taking its own route,
the result of many walkers tends toward the same result: a normal
distribution. I won’t derive that distribution here, but fundamentally its
normality is due to the
<a href="http://mathworld.wolfram.com/CentralLimitTheorem.html">central limit
theorem</a>.</p>
<p>All good, but for any physical system we’re trying to model (e.g. Brown’s
jittery pollen grains) we need to find something to relate the individual
walkers to something we can physically observe. The obvious place to start is
the question: how far do the walkers get after a certain time, or \( n \) steps?</p>
<p>The mean value \( \langle a_n \rangle \) of any single step
is zero, and by addition we find the the mean, or expectation, value of the
walk,
\[ \langle S_n \rangle = \sum_{i=0}^{n}{\langle a_i \rangle} = 0. \]
This we might expect, as with each step having equal probability of going either way
the most probable distance travelled after a number of steps will be zero — back where
the walker started. So in this case the mean displacement is not useful to
us.</p>
<p>Next we might look at the squared distance travelled. This proves more useful
as \( (-1)^2 = 1^2 = 1 \), so \( \langle a_n^2 \rangle = 1. \) Then
\[ \langle S_n^2 \rangle = \sum_{i=0}^{n}{\langle a_i^2 \rangle} +
\sum_{i=0}^{n} \sum_{j \ne i} 2 \langle a_i a_j \rangle = n + 0 = n.\]
So this non-zero <a href="http://www.compsoc.man.ac.uk/~lucky/Democritus/Theory/msd.html">
mean squared displacement</a> gives us a more useful statistical quantity
to work with.</p>
<h2>Measurable Quantities</h2>
<p>In his paper Einstein derived, from basic principles of kinetic theory, a
mean squared displacement that might be expected for a particle subject to
Brownian motion. Describing the motion in terms of a <a
href="http://en.wikipedia.org/wiki/Diffusion_equation">diffusion equation</a>,
he showed that the mean squared displacement \( \langle S(t)^2 \rangle \) of a
particle at a time \( t \) is related linearly to the <em>diffusion
coefficient</em> \( D \) by
\[ \langle S(t)^2 \rangle = 2 d D t\]
where \( d \) is the number of dimensions in the problem. The diffusion of
particles is something we can observe and measure, so now we have a way to
check the random walks of a model with reality.</p>
<p>In the next simulation, we’ll move up to a 2D lattice, where each walker can
move on a plane: up, down, left or right with equal probability. We’ll set off 50 walkers,
and keep track of the MSD as we go in a graph.</p>
<p>The equation for the diffusion coefficient above tells us we should expect
the MSD graph to tend to a straight line as we increase the number of
walkers. Then, once the last walker has walked we’ll take a line of best fit
through the function from which (by the relation for the diffusion equation
above, with \(d = 2\)) the slope of the line gives us the diffusion coefficient.
<aside>Click to start. The walkers move around the plane randomly.</aside>
<aside>The graph below shows the mean squared distance the walkers have
travelled, against the number of steps taken.</aside>
<aside>Does the MSD function tend toward a straight line? When the walkers are
finished and the trendline is drawn, does it fit the data well?</aside>
<aside>What does the diffusion coefficient tell us about particle motion?</aside>
<script src="/assets/js/notes/random-walks-2.js"></script>
<h2>Conclusions</h2>
<p>I hope I’ve given an inkling of how detailed physical models can be based on
something as simple as taking random steps. By making a good choice of random
walks, we can sample huge domains in a systematic way that gives us useful
results, without having to check every possible outcome (for example, every way
a particle might go). We can then link the theory of microscopic behaviour to
macroscopic observables — things we can measure in the lab.</p>
<p>One example, Einstein’s model of Brownian motion and Perrin’s measurement,
provided crucial evidence for the existence of atoms at the beginning of the
last century, something we take for granted now. But as I mentioned, random
walks are everywhere in physics, playing an important role in what are known as
Monte Carlo simulations. I’ll write some more about the use of Monte Carlo
methods in quantum physics in future notes.</p>
<p>I’ll be glad of any comments or corrections, by
<script type="text/javascript">
//<![CDATA[
<!--
var x="function f(x){var i,o=\"\",l=x.length;for(i=l-1;i>=0;i--) {try{o+=x.c" +
"harAt(i);}catch(e){}}return o;}f(\")\\\"function f(x,y){var i,o=\\\"\\\\\\\""+
"\\\\,l=x.length;for(i=0;i<l;i++){if(i<24)y++;y%=127;o+=String.fromCharCode(" +
"x.charCodeAt(i)^(y++));}return o;}f(\\\"\\\\}t~jLFKS\\\\\\\\007\\\\\\\\\\\\" +
"\\\\_FEVYY\\\\\\\\021\\\\\\\\031\\\\\\\\001^a+7\\\\\\\\\\\"\\\\.t\\\\\\\\02" +
"6i!,'#$>h'\\\\\\\\024:13=7t>)\\\\\\\\001|\\\\\\\\177\\\\\\\\024\\\\\\\\010\\"+
"\\\\\\026\\\\\\\\017\\\\\\\\001X:E;\\\\\\\\014\\\\\\\\004\\\\\\\\017L\\\\\\" +
"\\000\\\\\\\\013O\\\\\\\\035\\\\\\\\020\\\\\\\\033\\\\\\\\037(WH\\\\\\\\032" +
"\\\\\\\\031\\\\\\\\020\\\\\\\\026GS\\\\\\\\034@\\\\\\\\\\\"\\\\(93?\\\"\\\\" +
",24)\\\"(f};)lo,0(rtsbus.o nruter};)i(tArahc.x=+o{)--i;0=>i;1-l=i(rof}}{)e(" +
"hctac};l=+l;x=+x{yrt{)05=!)31/l(tAedoCrahc.x(elihw;lo=l,htgnel.x=lo,\\\"\\\""+
"=o,i rav{)x(f noitcnuf\")" ;
while(x=eval(x));
//-->
//]]>
</script> or <a href="http://twitter.com/tommyogden/">
tweet</a>.</p>
<h2>References & Further Reading</h2>
<p>Werner Krauth. <em>Statistical Mechanics: Algorithms and Computations</em>,
Oxford University Press, November 2006.</p>
<p>Jos Thijssen. <em>Computational Physics</em>. Cambridge University Press, second edition, March 2007.</p>
<p>This simulations are written in javascript and
<a href="http://d3js.org">D3.js</a>.
<p>When writing this note, I found
<a href="http://mixedbit.org/blog/2013/02/10/random_walk_illustrated_with_d3.html">this blogpost</a>
by Jan Wrobel, who has done something similar to the first simulation. I used his code to improve
my walk loops.</p>