Increase size of per-process request ID nonce

Change the per-process nonce part of the request ID from 8 characters to
10, and wrap the entire thing in a retry loop so you can never get an
"unlucky" panic. I know this will "never" happen in practice, but it
doesn't hurt to make sure we never, ever have any collisions, and never,
ever have any runtime panics.

It's also worth documenting the math ("math") I used to calculate the
numbers here.

web/middleware/request_id.go

@@ -18,19 +18,38 @@ const RequestIDKey = "reqID"
 var prefix string
 var reqid uint64
 
+/*
+A quick note on the statistics here: we're trying to calculate the chance that
+two randomly generated base62 prefixes will collide. We use the formula from
+http://en.wikipedia.org/wiki/Birthday_problem
+
+P[m, n] \approx 1 - e^{-m^2/2n}
+
+We ballpark an upper bound for $m$ by imagining (for whatever reason) a server
+that restarts every second over 10 years, for $m = 86400 * 365 * 10 = 315360000$
+
+For a $k$ character base-62 identifier, we have $n(k) = 62^k$
+
+Plugging this in, we find $P[m, n(10)] \approx 5.75%$, which is good enough for
+our purposes, and is surely more than anyone would ever need in practice -- a
+process that is rebooted a handful of times a day for a hundred years has less
+than a millionth of a percent chance of generating two colliding IDs.
+*/
+
 func init() {
 	hostname, err := os.Hostname()
 	if hostname == "" || err != nil {
 		hostname = "localhost"
 	}
 	var buf [12]byte
-	rand.Read(buf[:])
-	b64 := base64.StdEncoding.EncodeToString(buf[:])
-	// Strip out annoying characters. We have something like a billion to
-	// one chance of having enough from 12 bytes of entropy
-	b64 = strings.NewReplacer("+", "", "/", "").Replace(b64)
+	var b64 string
+	for len(b64) < 10 {
+		rand.Read(buf[:])
+		b64 = base64.StdEncoding.EncodeToString(buf[:])
+		b64 = strings.NewReplacer("+", "", "/", "").Replace(b64)
+	}
 
-	prefix = fmt.Sprintf("%s/%s", hostname, b64[0:8])
+	prefix = fmt.Sprintf("%s/%s", hostname, b64[0:10])
 }
 
 // RequestID is a middleware that injects a request ID into the context of each