It sounds like you worry that this might be sort of crankish, but Marcus du Sautoy is a respected expositor, and a professor of mathematics at Oxford. In fact, according to Wikipedia, his title is "Simonyi Professor for the Public Understanding of Science", so it's hard to think of a person who, in title at least, is better qualified to offer this sort of exposition.
The key point is that the correlation function between pairs of zeros of the Reimann zeta function is the same as the pair correlation function of a random Hermitian operator (I presume this means the pair correlation function of the Eignenvalues of a random Hermitian operator).
This is what the OP is getting at when it talks about the distribution of energy levels of heavy nuclei, which are described by the Eigenvalues of a (more-or-less) random Hermitian operator because they contain enough particles that we can start doing statistical physics with them rather than worrying about each individual nucleon.
So this whole thing is less about some deep connection between quantum mechanics and primes than it is about a connection between primes and some of the mathematical machinery that quantum mechanics happens to use to describe the world in a particular approximation where the number of strongly-interacting particles is fairly large.
Physicists know a ridiculous amount of stuff about Hermitian matrices, which are a generalization of real symmetric matrices that have real Eigenvalues (this condition is important because it means the Eigenvalues can describe the outcome of experiments, which always come as real numbers.) So it is no surprise that a physicist of Dyson's stature would know something about these matrices that a mathematician might not.
