2016MNRAS.457.4005W

We describe a new test of photometric redshift performance given a spectroscopic redshift sample. This test complements the traditional comparison of redshift differences by testing whether the probability density functions p(z) have the correct width. We test two photometric redshift codes, bpz and eazy, on each of two data sets and find that bpz is consistently overconfident (the p(z) are too narrow) while eazy produces approximately the correct level of confidence. We show that this is because eazy models the uncertainty in its spectral energy distribution templates, and that post-hoc smoothing of the bpzp(z) provides a reasonable substitute for detailed modelling of template uncertainties. Either remedy still leaves a small surplus of galaxies with spectroscopic redshift very far from the peaks. Thus, better modelling of low-probability tails will be needed for high-precision work such as dark energy constraints with the Large Synoptic Survey Telescope and other large surveys.