2024MNRAS.532..920H

Potential contamination from low/intermediate-redshift galaxies, such as objects with a prominent Balmer break, affects the photometric selection of high-redshift galaxies through identification of a Lyman break. Traditionally, contamination is estimated from spectroscopic follow-up and/or simulations. Here, we introduce a novel approach to estimating contamination for Lyman-break galaxy (LBG) samples based on measuring spatial correlation with the parent population of lower redshift interlopers. We propose two conceptual approaches applicable to different survey strategies: a single large contiguous field and a survey consisting of multiple independent lines of sight. For a large single field, we compute the cross-correlation function between galaxies at redshift $z \sim 6$ and intermediate-redshift galaxies at $z \sim 1.3$. We apply the method to the CANDELS GOODS-S and XDF surveys and compare the measurement with simulated mock observations, finding that the contamination level in both cases is not measurable and lies below 5.5 per cent (at 90 per cent confidence). For random-pointing multiple field surveys, we measure instead the number count correlation between high-redshift galaxies and interlopers, as a two-point correlation analysis is not generally feasible. We show an application to the LBG samples at redshift $z \sim 8$ and the possible interloper population at $z \sim 2$ in the Brightest of Reionizing Galaxies (BoRG) survey. By comparing the Pearson correlation coefficient with the result from Monte Carlo simulations, we estimate a contamination fraction of $62^{+13}_{-39} \, \mathrm{ per}\, \mathrm{ cent}$, consistent with previous estimates in the literature. These results validate the proposed approach and demonstrate its utility as an independent check of contamination in photometrically selected samples of high-redshift galaxies.