Polynomial values modulo primes on average and sharpness of the larger sieve

This paper is motivated by the following question in sieve theory. Given a subset X ⊂ [N] and α ∈ (0,1/2). Suppose that |X (mod p)| ≤ (α + o(1)) p for every prime p. How large can X be? On the one hand, we have the bound |X| ≪_α N^α from Gallagher’s larger sieve. On the other hand, we prove, assuming the truth of an inverse sieve conjecture, that the bound above can be improved (for example, to |X|≪ _αN^O(α²⁰¹⁴) for small α). The result follows from studying the average size of |X (mod p)| as p varies, when X = f (ℤ) ∩[N] is the value set of a polynomial f (x) ∈ ℤ[x].