Using trades and quotes data from the Paris stock market, we show that the random walk nature of traded prices results from a very delicate interplay between two opposite tendencies: long-range correlated market orders that lead to super-diffusion (or persistence), and mean reverting limit orders that lead to sub-diffusion (or anti-persistence). We define and study a model where the price, at any instant, is the result of the impact of all past trades, mediated by a non-constant 'propagator' in time that describes the response of the market to a single trade. Within this model, the market is shown to be, in a precise sense, at a critical point, where the price is purely diffusive and the average response function almost constant. We find empirically, and discuss theoretically, a fluctuation-response relation. We also discuss the fraction of truly informed market orders, that correctly anticipate short-term moves, and find that it is quite small.