In this essay, I address the question of whether the indisputable progress being made by the neurosciences poses a genuine threat to the language game of responsible agency. I begin by situating free will as an ineliminable component of our practices of attributing responsibility and holding one another accountable, illustrating this via a discussion of legal discourse regarding the attribution of responsibility for criminal acts. I then turn to the practical limits on agents' scientific self-objectivation, limits that turn out to be mirrored philosophically in the conceptual problems that plague reductionist strategies. Having shown that free will is rooted in unavoidable performative presuppositions belonging to agents' participant perspective, I then take up the difficult issue of how to reconcile an epistemic dualism of participant and observer perspectives with the assumption of ontological monism. I critically review a range of proposed physicalist solutions, including non-reductionist and (standard) compatibilist approaches. An underlying problem with scientistic, physicalist approaches is the methodological fiction of an exclusive 'view from nowhere' which relies on the problematic move of disengaging the objectivating perspective of the scientific observer from the investigators' participant perspective of those engaged in scientific practice. Since there is no way of getting around the requisite complementarity of both the observer's encounter with the objective world and the participant's involvement in shared lifeworld practices, the remaining option is to take an epistemological turn. But even the recognition that science is ultimately constituted from within the lifeworld still leaves us with the question as to how the human mind can understand itself as the product of natural evolution. I conclude with some tentative suggestions as to how this difficult question might be addressed.