We report three PET experiments that examine the neural substrates of the conceptual components of action retrieval. In all three experiments, subjects made action or screen-size decisions to familiar objects presented either as pictures or written words (the names ofthe objects). In Experiment 1, a third task was included, requiring a decision on the real-life size of the stimuli. In Experiment 2, a third stimulus type was included, with action and size decisions also performed on pictures of meaningless novel objects. Finally, in Experiment 3, we changed the response mode from a button press to a more explicit movement made with a ''manipulandum''. Based on neuropsychological findings, we predicted that when action responses were made to pictures of familiar or novel objects, relative to words, there would be less activation in semantic regions but greater activation in visual, motor, and perhaps parietal cortices. We found that, action relative to screen-size decisions on both pictures and words activated the left hemisphere temporo-frontal semantic system with activation in the left posterior middle temporal cortex specific to action retrieval (Experiment 1). In addition, action retrieval elicited more activation for (1) words than pictures in areas associated with semantics; and (2) novel objects than words or familiar objects in areas associated with pre-semantic object processing. These results are discussed in the context of semantic and visual routes to action retrieval.