The effect of background layout on visual search performance, and more specifically on the tendency to refixate previously inspected locations and objects, was investigated. Older and younger adults performed a search task in which a background layout or landmark was present or absent in a gaze contingent visual search paradigm. Regardless of age, participants demonstrated fewer refixations when landmarks were present, with older adults showing a larger landmark advantage. This visual search advantage did not come at the cost of saccadic latency. Furthermore, the visual search performance advantage obtained in the presence of a background layout or landmark was observed both for individuals with small and large memory spans.