This article presents an ILP system, called ILP-R, which has several properties that address the demands ofknowledge discovery in databases (KDD) quite nicely. The system uses Relief for its literal quality estimation, which can be as efficient as Information gain but more effective in detecting dependencies between literals. We introduce a weak language bias and exploit its properties for storing partial proofs in a mesh-like structure. We show the linear space bounds of this encoding scheme, with respect to the clause length. Finally, we present the first-order Bayesian classification framework, which can sometimes lead to significantly better classification and better noise resistance. It is also flexible enough to be used as an experimentation tool for revealing some underlying properties of the domain. We empirically tested our system on a set ofartificial and one real-world domain, both propositional and relational. We discuss the advantages and deficiencies ofour approach.