In online advertising, it is essential to show appropriate ads to target users. However, this is a challenging process. Although conventional targeting methods yield successful results, they cannot effectively select different ads for all users. In this study, we explore collaborative filtering techniques on an online ad dataset. We propose a method of recommending different and effective ads to users. The proposed method, which is based on biclustering and ordered weighted average aggregation operators, can address situations such as the lack of implicit feedback on items. We present the results of an offline analysis of the proposed method together with those of collaborative filtering methods. It is shown that collaborative filtering methods are beneficial, and that the proposed method provides superior results, especially in systems where user navigation histories are well known.