Heat transfer mechanisms and their thermal performances need to be comprehensively studied in order to optimize efficiency and minimize energy losses. Different nanoparticles in the base fluid are investigated to upgrade the thermal performance of heat exchangers. In this numerical study, a finned shell and tube heat exchanger has been designed and different volume concentrations of nanofluid were tested to determine the effect of utilizing nanofluid on heat transfer. Fe2O3/water nanofluids with volume concentration of 1%, 1.5% and 2% were utilized as heat transfer fluid in the heat exchanger and the obtained results were compared with pure water. ANSYS Fluent software as a CFD method was employed in order to simulate the mentioned problem. Numerical simulation results indicated the successful utilization of nanofluid in the heat exchanger. Also, increasing the ratio of Fe2O3 nanoparticles caused more increment in thermal energy without important pressure drop. Moreover, it was revealed that the highest heat transfer rate enhancement of 19.1% can be obtained by using nanofluid Fe2O3/water with volume fraction of 2%.