Milling is a machining method used extensively nowadays in the manufacturing industry. In the milling process, one of the most important parameters affecting the machining performance is cutting tool geometry. In general, the effects of helix and rake angles, which are among the most important geometric parameters, on tool performance, are significant. In this study, the cutting force performance of wave form end mills that have a limited investigation both in the industry and academia is analyzed. The essential geometric factors in wave form end mills are designed with variable helix and rake angle along the helical cutting edges. In this work, the cutting force performance was experimentally and numerically compared with wave form tools, which are designed for reducing cutting forces and standard tools in terms of cutting forces on gray cast iron (GG25) according to ISO 8688-2 standard. The experimental results showed that an increase in the cutting force performance of wave form tools up to 11.4% compared to standard tools. Wave form tools with different geometric forms were compared with the validated finite element method (FEM) simulations based on experimental studies. As a result of FEM analysis, lower cutting forces (14.42%, 11.45%, 8.49%) obtained by 8-0.250, 6-0.450, 8-0.350 (wavelength-wave amplitude, mm) wave form tools compared to the standard tool. In addition, increment in cutting forces (- 6.37%, - 6.06%, - 5.92%) obtained by 8-0.150, 4-0.350, 8-0.650 tools.