Hierarchical fuzzy systems are proposed to handle the curse of dimensionality problem sourced from the use of single fuzzy inference systems with a large number of input parameters. While they are being used in various research problems, each of them is based on a constant hierarchic structure. In this study, this strategy is criticized because it is argued that using a constant hierarchic structure does not guarantee to obtain the most accurate solution for the problem. To observe the effects of structural differences on the prediction performance, experiments are performed on two logical gates by not only utilizing different structures but also different defuzzifiers. In the findings of the experiments, it is proved that the structural variations directly affect the systems' output, and this differentiation cannot be overcome by changing the defuzzifiers. In addition none of the utilized structures can provide the outputs of equivalent single system. It can be concluded that while applying the hierarchical fuzzy systems on any problem, different structures should be considered to find out the most accurate one that can be constantly utilized for that problem.