Structural damage and fatigue life of engineering materials are controlled by inherent defects and the nature, level, and location of applied external loads. This paper studies the effect of cyclic lateral loads on the pinless and conical-pin assisted friction stir spot welds of AA2219 alloy. The results reveal the presence of upward-flow induced hook defect and outstretching-flow induced microscopic toe-notch in the conical pin and pinless welds, respectively. Extruded flash root contributed to the fracture pattern of the conical pin welds while it did not influence the fracture of the pinless weld. Grain refinement, microhardness, and microscopic toe-notch affect the fracture mode of pinless weld whereas complex fracture morphology ensues in conical pin weld. A decrease in the cyclic lateral stress level improves the lateral fatigue lives of both weld categories. Lateral fatigue improvement of welds requires the minimization of inherent nugget defects and extruded flash.