Recent developments in wireless communication systems in indoor environments require high data rates and high transmission qualities especially for multimedia applications in wireless local area network (WLAN) systems. The support of high data rate multiple-input multiple-output (MIMO) spatial-multiplexing communication in orthogonal frequency-division multiplexing WLAN systems conforming to IEEE802.11n standard requires the use of compact antennas with low correlation ports. In this paper, we analyse a MIMO array consisting of higher-order space-multimode-polarization diversity stacked circular microstrip patch uniform linear antenna arrays (SMP-SCP-ULAs) to exploit SMP diversity for use in WLAN systems. Using higher number of antenna ports in lower space and low cross-polar discrimination values between each transmit and receive antenna, SMP-SCP-ULA yields higher multiplexing gain and satisfies more restrictive size constraints in clustered MIMO systems. The performance gains of higher-order modal SMP-SCP-ULA are presented in terms of modal correlation, ergodic spectral efficiency, average bit error rate and compactness gain with respect to dipole and circular microstrip patch uniform linear arrays. Copyright (C) 2011 John Wiley & Sons, Ltd.