An experimental study was carried out to understand the electrical percolation thresholds of different carbon-based nano- and micro-scale materials in cementitious composites. Multi-walled carbon nanotubes (CNTs), graphene nanoplatelets (GNPs) and carbon black (CB) were selected as the nano-scale materials, while 6 and 12 mm long carbon fibers (CF6 and CF12) were used as the micro-scale carbon-based materials. After determining the percolation thresholds of different electrical conductive materials, mechanical properties and piezoresistive properties of specimens produced with the abovementioned conductive materials at percolation threshold were investigated under uniaxial compressive loading. Results demonstrate that regardless of initial curing age, the percolation thresholds of CNT, GNP, CB and CFs in ECC mortar specimens were around 0.55%, 2.00%, 2.00% and 1.00%, respectively. Including different carbon-based conductive materials did not harm compressive strength results; on the contrary, it improved overall values. All cementitious composites produced with carbon-based materials, with the exception of the control mixtures, exhibited piezoresistive behavior under compression, which is crucial for sensing capability. It is believed that incorporating the sensing attribute into cementitious composites will enhance benefits for sustainable civil infrastructures.