Machine Learning is one of the frequently studied issues in the last decade. The major part of these research area is related with classification. In this study, we suggest a novel Lorentzian Distance Classifier for Multiple Features (LDCMF) method. The proposed classifier is based on the special metric of the Lorentzian space and adapted to more than two features. In order to improve the performance of Lorentzian Distance Classifier (LDC), a new Feature Selection in Lorentzian Space (FSLS) method is improved. The FSLS method selects the significant feature pair subsets by discriminative criterion which is rebuilt according to the Lorentzian metric. Also, in this study, a data compression (pre-processing) step is used that makes data suitable in Lorentzian space. Furthermore, the covariance matrix calculation in Lorentzian space is defined. The performance of the proposed classifier is tested through public GESTURE, SEEDS, TELESCOPE, WINE and WISCONSIN data sets. The experimental results show that the proposed LDCMF classifier is superior to other classical classifiers.