The way in which public transport buses are driven has an influence in users’perception and satisfaction with the service. Bus driver’s behavior is usually obtained surveying passengers and/or using the mystery passenger method, not necessarily allowing for an objective and continuous evaluation. In this work, we introduce a novel methodology to automatically classify drivers’ behavior in a more consistent and objective manner, based on data from inertial measurement units, and machine learning techniques. By substituting human evaluators with automatic data collection and classification algorithms, we are able to reduce the subjectivity and cost of the current methodology, while increasing sample size. Our approach is based on three components: i) data capture using inertial measurement units (e.g. mobile devices), ii) carefully tuned classifiers that deal with sample imbalance problems, and iii) an interpretable scoring system. Results show that collected data captures several types of undesirable maneuvers, providing a rich information to the classification process. In terms of categorization performance, the evaluated classifiers, namely support vector machines, decision trees and k-NN, deliver high and consistent accuracy after the tuning process, even in the presence of a highly imbalanced sample. Finally, the proposed driver’s behavior score shows high discriminative power, effectively characterizing differences between drivers, and providing driver-tailored driving recommendations, that can be generated in specific spots, in order to improve passengers’ experience. The resulting methodology can be cost-effectively deployed at a large scale with good performance.