Measurements of acoustic properties of sound absorbing materials in impedance tubes show poor reproducibility, which was demonstrated in round robin tests. The impedance tube measurements are standardized but lack precise definitions of the actual measurement setup, specimen preparation, and other factors that introduce uncertainty in practice. In this paper, machine learning models identify those factors that mostly affect the sound absorption coefficient from a large data set of more than 3000 absorption spectra measured in one impedance tube. The specimens are manufactured from one polyurethane foam, and different cutting technologies, different operators, different specimen diameters, different specimen thicknesses, and two different approaches to mount the specimens in the impedance tube are considered. Explainable machine learning techniques allow the identification and quantification of the most influential factors and, furthermore, the frequency ranges that are the most affected by the choice of these setup factors. The results indicate that besides the specimen thickness, also the operator affects the absorption coefficient by a directional and non-random relationship. Hence, it needs to be controlled carefully. The method proves to be a promising pathway for knowledge discovery from acoustic measurement data using explainability approaches for machine learning models.