XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts

Ben Heritage; Luca Resti; Monica Villanueva Aylagas; Timothy Mehlenbacher; Konrad Tollmar; James Walker

XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts

Journal article

Ben Heritage, Luca Resti, Monica Villanueva Aylagas, Timothy Mehlenbacher, Konrad Tollmar, James Walker
arXiv.org (submitted to ICASSP 2027), 2026

Semantic Scholar ArXiv DBLP DOI

Cite

APA Click to copy
Heritage, B., Resti, L., Aylagas, M. V., Mehlenbacher, T., Tollmar, K., & Walker, J. (2026). XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts. ArXiv.org (Submitted to ICASSP 2027).

Chicago/Turabian Click to copy
Heritage, Ben, Luca Resti, Monica Villanueva Aylagas, Timothy Mehlenbacher, Konrad Tollmar, and James Walker. “XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts.” arXiv.org (submitted to ICASSP 2027) (2026).

MLA Click to copy
Heritage, Ben, et al. “XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts.” ArXiv.org (Submitted to ICASSP 2027), 2026.

BibTeX Click to copy

@article{ben2026a,
  title = {XSQ-AST: An Explainable Audio Spectrogram Transformer Framework for Localising Synthetic Speech Artifacts},
  year = {2026},
  journal = {arXiv.org (submitted to ICASSP 2027)},
  author = {Heritage, Ben and Resti, Luca and Aylagas, Monica Villanueva and Mehlenbacher, Timothy and Tollmar, Konrad and Walker, James}
}

Abstract

Localising artifacts in synthetic speech remains challenging, as most evaluation methods yield only global quality scores. This paper presents XSQ-AST, a framework that combines the SQ-AST speech quality model with WhisperX phoneme alignment and multiple saliency methods to produce temporally localised artifact diagnostics without model retraining. Saliency maps are projected onto continuous distributions via kernel density estimation and onto phoneme boundaries via phoneme-discretised saliency maps. A 40-participant listening test validated the framework across five perceptual dimensions. Attention Rollout, Attention Flow and an adapted GradCAM produced temporal distributions that correlated with listener highlights, with different methods best suited to different artifact types. An AUC-ROC analysis confirmed discrimination above chance.