Speckle Transformer: direct classification through scattering media with limited information

Retrieving high-fidelity images from optical speckles remains challenging, especially when the information in speckles is severely insufficient. To address classification through scattering media under such constraints, we propose Speckle Transformer, a vision-transformer-based model that directly classifies objects using raw speckle patterns without intermediate image retrieval. By leveraging inherent features within speckles to extract discriminative features, our approach achieves nearly 90% accuracy for classifying speckles encoded with different images, outperforming traditional retrieval-classification pipelines by up to five times, even with extreme information sparsity (i.e., 1/1024 speckle regions of interest). In addition, we quantify speckle information capacity via information entropy analysis, demonstrating that classification accuracy correlates strongly with the information entropy of speckle autocorrelation. We not only overcome limitations of conventional methods but also establish a paradigm for real-time classification in scattering environments with constrained data.