Deep diffractive cooperative optimization

Traditional imaging systems are designed for human perception, but the growing role of artificial intelligence (AI) in decision-making tasks raises the question of whether optical systems can be optimized specifically for AI consumption. This work introduces a cooperative optimization framework that fine-tunes a physical imaging system using a pretrained classifier. Unlike co-optimization approaches that require joint training of optical and AI models, this work decouples the two, leveraging pretrained classifiers to guide optical system optimization. Additionally, a gradient-based simulation-to-reality calibration technique (Sim2Real) is proposed to improve the fidelity of differentiable surrogate models, reducing discrepancies between the simulated and real-world optical system. The effectiveness of the approach is demonstrated by intentionally degrading an imaging system through defocus and using classification error to optimize the optics. Results show significant performance recovery and highlight the impact of Sim2Real calibration in bridging the simulation-to-reality gap, improving optimization outcomes.