Generation process of diffusion-based models can be formulated as ODE or SDE, so the quality of sampled images is highly related to truncation error from numerical solutions.
In this paper, using Richardson extrapolation, the author reduces the error order by linearly combining two estimates: one from k steps and one from a single step. That is, this method focuses on improving sampling estimation, not noise estimation.
With the same number of function evaluations, extrapolated sampling shows lower FID score and higher CLIP score.
One issue is that model evaluation can cause GPU–CPU bottleneck, so sampling time can increase a bit. But this problem can be reduced by saving previous results and reusing them.
Generation process of diffusion-based models can be formulated as ODE or SDE, so the quality of sampled images is highly related to truncation error from numerical solutions.
In this paper, using Richardson extrapolation, the author reduces the error order by linearly combining two estimates: one from k steps and one from a single step. That is, this method focuses on improving sampling estimation, not noise estimation.
With the same number of function evaluations, extrapolated sampling shows lower FID score and higher CLIP score.
One issue is that model evaluation can cause GPU–CPU bottleneck, so sampling time can increase a bit. But this problem can be reduced by saving previous results and reusing them.