I recently came across a paper Flow Matching(FM) from ICLR paper and noticed a strong similarity between flow matching and diffusion models. It seems that the authors were strongly motivated by the derivation of diffusion models as well. I will talk about this paper from the following aspects:
Flow Matching aims to solve a common issue with neural ODEs, which need to be simulated multiple times in a single forward pass, therefore you don’t need backpropagation through time(BPTT) like RNN. This can significantly hinder the scalability(higher dimensions) of training for these ODEs.
Furthermore, since the flow is parameterized by an ODE, we can use efficient ODE solvers during inference, rather than having to solve an SDE as is required in diffusion models.
The general idea is to learn a target probability path that can align well with the vector field, those vector fields are quite similar to the score direction in score matching.