Signature of Ultrafast Formation and Annihilation of Polaronic States in a Layered Ferromagnet