We discuss the evolution of the interstellar medium of quiescent galaxies, currently emerging from recent analyses, with the help of a simple model based on well-established empirical relations such as the stellar mass functions and the main sequence of star formation. This model is meant to describe observed quantities without making specific assumptions on the nature of quenching processes, but relying on their observable consequences. We find that the high gas fractions seen or suggested at high redshift in quiescent galaxies, and their apparent mild evolution at early times, can be mostly attributed to a progenitor effect where recently quenched galaxies with similar to 10% gas fractions dominate the quiescent galaxy population until z similar to 1. In the same context, the much lower gas and dust fractions measured in local early-type galaxies are interpreted as the product of the steady depletion of their interstellar medium on a similar to 2 Gyr timescale, coupled with a higher fraction of more gas-exhaustive events.