We present new determinations of the rest-UV luminosity functions (LFs) at z = 2-9 to extremely low luminosities (>-14 mag) from a sample of >2500 lensed galaxies found behind the Hubble Frontier Fields (HFF) clusters. For the first time, we present faint-end slope results from lensed samples that are fully consistent with blank-field results over the redshift range z = 2-9, while reaching to much lower luminosities than possible from the blank-field studies. Combining the deep lensed sample with the large blank-field samples allows us to set tight constraints on the faint-end slope alpha of the z = 2-9 UV LFs and its evolution. We find a smooth flattening in alpha from -2.28 +/- 0.10 (z = 9) to -1.53 +/- 0.03 (z = 2) with cosmic time (d alpha/dz = -0.11 +/- 0.01), fully consistent with dark matter halo buildup. We utilize these new results to present new measurements of the evolution in the UV luminosity density rho ( UV ) brighter than -13 mag from z similar to 9 to z similar to 2. Accounting for the star formation rate (SFR) densities to faint luminosities implied by our LF results, we find that unobscured star formation dominates the SFR density at z greater than or similar to 4, with obscured star formation dominant thereafter. Having shown we can quantify the faint-end slope alpha of the LF accurately with our lensed HFF samples, we also quantify the apparent curvature in the shape of the UV LF through a curvature parameter delta. The constraints on the curvature delta strongly rule out the presence of a turn-over brighter than -13.1 mag at z similar to 3, -14.3 mag at z similar to 6, and -15.5 mag at all other redshifts between z similar to 9 and z similar to 2.