Binary-single interactions play a crucial role in the evolution of dense stellar systems such as globular clusters. In addition, they are believed to drive black hole (BH) binary mergers in these systems. A subset of binary-single interactions are secular encounters, for which the third body approaches the binary on a relatively wide orbit, and such that it is justified to average the equations of motion over the binary's orbital phase. Previous works used first-order (FO) perturbation theory to compute the effects of such secular encounters on the binary. However, this approach can break down for highly eccentric binaries, which are important for BH binary mergers and gravitational wave sources. Here, we present an analytic computation using second-order perturbation techniques, valid to the quadrupole-order approximation. In our calculation, we take into account the instantaneous back reaction of the binary to the third body, and compute corrections to previous FO results. Using singly averaged and direct three-body integrations, we demonstrate the validity of our expressions. In particular, we show that the eccentricity change for highly eccentric binaries can reach a plateau, associated with a large inclination change, and can even reverse sign. These effects are not captured by previous FO results. We provide a simple script to conveniently evaluate our analytic expressions, including routines for numerical integration and verification.