We analyze a tight-binding model of ultracold fermions loaded in an optical square lattice and subjected to a synthetic non-Abelian gauge potential featuring both a magnetic field and a translationally invariant SU(2) term. We consider in particular the effect of broken time-reversal symmetry and its role in driving nontrivial topological phase transitions. By varying the spin-orbit coupling parameters, we find both a semimetal-insulator phase transition and a topological phase transition between insulating phases with different numbers of edge states. The spin is not a conserved quantity of the system, and the topological phase transitions can be detected by analyzing its polarization in time-of-flight images, providing a clear diagnostic for the characterization of the topological phases through the partial entanglement between spin and lattice degrees of freedom.