We present a general analytical formalism to determine the energy spectrum of a quantum particle in a cubic lattice subject to translationally invariant commensurate magnetic fluxes and in the presence of a general spaceindependent non-Abelian gauge potential. We first review and analyze the case of purely Abelian potentials, showing also that the so-called Hasegawa gauge yields a decomposition of the Hamiltonian into sub-matrices having minimal dimension. Explicit expressions for such matrices are derived, also for general anisotropic fluxes. Later on, we show that the introduction of a translational invariant non-Abelian coupling for multi-component spinors does not affect the dimension of the minimal Hamiltonian blocks, nor the dimension of the magnetic Brillouin zone. General formulas are presented for the U(2) case and explicit examples are investigated involving π and 2π/3 magnetic fluxes. Finally, we numerically study the effect of random flux perturbations.