We study the effect of a non-Abelian SU(2) gauge potential mimicking spin-orbit coupling on the topological semimetal induced by a magnetic field having π flux per plaquette and acting on fermions in a three-dimensional (3D) cubic lattice. The Abelian π -flux term gives rise to a spectrum characterized by Weyl points. The non-Abelian term is chosen to be gauge equivalent to both a 2D Rashba and a Dresselhaus spin-orbit coupling. As a result of the anisotropic nature of the coupling between spin and momentum and of the presence of a C4 rotation symmetry, when the non-Abelian part is turned on, the Weyl points assume a quadratic dispersion along two directions and constitute double monopoles for the Berry curvature. We examine the main features of this system both analytically and numerically, focusing on its gapless surface modes, the so-called Fermi arcs. We discuss the stability of the system under confining hard-wall and harmonic potentials, relevant for the implementation in ultracold atom settings, and the effect of rotation symmetry breaking.