Inhomogeneous interior Neumann problem

I try to solve an interior inhomogeneous Neumann problem for the so-called Stokes-Joukowsky potential. This (velocity) potential describes the flow inside a completely filled container under rotational excitation.
I am aware of the fact that this is a pure Neumann problem, the solution will not be unique and will only exist if the right hand side is in the range space of the discrete operator that is constructed.

My current approach is based on the direct formulation that leads to a Fredholm equation of the second kind.
The domain and the right hand side are geometrically symmetric and I therefore expect a geometrically symmetric solution as well. However, it is not.

I would really appreciate advise on the best approach to solve this kind of interior Neumann problem: which formulation to use and how to solve/regularize the resulting linear system.

Thanks in advance,