The German armed forces, the Wehrmacht, were not directly associated with the Nazi party in WWII. Think of them as being like our current armed forces - not democrat or republican. There was a lot of dislike among the armed forces toward Hitler, up to and including assassination attempts. Rommel himself died via 'forced suicide' as he was charged with treason. The Nazi's did have their own fighting force too, the SS and although tactically they fought along side Germany's regular armed forces, those regular armed forces were not Nazi's. This is to this day a common mistake - to refer to Germany's WWII army, navy, etc. as "nazis".
Also, I am seeing a lot of comments about Hitler getting so many people to go along with his beliefs. A lot of this "agreement" was part of a sophisticated system of fear, forced nationalism, and all the other things that go along with an authoritarian dictatorship. People were literally afraid to pick up loose change in the streets for fear of it being watched and outed as not loyal to the cause. Yes, there were enthusiastic followers too but in general the "hitler youth" and things like it were mostly filled with people just trying to stay out of trouble.