Hockey specialization might not account for it all but there are a lot of quality athletes getting pulled out of the mix due to hockey in this state. Between the Gophers and Wisconsin there are 31 MN kids and only 8 from Wisconsin.
Now granted, some of those guys aren't going to be big enough to play football, but a quick glance at the U roster shows 2 guys that are from MN and are 6'4" and 230 lbs. That is LB territory right there.
Combine that with the number of quality basketball players from MN as well and you can see where the football numbers could take a pretty good hit.
Another way of looking at it would be if you could only make your team out of instate recruits. You could build championship level hockey and basketball teams, the same can not be said for football.