IMO conference record doesn't and shouldn't make a difference. The overall profile is what matters, you can have a 8-10 record in the B1G and you would probably finish 12-6 or better in the SEC just because of the difference in the conference strength, so when you really think about it why should conference record make a huge difference?
The only people who say this are fans of the team about to finish their conference season with a losing record.