Speculation? Of course. Malarkey? Absolutely not.
A guy who is focused on his current gig puts a quick end to public speculation about a lateral move unless he is trying to decide which situation best sets him up for the big job. Otherwise it's too damaging to recruiting and his reputation.
Rumors spouted every season while Tubby was here that he was leaving for another job. How many times did he come out and say he was staying?