I don't disagree with you if you're talking about an appraisal of a coach's overall tenure but, for me, that isn't the relevant question here. Johnson's last two years are water under the bridge. The important question here is whether his team will be good enough for him to merit a fourth year. Next year the question will be whether the team does well enough for him to merit a fifth year.
Scott Drew began his tenure at Baylor with four consecutive losing seasons (yes, I know they were on probation when he began) but he hasn't had one since. In his fifth year, the team went to the NCAA tournament. Baylor would have been foolish to base a retain or fire decision after five years on Drew overall record instead of the team's improvement. I certainly don't expect Johnson to develop into a Scott Drew but I think the similar logic should be applied.