I agree, especially in regards to how he's toughened them up overall. but the Bollman hire is troubling. and he's made some dumb coaching decisions (I assume the fake FGs and some of the QB decisions are his)
if he hires some schlub to replace him, then yes.
if he gets rid of Bollman, and gets a decent OC, they can probably continue to be a 7+ win team even with a weaker defense. if he hires a Greg Robinson and the defense slumps to a RichRod one, then he's clearly out of touch and probably should go.