I'm not drawing a hard line at all. You are actually doing that by basically saying that anyone that has been honored should continue to be so because it's debatable.
I'm willing to consider people on a case by case basis, but a monument or stadium name should be a really high bar to reach.
Higher bars than stadiums have been cleared. Ohio's state capital is named after Columbus. Do we begrudgingly live with that because it is simpler to change a stadium name than a city name?
So because one thing is named after an asshole, we can't change the names of other things named after assholes?
DJ didn't tell me why the DC football team name issue is different than this, but your defense can certainly be used to defend them and others with similar names.