"If people want to pursue accountability and the public has a strong opinion, we are willing to step down."