If your company is organized as a corporation (as opposed to a partnership), the CEO is accountable to a Board of Directors who, at least theoretically, have the ability to hire and fire the CEO and decide his or her salary. In most privately-held companies, the CEO selects a 'friendly' Board and the CEO herself may serve as the Chairman of the Board. In a startup with venture capital or other outside funding, the investors will demand seats on the board and if the company isn't performing well, will really put the screws on the CEO.
In public companies, the board is elected by the shareholders to protect their investments. The board is supposed to be independent from the company's management and underperforming CEOs should be summarily dumped. In practice, many public boards are 'interlocking' meaning that if you put me on your board, I'll put you on mine. Since the Enron scandal and the financial disasters of 2008, a lot more scrutiny is going into the composition of boards of public companies, and especially to the committees which are responsible for executive compensation. Whether this has had an effect on how good corporate management has become is something for you to decide.
