At most companies your direct manager can't fire you either. They simply write reviews and escalate problems to upper management/executives/HR, who then decide whether to act on it or not.