Stay perpendicular to North/South and you follow a parallel.

None of these things is of course possible IRL.

Staying perpendicular to north/south requires turning, unless you’re on the equator.

Or listing (ie leaning); still it seems straight, YMMV.

Consider the degenerate case where you’re ten feet away from the North Pole. You’ll need to go in a tight circle. The situation at more moderate latitudes is the same, just less extreme. It only seems straight because we’re usually at moderate latitudes and the rate of turning is low.