Mentioning gender isn't inherently exclusive, like using "gentleman" to refer to a specific man.

The term "gentlemen's agreement" is exclusive because it's a relic from a time when men were the only people who would conceivably enter into such an agreement. Even if it's "just an expression", it has a lot of assumptions tied up in it, and there are many, many alternate ways to say the same thing these days without those connotations.

If you take it literally (not "just an expression"), then it's an agreement where only men are allowed to participate, and hopefully it's obvious that that's exclusive. Good terminology should ideally be understandable to someone who hasn't seen the term before, so by that metric, "gentlemen's agreement" is just not a good term and doesn't seem worth keeping in the language.

Thank you for posting what I found to be an unbiased and helpful response, alangpierce.

I know it's easy for equality to feel like oppression to people who've never known life without whatever priviledges they may enjoy (race, gender, looks, height, etc.). To combat that, I flip it and see if I feel the same way.

In this case, if "gentlewoman's agreement" were the dominant idiom, I'd probably feel excluded no matter how many women assured me that the phrase wasn't biased.