> which would mean I got to go first since I stopped first
Careful with that. No one has the right-of-way at four-way stops in some (most? all?) states. Whoever enters the intersection first gets the right-of-way. Same reason you have the right-of-way to finish a left turn after your light turns red, if you were already in the intersection waiting for cross-traffic to clear/stop.
Be careful with reliance on what many would consider to be a fairly obscure right-of-way rule. More commonly if it's not clear someone just waves someone else through. It's like depending on people knowing obscure operator precedence :-)
Following my rule, you would always either yield or verify that they have yielded. Seeing a wave can be a good verification.
The annoying thing from people not knowing the rule is the “Mississippi standoff” with two cars at a 4 way stop sign waving at each other and not moving. :)
Careful with that. No one has the right-of-way at four-way stops in some (most? all?) states. Whoever enters the intersection first gets the right-of-way. Same reason you have the right-of-way to finish a left turn after your light turns red, if you were already in the intersection waiting for cross-traffic to clear/stop.