Yeah, self driving cars may work in Arizona but how about rush hour Amsterdam or New York? In my country it might work for long haul trucks on highways in the near future. But I dont see a use case beyond that.
A lot of self-driving thought seems to go into deciding when to obey rigid rules and when to break them like everyone else.
We can't assume the pure chaos of Cairo or Delhi requires the same model. Self-driving might only need basic Roomba-level collision avoidance and basic pathfinding for a place where the humans navigate with the expectation that every object, moving or not, is a collision waiting to happen.
Think about how much of rules-based driving is about avoiding hitting people who aren't following the rules. If you start off assuming there are no rules, the model is simpler: everything the sensors pick up could come flying at you at any moment, so plan accordingly, and stop or slow down if you can't find a path that avoids an impact.
Hard to say what roadblocks will be there, 3 steps ahead. I suspect that if the US' easy cities become self-drivable, all cities will. But, maybe not.
In any case, I don't think we should be assuming infrastructure will remain the same. If self driving does replace driving, the infrastructure will change. Point problems (eg, computer can't handle these 43 tricky roads or intersections, those problems will be solvable be changing the road. It might just be adjusting so gns or minor parking rule changes.
There's a chicken-egg problem, which is why getting some of the way on existing infrastructure is smart. It also avoids scaring finance ministers, but ultimately infrastructure will meet driverless cars part way.
I suspect that "success" requires getting most of the way on existing infrastructure, especially to the degree that "self driving" is mostly for people who are well-off financially for a significant period.
That said, we redo intersections because they have a lot of accidents all the time. I'm not sure why intersections that self-driving vehicles have particular problems with would be any different. I can also imagine having some sort of RF "beacon" that gets used for construction sites and the like to flag drivers that they need to take over manual control in a mile or whatever.