A data point: `scrapy bench` is about 1.5x - 2x slower in Python 3.5 than in Python 2.7. This command starts a simple HTTP server which serves HTML pages linked in arbitrary depth, and crawls it using Scrapy. Profiler shows URL parsing is much slower in Python 3 for some reason, I'm not sure why. I haven't digged further.
Python 3 code also have to encode/decode strings more often, but the overhead was not significant for `scrapy bench`.
I think you should google the python.org bug tracker with the function names that seems to exhibit a performance regression according to your profiling session. If don't find any existing issue, please open a new one with a minimalist reproduction script that demonstrates the performance regression.
Yeah, totally agree; I did it for io.BytesIO vs cStringIO performance issue (https://news.ycombinator.com/item?id=11037213), and Python core devs fixed it very fast even though the fix was far from trivial. You have to wait a year or so before next release to use the fix, but that's expected.
For URL parsing it was just a quick profiling session. Also, stdlib URL parsing was a bottleneck even in Python 2. It takes time to benchmark it properly and submit a meaningful issue to bug tracker or explain the problem in python-dev. It is in my todo list, but you only have so much time :)
Python 3 code also have to encode/decode strings more often, but the overhead was not significant for `scrapy bench`.