I, and quite a few colleagues, find it really good for ideation, and not too shabby for finished work. David Hockney's recent exhibition at the Royal Academy featured iPads heavily. Anecdotal, perhaps. My point was that to say that iPads are just for consumption is wrong.
So much is dependent on the app. It's hard to discuss the iPad without tacitly including the entire associated software ecosystem (without really being conscious of the scope of that oversight).
I agree with your main point, and I have no doubt that as the medium continues to expand, the quality of the software and interaction will also grow.
Just curious, what visual design apps are you using / would you recommend?
Paper from FiftyThree[1] is very good. Omnigraffle is a useful tool to, but pricey. Photoshop for ipad looks promising too. Matt Gemmell recently wrote a blog post about using the iPad from a UX designers point of view[2].