Yes exactly, the speed of throwing up pre-built AMIs is tons faster than building from scratch each time you launch.
However, we're probably going to make a deployment script (we use fabric) that builds the AMI from scratch. Then, when we need to update the AMI, we just update the fabric script and run it to make the new AMI. That way if we ever need to make AMI's in a different region, we can just run the fabric script in that region.
I use a similar process to this in my infrastructure - I use Puppet to configure instances, then a Fabric command to create new AMIs based on these Puppet configurations.
This gives the best of both worlds - version controlled configuration files, and an automated process for making new AMIs. Instances also boot quickly, as nearly all of the configuration is baked-in to the image.
Using EC2 tags gives you even more room for automation here.
If you used puppet (or chef etc) instead of fabric for config, you could have the best of both worlds - it should be much faster to run puppet on existing instances to keep them up to date, rather than failover all your instances to new AMIs every time. But by still creating those AMIs, you'll have them ready when you need to launch additional instances.
However, we're probably going to make a deployment script (we use fabric) that builds the AMI from scratch. Then, when we need to update the AMI, we just update the fabric script and run it to make the new AMI. That way if we ever need to make AMI's in a different region, we can just run the fabric script in that region.