Go talk to the guys at ClusterK.com for their balancer device. It will logicall auto-balance your spawn of spot instances across many AZs and make you very resilient. Tell them you want to prove out their balancer.
Do this, because intelligently following the cheapest spot price will save you 90% of the typical costs.
THis will make your 100K be able to support a ton more instances than you would otherwise.
Make sure that whatever you do though is not chatty between the nodes - if you have a ton of instances talking across zones the data transfer fee can be significant.
Make sure that you store large data on an EBS that an instance mounts to prevent large transfer fees between instances and S3.
All instance limits can be raised. The only hard limit in AWS is 100 S3 buckets per account.
Just put in a limit increase request via the console. Email gilleyt@amazon.com if you have issues.
Do this, because intelligently following the cheapest spot price will save you 90% of the typical costs.
THis will make your 100K be able to support a ton more instances than you would otherwise.
Make sure that whatever you do though is not chatty between the nodes - if you have a ton of instances talking across zones the data transfer fee can be significant.
Make sure that you store large data on an EBS that an instance mounts to prevent large transfer fees between instances and S3.
All instance limits can be raised. The only hard limit in AWS is 100 S3 buckets per account.
Just put in a limit increase request via the console. Email gilleyt@amazon.com if you have issues.