Staying on Rails 2.3.x is certainly fine for existing apps, but starting a new app on 2.3 is a very bad idea. You're setting yourself up for a lot of headache for the inevitable migration. Let alone depriving yourself of all the wonders of Rails 3.

That being said, nobody is forcing you to jump on a release candidate. Just use the latest released, stable version. In this case Rails 3.0.x.