I don't have facts to back up my opinion - It is formed by memories of articles I've read.
I believe CouchDB is a better choice for very large data sets because of its design.
+ CouchDB uses a Map Reduce design that I believe would scale better over very large data sets.
+ CouchDB always stores data in a consistent state on disk. You can literally pull the plug on the server at any time and the data will never be inconsistent.
MongoDB is geared for performance and is a great bridge between a relational database and a high-performance No-SQL database. But I don't recall that it's strength is handling large datasets.
Comparing map/reduce in Mongo and Couch is really apples and oranges. They are designe to do two different things. i.e data processing vs building views.
Mongo is designed from the ground up to deal with large datasets. Take a look at their sharding architecture.
I guess it depends what you consider "very large". If you're talking about multi-petabyte, then I'd probably use hdfs, but otherwise mongodb might fit. I hear craigslist uses mongodb to store their data since 1997, which is a fair amount of data I believe.
CouchDB is better'ish for larger datasets, but not for arbitrary scaling. MapReduce in CouchDB requires dumb full-scans if you're not just refreshing an existing view.
Arbitrarily large data is the exclusive domain of hadoop/hypertable/cassandra AFAIK atm.
To be fair CouchDB is very explicit that to get any sort of performance, everything must be a view. "Ad-hoc queries" (i.e. queries that are written on the fly instead of uploaded as a view) are clearly stated as "for development only".
Where CouchDB really falls flat is for write-heavy applications. The default configuration in CouchDB is to not reindex a view until it has been read. When a read occurs, any new data in a view that was added since the last read must be re-indexed by executing the map/reduce functions on that data. If you're writing frequently to CouchDB but not reading a lot (as in a data warehouse) the first query you run is going to be extremely slow, since it will need to run map/reduce on a lot of new data. CouchDB doesn't distribute work to multiple nodes like Hadoop, and I've found even simple reduce functions to slow down re-indexing by a factor of 10. I think CouchDB has settings now to update the index on commit, or you could always run a cron job to regularly query the view and force a reindex, but it's still going to be slow.
BigCouch (https://cloudant.com/#!/solutions/bigcouch) might be a potential choice for data warehousing, since it advertises full compatibility with the CouchDB API but offers distributed map/reduce like Hadoop/Hive/etc. I haven't used it though.
Couch is definitely a lot more honest about their limitations than mongo or riak, but my experiences make me hesitant to recommend it to anyone not intimately familiar with those limitations.
This is part of what we are addressing with Couchbase Server, an autosharding rebalancing Couch fronted by memcached. For K/V read and write we measure microsecond latency.
We are currently optimizing the views for cluster access, but the design goal is to offer at least the query performance CouchDB offers on small datasets, even on very large clusters.
i would not use it in future projects, myself, because my company is currently using it in several products in several different ways, and it has been nothing but headaches, problems, etc..
the theory behind the thing is great. in reality, its buggy and not fun to work with.
Regarding the heat issue you are having with your laptop: I was having similar problems with a different laptop, and I got an "XPad Slim" (http://www.xpad4laptop.com/) cooling pad to put under it. It weighs around a pound, has no fans, and does a pretty decent job of dispersing heat from the computer. No more burning legs!
I doubt I'll make anything. I honestly just posted the article to show how easy it is to upgrade the MBP hard drive. It's my first time messing with Amazon Affiliates, and blogspot has a nice plug-in that makes it really easy to go amazon-link-happy.
Ah, that explains a lot. I was wondering why quite a few blogspot articles have amazon links in them. The problem is that it becomes very hard to see the difference between borderline spam and genuine articles. That's also how you got those tor-x links in there, I actually figured that was proof positive it was a spam article, because who would go out of their way to link an affiliate code to a bunch of tools they used.
Interesting. I didn't realize amazon affiliate links carried such a negative connotation. I'll definitely keep that in mind for future posts and remove a few of the affiliate links from this post.
Last year a friend taught me about the bash trick "CTRL-R" <start typing 'ssh' or some other previously run command> on the command line for reverse history searching, and it is an amazing time saver. It acts as a great alternative to #8, "Find the last command that begins with “whatever,” but avoid running it"
How do you reverse the search? <S-C-R> didn't do it for me. Either way this could be a good substitute for what I do now, tediously grepping .zsh/bash_history...
If by reverse you mean going forward instead of backwards, that should be done with Ctrl-S; it does not work in Bash though, since Ctrl-S locks the scrolling of the terminal. I believe this behavior can be overridden, through some .xinputrc settings perhaps, but I haven't still found enough motivation to look it up!
I learned the basics of Haskell by doing some problems from http://projecteuler.net. It's probably not as exciting as actually building something using the language, but it's definitely worth checking out.
The maintainers of the contraptor project have done some pretty neat demonstrations with an Arduino. I was particularly impressed with this X-Y plotter/drilling assembly: http://www.contraptor.org/fast-drilling-contraption
Yeah the title is definitely misleading. The word "hacking" isn't even mentioned in the article besides in the title. Replace "hacking contest" with "programming competitions" and "NSA" with "NSA sponsored" and you have a more accurate title.
