Comments on: B is for Billion http://blog.wordnik.com/b-is-for-billion Connecting people with meaning. Wed, 09 May 2012 19:58:41 +0000 hourly 1 http://wordpress.org/?v=3.3.1 By: tonytam http://blog.wordnik.com/b-is-for-billion/comment-page-1#comment-1047 tonytam Mon, 12 Jul 2010 21:01:40 +0000 http://blog.wordnik.com/?p=528#comment-1047 Hi Alex, we use MongoDB for a number of different data structures. One structure is not sharded and has 800M records in it. Another (the biggest) is sharded at the application layer. Distribution across the shards is pretty good but there are a couple hot spots of ~100M records. The sharding was primarily to improve bulk insert performance, not seek performance. We don't use MR within MongoDB. It is powerful but we are treating the system as storage & (deep object) retrieval. We use Lucene for analytics. We'll be using auto-sharding after 1.6 comes out and we get enough testing on it. Hi Alex, we use MongoDB for a number of different data structures. One structure is not sharded and has 800M records in it. Another (the biggest) is sharded at the application layer. Distribution across the shards is pretty good but there are a couple hot spots of ~100M records. The sharding was primarily to improve bulk insert performance, not seek performance.

We don’t use MR within MongoDB. It is powerful but we are treating the system as storage & (deep object) retrieval. We use Lucene for analytics.

We’ll be using auto-sharding after 1.6 comes out and we get enough testing on it.

]]> By: Alex S http://blog.wordnik.com/b-is-for-billion/comment-page-1#comment-1044 Alex S Mon, 12 Jul 2010 09:59:30 +0000 http://blog.wordnik.com/?p=528#comment-1044 I'd be very interested to know a couple of things about your MongoDB setup: 1. Are you using sharding at all? If so, how many records can you handle per shard? 2. Are you doing any map/reduce queries across all of those 9bn records? How do you deal with MongoDB's poor map/reduce performance? I’d be very interested to know a couple of things about your MongoDB setup:

1. Are you using sharding at all? If so, how many records can you handle per shard?
2. Are you doing any map/reduce queries across all of those 9bn records? How do you deal with MongoDB’s poor map/reduce performance?

]]> By: tonytam http://blog.wordnik.com/b-is-for-billion/comment-page-1#comment-1032 tonytam Fri, 09 Jul 2010 22:58:59 +0000 http://blog.wordnik.com/?p=528#comment-1032 Hi Marc, take a look here, this covers some of your question: http://www.slideshare.net/fehguy/migrating-from-mysql-to-mongodb-at-wordnik Hi Marc, take a look here, this covers some of your question:

http://www.slideshare.net/fehguy/migrating-from-mysql-to-mongodb-at-wordnik

]]> By: Marc http://blog.wordnik.com/b-is-for-billion/comment-page-1#comment-1031 Marc Fri, 09 Jul 2010 21:56:02 +0000 http://blog.wordnik.com/?p=528#comment-1031 More information about the setup (Servers, RAM, Replication, backup procedure, ...) would be awesome! More information about the setup (Servers, RAM, Replication, backup procedure, …) would be awesome!

]]>