Tech Pontificator

I have attempted to put together some guidance that can help in determining what sort of hardware makes sense for MongoDB in varying situations.  This is really an elaboration on publicly available information and an abbreviated version dashed off at 2:00 am by Dwight Merriman (who is far smarter than I, one of the original authors of MongoDB, and CEO of 10gen).

Philosophy

“The philosophy of MongoDB is to use commodity componentry to build clusters and to build them assuming that any single server can fail at any time. This is to say, a cloud-computing style of hardware is acceptable.” – Dwight Merriman

It seems almost too obvious to say, but the exact ratio of resources ideal for MongoDB will vary based upon the use case and performance requirements. Some applications, for example, which require the utmost read performance will wish to keep their entire data sets resident in memory. Other applications have high write loads and IOPS (Input Output Operations per Second) become more important. In general, however, total RAM and random disk IOPS are important considerations. Below, we shall examine in more detail the importance of the essential computing resources of CPU, RAM, Storage, Network, and IOPS.

General Rule on Server Size

While ratios of resources will vary, it is usually best to go with a server that is reasonably large so that there are not too many servers to administer, yet not so expensive that it is uneconomic from diminishing returns.  Commodity servers costing about $10,000 are common in mongo deployments.

CPU

MongoDB is optimized for 64 bit architecture at both the hardware and OS level. One should use a 64 bit Intel or AMD processor with a corresponding 64 bit OS.  NUMA is not very helpful in database applications, thus non-NUMA is fine.  NUMA machines work but NUMA should be disabled (interleaved) when used with MongoDB for best results.  For more information on NUMA see http://www.mongodb.org/display/DOCS/NUMA

CPU is not a common limiting resource for MongoDB.  One usage profile that can be the exception is where heavy usage of MongoDB’s map-reduce or aggregation framework will be taking place. This usage profile also raises an additional consideration for CPU of boxes running the mongos process.  The mongos process is responsible for coordinating client requests across sharded configurations.  Performance concerns for the mongos process are rarely considered since typically they are lightweight processes that are installed on application servers running the MongoDB client code.  Aggregation, whether from map-reduce, the aggregation framework, or the hadoop connector against a sharded MongoDB collection will drive CPU cycles to the boxes running the mongos instances.  Latency requirements and testing should be done to determine how much CPU should be allocated for mongos processes under these circumstances.

RAM

Since MongoDB makes extensive use of memory mapping the more you have the merrier you will be!  MongoDB will make much better use of memory for accelerating operations than a typical RDBMS.  It is very common to run MongoDB on servers configured with 64GB, 128GB, or even 256GB of memory.  Many customers, requiring the utmost query and read performance, will have enough memory across the cluster to keep their indexes and full data set in memory.  The beauty of MongoDB is that it allows you to scale this out across a cluster of machines by simply adding more shards as your dataset grows.  At a minimum you will want enough memory in each box to store indexes and in many use cases additional memory beyond this to store commonly accessed documents.

Network

Commodity gigabit ethernet usually works fine in practice.  Be sure to have switches that can handle a high backplane load from the cluster if the cluster is large. A very common mistake is to use gigabit ethernet but have a 100 mbps switch somewhere in the cluster throttling throughput. It is generally recommend that one uses only one ethernet port per server, with that port servicing both client requests as well as intra-cluster communications.

Storage

In the intensive read/query usage profile, system memory outweighs storage as the most important resource. Where the importance of storage and IOPS becomes more significant is in cases where you have heavy insert-update-delete (IUD) workloads. Within this broad category there are many specific usage patterns that will have still further implications on the storage technology used. So in talking about storage below, we are very much thinking in general terms.

While MongoDB will work well with SANs, internal storage works just fine. Often the best solution is simply whatever is most economic — i.e. commodity internal storage. The one caveat would be that if you are using conventional spinning disks (not SSDs), you may need a lot of them and/or fast drives to get to a decent random I/O capacity, as a single SATA drive might support only 100 random I/O’s per second. The flip side is that using local storage can avoid common problems with over subscribed SANs. Often times a SAN will have plenty of disk but not enough IO throughput. With local storage every time you add a shard to the cluster you are augmenting all your essential computing resources including disk and IO throughput.

For cases where the cost is acceptable, solid state disks (SSDs) are highly recommended. Even though SSDs are fast, RAM is still faster. Thus for the highest read performance possible, having enough RAM to contain the working set of data from the database is optimal. However, it is common to have a request rate that is easily met by the speed of random IO’s with SSDs, and SSD cost per byte is lower than RAM (and persistent too).

Let us consider the case where the economics or use case dictates just enough memory to contain the indexes and some subset of the working data set. A system with less RAM and SSDs will often outperform a system with more RAM and spinning disks. For example a system with SSD drives and 64GB RAM will often outperform a system with 128GB RAM and spinning disks. (Results will vary by use case of course.)

One helpful characteristic of SSDs is they can facilitate fast “preheat” of RAM on a hardware restart. On a restart a system’s RAM file system cache must be repopulated. On a box with 64GB RAM or more, this can take a considerable amount of time – for example six minutes at 100MB/sec, and much longer when the requests are random IO to spinning disks. More information on SSDs and MongoDB can be found here

A final consideration here is the usage of Fusion-IO which is yet another tier of performance above SSD. One way to help balance all these choices is to examine performance metrics on a per dollar basis where one assumes a sharded configuration that will allow one to incrementally add capacity as required.  For some Fusion IO MongoDB use cases go here: Pairing MongoDB with Fusion IO Flash Memory Solutions

Specific Configurations

Here are some usage profiles, corresponding use cases, and plausible hardware scenarios.

• Low transaction rates, high query and read.
  • Example application: Product catalog or personnel database.
  • 2 quad core CPUs, 128GB memory, RAID 10 spinning disk.
• High ingest rate, low read query
  • Example application: Auditing store
  • 2 dual core CPUs, 32GB memory, RAID 10 spinning disk
• High ingest rate, low read query, regular aggregation
  • Example application: User behavior analysis, system log management
  • 2 hex-core CPUs, 64GB memory, SSD
• Balanced mixed workload, high transaction rate, high query, high read, occasional aggregation
  • Example application: Social media applications, e-commerce,
  • 2 hex-core CPUs, 128GB memory, SSD