Wednesday, July 30, 2014

New record for a large JDG cluster: 500 nodes

Today I ran a 500 node JDG (JBoss Data Grid) cluster on Google Compute Engine, topping the previous record of 256 nodes. The test used is IspnPerfTest [2] and it does the following:
  • The data grid has keys [1..20000], values are 1KB byte buffers
  • The cache is DIST-SYNC with 2 owners for every key
  • Every node invokes 20'000 requests on random keys
  • A request is a read (with an 80% chance) or a write (with a 20% chance)
    • A read returns a 1KB buffer
    • A write takes a 1KB buffer and sends it to the primary node and the backup node
  • The test invoker collects the results from all 500 nodes and computes an average which it prints to stdout.
But see for yourself; the YouTube video is here: [1].

In terms of cluster size, I could probably have gone higher: it was pretty effortless to go to 500 instances... any takers ? :-)


[1] http://youtu.be/Cz3CDr31EDU
[2] https://github.com/belaban/IspnPerfTest

5 comments:

  1. 20000 keys for 500 nodes?
    I think one node would be enough for that.

    ReplyDelete
  2. You're right, but I didn't want to test how much data can be put into a 500 node grid, but how fast access is on a 500 node cluster.

    ReplyDelete