jvm-serializers

WARNING: Benchmarks can be misleading.

• These tests use a specific data value (DataStructuresV2). A different data value will yield different results.
• The tools have different sets of features (BeyondNumbers). Some of these features make things safer or easier, but come with a performance cost.
• Different hardware and software environments will yield different results.
• We don't take memory usage into account.

 

In short, before you make a decision on which tool to use, make sure you try it out in an environment you care about. To start, download the benchmark code and run it on your hardware with data values you care about.

Setup¶

Hardware: Intel Core 2 Quad

Software: Sun JRE 1.6.0_20 (64-bit server VM), Ubuntu 9.10

JVM options: -Xmx16m -server

Data value being tested: DataStructuresV2.

Version of the benchmarking code: Git tree

Methodology:

• Before taking measurements, we warm things up by running the test several times.
• For a test, measure the time taken to perform 2000 operations (serialization, deserialization, etc.). Then divide the result by 2000.
• Run each test 100 times and report the best result.
• Look at the code for more details. BenchmarkRunner.java

 

Tool Versions (lib/):

• protobuf 2.3.0
• thrift 0.2.0
• avro 1.3.2
• kryo 1.0
• hessian 4.0.3
• activemq-protobuf 1.1-SNAPSHOT
• scala 2.8.0-rc1
• sbinary 0.3.1-SNAPSHOT
• google-gson 1.4
• jackson 1.5.1
• javolution 5.2.6
• protostuff 1.0.0.M3
• woodstox 4.0.7
• aalto 0.9.5
• fast-infoset 1.2.6
• xstream 1.3.1

 

Charts¶

Total Time ("total")¶

Create an object, serialize it to a byte array, then deserialize it back to an object.

Serialization Time ("ser")¶

Create an object, serialize it to a byte array.

• Java's built-in serializer faithfully represents arbitrary object graphs, which hurts performance. All the other serializers flatten the structure out to a tree.

 

Deserialization Time ("deser+deep")¶

Often the most expensive operation. To make a fair comparison, all fields of the deserialized instances are accessed - this forces lazy deserializers to really do their work. The raw data below shows additional measurements for deserialization.

Serialized Size ("size")¶

The size of the serialized data. These numbers may vary depending on the exact data value being used.

• Java's built-in serializer stores the full class name in serialized form. So you don't need to know ahead of time what kind of object you're reading in.
• The 'scala' test, which uses Java's built-in serialization, yields a larger serialized representation because it usually creates more Java classes under the hood.

 

Serialization Compressed Size ("size+dfl")¶

The size of the serialized data compressed with Java's built-in implementation of DEFLATE (zlib).

Object Creation Time ("create")¶

Object creation is not so meaningful since it takes in average 100 nano to create an object. However, the different tools vary in how "fancy" their objects are. Some just create a plain Java class and let you access fields directly, while others have set/get methods, while others use the "builder" pattern.

• Protobuf and Thrift use the "builder" pattern to create objects, which makes the operation more expensive.
• Avro stores Strings in UTF8 form. The time taken to convert from Java "String" values to UTF-8 is included under "create", "ser", "deser+shal", and "deser+deep", which isn't quite representative of real-world usage. Real code that uses Avro might be able to keep strings in UTF-8 form, thus avoiding the need to convert back and forth (in which case the "ser+same" and "deser" results might be more accurate reflections of Avro's performance).

 

Numbers¶

Times are in nanoseconds, sizes are in bytes.

                             create     ser   +same   deser   +shal   +deep   total   size  +dfl  protobuf                        482    4115    2087    2428    2516    2731    6846    239   149  thrift                          408    4139    3869    4609    4851    4894    9032    349   195  thrift-compact                  406    4028    3541    4881    4997    5243    9271    240   149  protobuf/activemq+alt           368    4006       9      23    1444    2779    6784    239   149  protobuf/protostuff             356    1399    1123    2197    2277    2436    3835    239   149  protobuf/protostuff-runtime     225    1745    1612    2228    2510    2585    4330    241   150  protostuff-core-ge              358    1328    1067    2265    2345    2486    3814    239   150  protostuff-runtime-ge           225    1686    1539    2273    2552    2617    4303    241   151  kryo                            225    2108    1948    2309    2383    2468    4577    233   147  kryo-opt                        225    2002    1852    2112    2188    2261    4263    219   135  avro                           1857    4674    2758    6065    7222    8001   12675    221   133  avro-generic                   2380    5226    2520    5557    6864    7935   13161    221   133  hessian                         221   11022   10083   11824   12008   11963   22985    501   313  java-built-in                   224   13025   11738   59070   59537   59759   72785    889   517  java-manual                     225    1808    1664    1301    1378    1478    3285    255   147  scala/java-built-in             756   20788   17855   88676   89203   89851  110640   1312   700  scala/sbinary                   748    4214    3322    2900    3061    3370    7583    255   147  json/jackson-manual             222    6648    6504    4525    4481    4574   11222    398   220  json/jackson-databind           231   10457   10105    6401    6508    6670   17127    503   270  json/protostuff                 361    7749    7073    4933    5049    5208   12957    382   194  json/protostuff+numeric         357    7934    7497    4787    4955    5101   13036    360   191  json/protobuf                   507   22287   21118  117870  120792  121540  143828    488   253  json/google-gson                241   82257   81695  133003  135203  135314  217571    486   259  xml/manual-woodstox             239    7031    6868    9437    9704    9917   16948    495   267  xml/manual-aalto                239    4680    4179    5910    6031    6135   10815    495   267  xml/manual-fastinfo             224   14245   14017   16293   17032   17309   31553    333   250  xml/xstream                     236   85322   82668  141576  142139  141402  226724    758   313  xml/xstream+c                   229   15140   14010   49303   49332   47802   62942    381   205  woodstox-xstream                239   87595   84340  141817  142207  140461  228056    796   344  woodstox-xstream+c              221   12674   11355   24154   24303   24579   37253    419   236  aalto-xstream                   225   85946   82487  137690  140350  138958  224903    796   344  aalto-xstream+c                 223   10812    9495   16718   16915   17339   28151    419   236  fastinfo-xstream                235  104124  100900  123098  124063  123837  227961    444   323  fastinfo-xstream+c              234   20251   18776   23823   24149   24325   44576    293   227  xml/javolution                  219    5732    5546    8851    8926    8971   14703    432   239

Columns:

• create: create an object (using the classes specified by the serialization tool)
• ser: create an object and serialize it
• +same: serialize the same object (i.e. doesn't include creation time)
• deser: deserialize an object
• +shal: deserialize an object and access the top-level fields
• +deep: deserialize an object and access all the fields
• total: create + serialize + deserialize and access all fields
• size: the size of the serialized data
• +dfl: the size of the serialized data compressed with Java's built-in implementation of DEFLATE (zlib)