Spark shuffle機制演進史及原理說明（sort-based/hash-based/bypassShuffleManager）

spark shuffle 演進的歷史 Spark 0.8及以前 Hash Based Shuffle Spark 0.8.1 爲Hash Based Shuffle引入File Consolidation機制 Spark 0.9 引入ExternalAppendOnlyMap Spark 1.1 引入Sort Based Shuffle，但默認仍爲Hash Based Shuffle Spark

>>阅读原文<<