# TheFlash TPCH on 4 nodes ## Config and result * CH-0001: partitionsPerSplit=1 * CH-0002: partitionsPerSplit=2 * CH-0003: partitionsPerSplit=4 * CH-0004: partitionsPerSplit=8 | Query | CH-0001 | CH-0002 | CH-0003 | CH-0004 | | ----- | ------: | ------: | ------: | ------: | | Q01 | 9.4 | 9.0 | 8.8 | 9.0 | | Q02 | 26.7 | 25.9 | 25.6 | 26.8 | | Q03 | 34.9 | 32.5 | 34.4 | 46.3 | | Q04 | 81.4 | 81.5 | 95.3 | 145.1 | | Q05 | 56.1 | 53.9 | 59.5 | 82.4 | | Q06 | 5.4 | 5.4 | 5.3 | 5.2 | | Q07 | 47.2 | 46.9 | 49.6 | 61.3 | | Q08 | 58.9 | 61.4 | 74.0 | 100.6 | | Q09 | 74.5 | 72.8 | 82.1 | 108.8 | | Q10 | 29.6 | 28.7 | 28.3 | 32.4 | | Q11 | 13.3 | 12.5 | 12.1 | 13.5 | | Q12 | 22.5 | 22.1 | 24.7 | 38.7 | | Q13 | 33.1 | 32.3 | 37.8 | 54.8 | | Q14 | 11.5 | 10.6 | 10.3 | 10.7 | | Q15 | 14.1 | 13.2 | 12.3 | 11.4 | | Q16 | 80.2 | 95.1 | 174.7 | 184.8 | | Q17 | 94.0 | 92.0 | 98.1 | 77.4 | | Q18 | 108.7 | 111.7 | 120.8 | 93.3 | | Q19 | 17.8 | 16.1 | 15.6 | 15.2 | | Q20 | 41.2 | 40.8 | 43.0 | 49.2 | | Q21 | 314.8 | 341.1 | 374.8 | 393.1 | | Q22 | 38.4 | 37.1 | 44.3 | 62.3 | ## Raw result data * CH-0001: partitionsPerSplit=1 ``` Q01, avg: 9.4, detail: [9.8, 9.1, 9.4, 9.3, 9.6] Q02, avg: 26.7, detail: [26.9, 26.4, 26.9, 26.3, 26.9] Q03, avg: 34.9, detail: [36.0, 34.4, 34.1, 35.6, 34.5] Q04, avg: 81.4, detail: [79.9, 83.0, 81.2, 82.1, 80.7] Q05, avg: 56.1, detail: [56.9, 54.6, 56.0, 56.3, 56.8] Q06, avg: 5.4, detail: [5.6, 5.3, 5.3, 5.4, 5.5] Q07, avg: 47.2, detail: [47.4, 47.6, 47.6, 47.4, 45.9] Q08, avg: 58.9, detail: [58.5, 58.7, 59.4, 59.3, 58.8] Q09, avg: 74.5, detail: [74.0, 73.8, 75.4, 74.5, 74.9] Q10, avg: 29.6, detail: [29.3, 28.9, 29.7, 31.0, 29.2] Q11, avg: 13.3, detail: [13.3, 13.2, 13.5, 13.2, 13.5] Q12, avg: 22.5, detail: [22.3, 22.3, 22.8, 22.9, 22.0] Q13, avg: 33.1, detail: [32.1, 33.3, 33.5, 32.2, 34.2] Q14, avg: 11.5, detail: [11.2, 11.7, 11.7, 11.4, 11.3] Q15, avg: 14.1, detail: [13.9, 14.6, 14.4, 14.1, 14.8, 13.6, 13.8, 13.7, 14.1, 14.2] Q16, avg: 80.2, detail: [57.8, 69.8, 89.6, 97.7, 86.3] Q17, avg: 94.0, detail: [96.2, 95.9, 95.3, 95.9, 92.0, 93.4, 93.8, 89.8, 93.4, 94.7] Q18, avg: 108.7, detail: [109.2, 106.4, 107.5, 108.7, 110.5, 109.5, 107.8, 110.3, 108.2, 108.9] Q19, avg: 17.8, detail: [18.2, 17.5, 18.2, 17.7, 17.2] Q20, avg: 41.2, detail: [41.6, 42.0, 39.9, 41.1, 40.1, 41.0, 41.0, 40.5, 42.4, 42.0] Q21, avg: 314.8, detail: [312.5, 314.4, 312.0, 310.1, 316.0, 334.0, 311.7, 313.3, 309.9, 313.6] Q22, avg: 38.4, detail: [38.3, 38.3, 37.2, 39.5, 38.5] ``` * CH-0002: partitionsPerSplit=2 ``` Q01, avg: 9.0, detail: [9.0, 8.9, 9.0, 9.0, 9.2] Q02, avg: 25.9, detail: [25.5, 26.6, 25.3, 25.8, 26.2] Q03, avg: 32.5, detail: [32.4, 32.2, 33.3, 32.5, 31.9] Q04, avg: 81.5, detail: [80.8, 80.5, 82.2, 83.0, 81.0] Q05, avg: 53.9, detail: [53.9, 53.3, 54.6, 54.0, 53.6] Q06, avg: 5.4, detail: [5.1, 5.5, 5.5, 5.5, 5.3] Q07, avg: 46.9, detail: [46.9, 47.2, 46.3, 46.5, 47.7] Q08, avg: 61.4, detail: [57.8, 57.2, 58.7, 75.1, 58.3] Q09, avg: 72.8, detail: [73.9, 72.1, 72.0, 73.8, 72.4] Q10, avg: 28.7, detail: [28.6, 28.7, 28.7, 28.8, 28.8] Q11, avg: 12.5, detail: [12.6, 12.6, 12.8, 12.4, 12.1] Q12, avg: 22.1, detail: [21.4, 22.1, 21.6, 22.2, 23.1] Q13, avg: 32.3, detail: [31.9, 32.7, 31.9, 33.3, 31.9] Q14, avg: 10.6, detail: [10.5, 10.4, 10.7, 10.9, 10.3] Q15, avg: 13.2, detail: [13.7, 13.3, 13.3, 13.4, 13.8, 12.8, 13.0, 12.6, 12.8, 13.2] Q16, avg: 95.1, detail: [66.6, 140.5, 132.5, 63.6, 72.1] Q17, avg: 92.0, detail: [93.7, 93.8, 92.7, 90.9, 91.7, 91.3, 89.6, 90.8, 92.5, 92.6] Q18, avg: 111.7, detail: [109.3, 109.4, 112.1, 109.0, 108.8, 111.7, 109.5, 109.8, 109.2, 128.6] Q19, avg: 16.1, detail: [16.5, 15.7, 16.2, 15.9, 16.1] Q20, avg: 40.8, detail: [40.4, 39.2, 40.6, 39.9, 39.2, 40.1, 43.2, 41.6, 42.1, 41.7] Q21, avg: 341.1, detail: [339.8, 340.7, 342.8, 335.2, 344.5, 345.8, 338.8, 345.5, 337.4, 340.9] Q22, avg: 37.1, detail: [36.3, 36.4, 38.0, 36.3, 38.7] ``` * CH-0003: partitionsPerSplit=4 ``` Q01, avg: 8.8, detail: [8.7, 8.9, 8.7, 8.8, 8.9] Q02, avg: 25.6, detail: [25.4, 26.0, 25.4, 25.6, 25.8] Q03, avg: 34.4, detail: [34.7, 33.2, 33.9, 35.3, 34.8] Q04, avg: 95.3, detail: [93.9, 98.2, 99.7, 92.9, 91.6] Q05, avg: 59.5, detail: [59.1, 58.8, 59.4, 59.8, 60.2] Q06, avg: 5.3, detail: [5.2, 5.4, 5.0, 5.4, 5.5] Q07, avg: 49.6, detail: [47.4, 51.5, 48.1, 49.6, 51.6] Q08, avg: 74.0, detail: [76.9, 67.1, 71.2, 88.0, 66.8] Q09, avg: 82.1, detail: [81.8, 81.9, 84.2, 81.6, 81.0] Q10, avg: 28.3, detail: [28.4, 28.6, 28.1, 28.0, 28.6] Q11, avg: 12.1, detail: [11.8, 12.1, 12.2, 12.4, 12.2] Q12, avg: 24.7, detail: [30.2, 23.6, 22.9, 23.8, 23.2] Q13, avg: 37.8, detail: [37.0, 37.3, 37.7, 36.5, 40.3] Q14, avg: 10.3, detail: [10.2, 10.4, 10.5, 10.3, 10.1] Q15, avg: 12.3, detail: [12.1, 12.3, 12.6] Q16, avg: 174.7, detail: [131.8, 123.6, 130.4, 358.6, 129.3] Q17, avg: 98.1, detail: [95.5, 94.0, 102.4, 96.1, 104.8, 95.9] Q18, avg: 120.8, detail: [118.2, 124.6, 118.1, 121.3, 121.9] Q19, avg: 15.6, detail: [15.4, 15.7, 15.8, 15.6, 15.4] Q20, avg: 43.0, detail: [42.6, 43.1, 43.3] Q21, avg: 374.8, detail: [368.0, 395.9, 360.5] Q22, avg: 44.3, detail: [46.0, 41.7, 45.0, 47.5, 41.2] ``` * CH-0004: partitionsPerSplit=8 ``` Q01, avg: 9.0, detail: [8.9, 8.7, 9.1, 9.1, 9.2] Q02, avg: 26.8, detail: [26.7, 26.7, 26.9, 26.9, 27.0] Q03, avg: 46.3, detail: [46.3, 46.9, 45.4, 46.6, 46.3] Q04, avg: 145.1, detail: [145.5, 157.4, 139.1, 142.6, 140.9] Q05, avg: 82.4, detail: [84.2, 82.0, 81.1, 80.2, 84.6] Q06, avg: 5.2, detail: [5.3, 5.1, 5.3, 5.2, 5.2] Q07, avg: 61.3, detail: [63.0, 60.5, 65.3, 57.2, 60.7] Q08, avg: 100.6, detail: [111.0, 96.3, 97.4, 105.9, 92.6] Q09, avg: 108.8, detail: [107.8, 109.5, 107.4, 107.1, 112.4] Q10, avg: 32.4, detail: [32.3, 32.3, 32.3, 32.9, 32.0] Q11, avg: 13.5, detail: [13.3, 13.6, 13.1, 13.9, 13.6] Q12, avg: 38.7, detail: [38.4, 38.1, 40.2, 38.6, 38.4] Q13, avg: 54.8, detail: [54.8, 55.8, 55.1, 53.6, 54.9] Q14, avg: 10.7, detail: [10.8, 10.8, 10.7, 10.7, 10.5] Q15, avg: 11.4, detail: [11.8, 11.4, 11.2, 11.5, 11.2] Q16, avg: 184.8, detail: [225.7, 220.6, 134.8, 132.0, 211.0] Q17, avg: 77.4, detail: [75.8, 76.1, 78.2, 79.6] Q18, avg: 93.3, detail: [91.0, 92.9, 95.4, 91.5, 95.9] Q19, avg: 15.2, detail: [15.2, 15.2, 15.1, 15.1, 15.3] Q20, avg: 49.2, detail: [49.0, 51.0, 47.9, 49.3, 48.8] Q21, avg: 393.1, detail: [388.8, 386.4, 413.4, 384.7, 395.0, 390.2] Q22, avg: 62.3, detail: [63.3, 68.8, 62.6, 60.4, 56.4] ``` ## Environment ### Hardware * 4 vm nodes, each node: * 10 Cores, Intel Core Processor (Broadwell) @ 2.20GHz * 27G Memory 2133MHz * One NVME SSD, 3GB+/s on read, 1.6GB+/s on write, 1.5T * Network: * Brandwidth: 2.2GB+/s * Ping: 0.25ms (avg) ### Software * Centos-release-7-5.1804.el7 * Spark version 2.1.1 * TheFlash git hash b738d4c7a1722af07122f7d58cc0946082869266 ### Data Set * TPCH-100 100G data scala