# Theflash TPCH-100 benchmark on Ceph file system

## Result columns
* MergeTree:
    * Device: NVME
    * Engine: MergeTree
    * Args: partitions=16
* Mutable:
    * Device: NVME
    * Engine: MutableMergeTree
    * Args: partitions=16
* Ceph:
    * Device:
        * Ceph fs built on 3 nodes
        * 3 replica copies
        * Each node has one osd (one blue store on loop device)
    * Engine: MutableMergeTree
    * Args: partitions=16

| Query | MergeTree | Mutable |    Ceph |
| ----- | --------: | ------: | ------: |
| Q01   |       9.5 |    17.6 |    17.6 |
| Q02   |      27.7 |    29.2 |    29.4 |
| Q03   |      38.7 |    44.6 |    43.1 |
| Q04   |     132.8 |   139.5 |   138.6 |
| Q05   |      78.3 |    82.8 |    83.1 |
| Q06   |       4.2 |    10.5 |    10.8 |
| Q07   |      54.9 |    67.0 |    61.0 |
| Q08   |      99.4 |   118.1 |   103.0 |
| Q09   |     110.4 |   115.1 |   115.6 |
| Q10   |      31.6 |    33.5 |    34.1 |
| Q11   |      12.4 |    13.4 |    13.2 |
| Q12   |      24.3 |    27.5 |    27.2 |
| Q13   |      43.6 |    40.9 |    40.7 |
| Q14   |       9.5 |    13.8 |    13.6 |
| Q15   |           |         |         |
| Q16   |      99.0 |   107.3 |   108.5 |
| Q17   |           |         |         |
| Q18   |           |         |         |
| Q19   |      18.1 |    20.3 |    20.3 |
| Q20   |           |         |         |
| Q21   |           |         |         |
| Q22   |      54.1 |    55.6 |    58.1 |


## Raw result data
* MergeTree: partitions=16
```
Q01, avg:   9.5, detail: [9.5, 8.7, 9.6, 9.4, 9.9, 9.9]
Q02, avg:  27.7, detail: [27.4, 26.6, 27.5, 28.7, 27.7, 28.0]
Q03, avg:  38.7, detail: [39.4, 38.4, 39.7, 39.8, 37.3, 37.8]
Q04, avg: 132.8, detail: [131.1, 131.0, 139.4, 124.8, 143.1, 127.2]
Q05, avg:  78.3, detail: [74.9, 80.3, 79.2, 76.9, 81.9, 76.7]
Q06, avg:   4.2, detail: [4.3, 4.1, 4.3, 4.1, 4.3, 4.0]
Q07, avg:  54.9, detail: [55.4, 53.7, 54.5, 54.4, 57.4, 54.0]
Q08, avg:  99.4, detail: [89.8, 88.0, 159.7, 88.5, 84.7, 85.4]
Q09, avg: 110.4, detail: [112.3, 108.5, 106.6, 106.9, 115.5, 112.7]
Q10, avg:  31.6, detail: [32.1, 32.2, 30.5, 31.3, 30.9, 32.3]
Q11, avg:  12.4, detail: [12.3, 12.6, 12.9, 11.9, 12.4, 12.6]
Q12, avg:  24.3, detail: [24.7, 24.4, 24.8, 24.1, 24.0, 24.0]
Q13, avg:  43.6, detail: [43.6, 41.5, 39.7, 40.4, 41.1, 55.1]
Q14, avg:   9.5, detail: [9.2, 9.9, 9.5, 9.4, 9.1, 9.6]
Q16, avg:  99.0, detail: [99.4, 97.8, 100.1, 101.1, 96.6]
Q19, avg:  18.1, detail: [17.2, 20.3, 17.4, 17.7, 17.7]
Q22, avg:  54.1, detail: [53.8, 54.6, 52.7, 54.6, 54.8]
```
* Mutable: partitions=16
```
Q01, avg:  17.6, detail: [19.3, 18.3, 17.2, 17.0, 16.2]
Q02, avg:  29.2, detail: [30.2, 29.2, 28.1, 28.8, 29.5]
Q03, avg:  44.6, detail: [44.9, 41.4, 51.3, 45.0, 40.2]
Q04, avg: 139.5, detail: [138.2, 136.3, 149.0, 138.0, 136.1]
Q05, avg:  82.8, detail: [88.0, 81.0, 82.1, 78.4, 84.6]
Q06, avg:  10.5, detail: [10.3, 10.4, 10.5, 10.4, 10.8]
Q07, avg:  67.0, detail: [85.0, 59.2, 58.3, 61.3, 71.1]
Q08, avg: 118.1, detail: [142.0, 107.7, 157.4, 88.6, 94.8]
Q09, avg: 115.1, detail: [117.6, 112.1, 115.9, 116.1, 113.6]
Q10, avg:  33.5, detail: [34.5, 32.2, 33.4, 34.8, 32.7]
Q11, avg:  13.4, detail: [13.0, 13.7, 13.2, 13.2, 13.9]
Q12, avg:  27.5, detail: [28.1, 27.0, 27.7, 28.0, 26.6]
Q13, avg:  40.9, detail: [42.2, 38.2, 44.0, 40.3, 39.8]
Q14, avg:  13.8, detail: [14.4, 13.9, 13.0, 13.2, 14.6]
Q16, avg: 107.3, detail: [117.7, 117.2, 107.2, 104.5, 89.9]
Q19, avg:  20.3, detail: [22.1, 20.0, 19.8, 20.0, 19.5]
Q22, avg:  55.6, detail: [54.7, 55.6, 55.8, 55.1, 57.0]
```
* Ceph: partitions=16
```
Q01, avg:  17.6, detail: [16.4, 16.2, 18.3, 16.8, 17.4, 19.2, 17.1, 19.3, 17.9, 17.7]
Q02, avg:  29.4, detail: [29.0, 29.6, 29.6, 29.5, 28.8, 29.4, 30.0, 29.7, 28.8, 29.3]
Q03, avg:  43.1, detail: [40.5, 40.4, 52.7, 44.8, 41.2, 42.0, 42.2, 39.1, 42.6, 45.1]
Q04, avg: 138.6, detail: [138.2, 141.0, 144.3, 134.0, 139.2, 135.5, 139.4, 138.9, 139.6, 136.2]
Q05, avg:  83.1, detail: [79.6, 83.4, 87.2, 82.9, 78.5, 83.4, 81.9, 84.8, 87.0, 82.6]
Q06, avg:  10.8, detail: [10.7, 10.7, 10.5, 10.8, 12.0, 10.2, 11.0, 9.6, 10.3, 11.9]
Q07, avg:  61.0, detail: [61.5, 62.1, 58.8, 62.0, 59.9, 61.1, 66.0, 57.8, 61.1, 59.5]
Q08, avg: 103.0, detail: [142.2, 92.5, 92.2, 90.9, 93.6, 151.7, 91.7, 91.1, 91.6, 92.6]
Q09, avg: 115.6, detail: [116.0, 121.9, 111.3, 112.7, 120.1, 117.6, 112.5, 108.9, 118.4, 117.0]
Q10, avg:  34.1, detail: [34.1, 36.5, 32.1, 33.9, 35.3, 36.0, 33.8, 34.2, 33.0, 32.5]
Q11, avg:  13.2, detail: [13.1, 13.0, 13.1, 14.0, 13.1, 12.9, 13.3, 13.2, 13.1, 13.5]
Q12, avg:  27.2, detail: [29.4, 27.3, 26.1, 26.8, 25.6, 26.4, 27.7, 26.5, 27.1, 29.6]
Q13, avg:  40.7, detail: [41.4, 40.5, 38.9, 40.6, 44.5, 39.3, 40.8, 39.1, 40.3, 41.2]
Q14, avg:  13.6, detail: [13.4, 12.7, 13.0, 14.7, 13.1, 15.3, 13.4, 13.0, 13.3, 13.9]
Q16, avg: 108.5, detail: [99.1, 106.8, 145.7, 104.8, 102.4, 110.2, 104.0, 116.6, 97.8, 97.5]
Q19, avg:  20.3, detail: [19.8, 21.2, 21.9, 19.3, 19.5, 21.7, 19.3, 21.5, 19.7, 19.5]
Q22, avg:  58.1, detail: [57.6, 62.4, 56.0, 58.5, 55.6, 57.4, 61.0, 56.2, 58.0]
```


## Environment

### Hardware
* 40 Cores, Intel(R) Xeon(R) CPU E5-2630 v4 @ 2.20GHz
* 128G Memory 2133MHz
* One NVME SSD 2.8GB+/s, 3.6T

### Software
* CentOS Linux release 7.4.1708
* Spark version 2.1.1
* Theflash git hash 7d597dfd79cd91471dfca9689bbf2d9c86caef43

### Data Set
* TPCH-100 100G data scala