Add design documents (#757)

* add design doc and instructions for inserting figures
Co-authored-by: Zesong Sun <szs19@mails.tsinghua.edu.cn>

.travis.yml

@@ -175,7 +175,7 @@ matrix:
 #        - mvn -B apache-rat:check
 #        - mvn -B clean package -pl server,grafana,client,example,:kafka-example,:rocketmq-example -am integration-test
     - os: linux
-#      if: fork = false #only fork=true (i.e., the committer has permission to write the repo)
+      if: fork = false #only fork=true (i.e., the committer has permission to write the repo)
       name: sonar-analysis
       dist: xenial
       jdk: openjdk8

docs/Development/Development-Contributing.md

@@ -95,6 +95,9 @@ Then,
    * Step 1: create a new folder named "7-RoadMap", and add document named `1-B.md` in folder "7-RoadMap";
    * Step 2: modify `0-Content.md` file by adding "# Chapter 7: RoadMap" in the end, and  adding `* 1-B.md` in the list of this new chapter.
 
+If you need to insert **figures** into documents, you can firstly update the figures in [this issue](https://github.com/thulab/iotdb/issues/543) for storing pictures in IoTDB website or other MD files.
+Drag a picture and then quote the figure's URL link. 
+
 ### Contributing Bug Reports
 
 If you encounter a problem, try to search the mailing list and JIRA to check whether other people have faced the same situation. If it is not reported before, please report an issue.

docs/Development/Development-Document.md

@@ -45,4 +45,7 @@ Then,
    * Step 1: create a new folder named "7-RoadMap", and add document named `1-B.md` in folder "7-RoadMap";
    * Step 2: modify `0-Content.md` file by adding "# Chapter 7: RoadMap" in the end, and  adding `* 1-B.md` in the list of this new chapter.
 
+If you need to insert **figures** into documents, you can firstly update the figures in [this issue](https://github.com/thulab/iotdb/issues/543) for storing pictures in IoTDB website or other MD files.
+Drag a picture and then quote the figure's URL link. 
+
 > If you want to contribute more (for example, reviewing Changes, reporting bugs, or even being commiters), please refer to [this page](/#/Development/Contributing).

docs/Documentation-CHN/SystemDesign/0-Architecture/1-Architecture.md

@@ -0,0 +1,55 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 应用概览
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/73625222-ddd88680-467e-11ea-9098-e808ed4979c5.png">
+
+物联网时序数据库 Apache IoTDB 的架构图如上所示，覆盖了对时序数据的采集、存储、查询、分析以及可视化等全生命周期的数据管理功能，其中灰色部分为 IoTDB 组件。
+
+## IoTDB 架构介绍
+
+IoTDB 采用客户端-服务器架构，如下图所示。
+
+<img style="width:100%; max-width:400px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/73625221-ddd88680-467e-11ea-9cf3-70367e5886f4.png">
+
+其中服务器端主要包括查询引擎，用来处理用户的所有请求，并分发到对应的管理组件，包括数据写入层、数据查询、元数据管理、权限管理等模块。
+
+* [数据文件](../1-TsFile/1-TsFile.md)
+* [查询引擎](../2-QueryEngine/1-QueryEngine.md)
+* [元数据管理](../3-SchemaManager/1-MManager.md)
+* [存储引擎](../4-StorageEngine/1-StorageEngine.md)
+* [数据查询](../5-DataQuery/1-DataQuery.md)
+* [权限管理](../6-Administration/1-Administration.md)
+
+## 连接器
+
+IoTDB 与大数据系统进行了对接。
+
+* [Hadoop-TsFile](../7-Connector/1-Hadoop-TsFile.md)
+* [Hive-TsFile](../7-Connector/2-Hive-TsFile.md)
+* [Spark-TsFile](../7-Connector/3-Spark-TsFile.md)
+* [Spark-IoTDB](../7-Connector/4-Spark-IoTDB.md)
+* [Grafana](../7-Connector/5-Grafana.md)
+
+## 工具
+
+* [同步工具](../8-Tools/1-Sync.md)

docs/Documentation-CHN/SystemDesign/1-TsFile/1-TsFile.md

@@ -0,0 +1,31 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 数据文件
+
+TsFile 是 IoTDB 的底层数据文件，专门为时间序列数据设计的列式文件格式。
+
+
+## 相关文档
+
+* [文件格式](2-Format.md)
+* [写流程](3-Write.md)
+* [读流程](4-Read.md)

docs/Documentation-CHN/SystemDesign/1-TsFile/2-Format.md

@@ -7,9 +7,9 @@
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at
-    
+
         http://www.apache.org/licenses/LICENSE-2.0
-    
+
     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -19,7 +19,8 @@
 
 -->
 
-# 第8章: 系统设计
+# TsFile 文件格式
+
 
 ## 1. TsFile 设计
 

docs/Documentation-CHN/SystemDesign/1-TsFile/3-Write.md

@@ -0,0 +1,65 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# TsFile 写流程
+
+* org.apache.iotdb.tsfile.write.*
+
+TsFile 的写入流程如下图所示：
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/73625238-efba2980-467e-11ea-927e-a7021f8153af.png">
+
+其中，每个设备对应一个 ChunkGroupWriter，每个传感器对应一个 ChunkWriter。
+
+文件的写入主要分为三种操作，在图上用 1、2、3 标注
+
+* 1、写内存换冲区
+* 2、持久化 ChunkGroup
+* 3、关闭文件
+
+## 1、写内存缓冲区
+
+TsFile 文件层的写入接口有两种
+
+* TsFileWriter.write(TSRecord record)
+
+	写入一个设备一个时间戳多个测点。
+
+* TsFileWriter.write(RowBatch rowBatch)
+
+	写入一个设备多个时间戳多个测点。
+
+当调用 write 接口时，这个设备的数据会交给对应的 ChunkGroupWriter，其中的每个测点会交给对应的 ChunkWriter 进行写入。ChunkWriter 完成编码和打包（生成 Page）。
+
+
+## 2、持久化 ChunkGroup
+
+* TsFileWriter.flushAllChunkGroups()
+
+当内存中的数据达到一定阈值，会触发持久化操作。每次持久化会把当前内存中所有设备的数据全部持久化到磁盘的 TsFile 文件中。每个设备对应一个 ChunkGroup，每个测点对应一个 Chunk。
+
+持久化完成后会在内存中缓存对应的元数据信息，以供查询和生成文件尾部 metadata。
+
+## 3、关闭文件
+
+* TsFileWriter.close()
+
+根据内存中缓存的元数据，生成 TsFileMetadata 追加到文件尾部，最后关闭文件。

docs/Documentation-CHN/SystemDesign/1-TsFile/4-Read.md

@@ -0,0 +1,27 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# TsFile 读流程
+
+* org.apache.iotdb.tsfile.read.*
+
+
+

docs/Documentation-CHN/SystemDesign/2-QueryEngine/1-QueryEngine.md

@@ -0,0 +1,64 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 查询引擎
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/73625242-f648a100-467e-11ea-921c-b954a3ecae7a.png">
+
+## 设计思想
+
+查询引擎负责所有用户命令的解析、生成计划、交给对应的执行器、返回结果集。
+
+## 相关类
+
+* org.apache.iotdb.db.service.TSServiceImpl
+
+	IoTDB 服务器端 RPC 实现，与客户端进行直接交互。
+	
+* org.apache.iotdb.db.qp.Planner
+	
+	解析 SQL，生成逻辑计划，逻辑优化，生成物理计划。
+
+* org.apache.iotdb.db.qp.executor.PlanExecutor
+
+	分发物理计划给对应的执行器，主要包括以下四个具体的执行器。
+	
+	* MManager: 元数据操作
+	* StorageEngine: 数据写入
+	* QueryRouter: 数据查询
+	* LocalFileAuthorizer: 权限操作
+
+* org.apache.iotdb.db.query.dataset.*
+
+	分批构造结果集返回给客户端，包含部分查询逻辑。
+
+## 查询流程
+
+* SQL 解析
+* 生成逻辑计划
+* 生成物理计划
+* 构造结果集生成器
+* 分批返回结果集
+
+## 相关文档
+
+* [查询计划生成器](2-Planner.md)
+* [计划执行器](3-PlanExecutor.md)

docs/Documentation-CHN/SystemDesign/2-QueryEngine/2-Planner.md

@@ -0,0 +1,63 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 执行计划生成器 Planner
+
+* org.apache.iotdb.db.qp.Planner
+
+将 SQL 解析出的语法树转化成逻辑计划，逻辑优化，物理计划。
+
+## SQL 解析
+
+SQL 解析采用 Antlr4
+
+* server/src/main/antlr4/org/apache/iotdb/db/qp/strategy/SqlBase.g4
+
+mvn clean compile 之后生成代码位置：server/target/generated-sources/antlr4
+
+## 逻辑计划生成器
+
+* org.apache.iotdb.db.qp.strategy.LogicalGenerator
+
+## 逻辑计划优化器
+
+目前有三种逻辑计划优化器
+
+* org.apache.iotdb.db.qp.strategy.optimizer.ConcatPathOptimizer
+
+	路径优化器，将 SQL 中的查询路径进行拼接，与 MManager 进行交互去掉通配符，进行路径检查。
+
+* org.apache.iotdb.db.qp.strategy.optimizer.RemoveNotOptimizer
+
+	谓词去非优化器，将谓词逻辑中的非操作符去掉。
+
+* org.apache.iotdb.db.qp.strategy.optimizer.DnfFilterOptimizer
+	
+	将谓词转化为析取范式。
+	
+* org.apache.iotdb.db.qp.strategy.optimizer.MergeSingleFilterOptimizer
+
+	将相同路径的谓词逻辑合并。
+	
+## 物理计划生成器
+
+* org.apache.iotdb.db.qp.strategy.PhysicalGenerator
+

docs/Documentation-CHN/SystemDesign/2-QueryEngine/3-PlanExecutor.md

@@ -0,0 +1,26 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 计划执行器
+
+* org.apache.iotdb.db.qp.executor.PlanExecutor
+
+执行物理计划。

docs/Documentation-CHN/SystemDesign/3-SchemaManager/1-SchemaManager.md

@@ -0,0 +1,26 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 元数据管理
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/73625246-fc3e8200-467e-11ea-8815-67b9c4ab716e.png">
+
+IoTDB 的元数据管理采用目录树的形式，倒数第二层为设备层，最后一层为传感器层。

docs/Documentation-CHN/SystemDesign/4-StorageEngine/1-StorageEngine.md

@@ -0,0 +1,92 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 存储引擎
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/73625255-03fe2680-467f-11ea-91ae-64407ef1125c.png">
+
+## 设计思想
+
+存储引擎基于 LSM 设计。数据首先写入内存缓冲区 memtable 中，再刷到磁盘。内存中为每个设备维护当前持久化的（包括已经落盘的和正在持久化的）最大时间戳，根据这个时间戳将数据区分为顺序数据和乱序数据，不同种类的数据通过不同的 memtable 和 TsFile 管理。
+
+每个数据文件 TsFile 在内存中对应一个文件索引信息 TsFileResource，供查询使用。
+
+此外，存储引擎还包括异步持久化和文件合并机制。
+
+## 写入流程
+
+### 相关代码
+
+* org.apache.iotdb.db.engine.StorageEngine
+
+	负责一个 IoTDB 实例的写入和访问，管理所有的 StorageGroupProsessor。
+	
+* org.apache.iotdb.db.engine.storagegroup.StorageGroupProcessor
+
+	负责一个存储组一个时间分区内的数据写入和访问。管理所有分区的TsFileProcessor。
+
+* org.apache.iotdb.db.engine.storagegroup.TsFileProcessor
+
+	负责一个 TsFile 文件的数据写入和访问。
+	
+	
+### 单行数据（一个设备一个时间戳多个值）写入
+
+* 对应的接口
+	* JDBC 的 execute 和 executeBatch 接口
+	* Session 的 insert 和 insertInBatch
+
+* 总入口: public void insert(InsertPlan insertPlan)
+	* 找到对应的 StorageGroupProsessor
+	* 根据写入数据的时间以及当前设备落盘的最后时间戳，找到对应的 TsFileProcessor
+	* 写入 TsFileProcessor 对应的 memtable 中
+	* 记录写前日志
+	* 根据 memtable 大小，来判断是否触发异步持久化 memtable 操作
+	* 根据当前磁盘 TsFile 的大小，判断是否触发文件关闭操作
+
+### 批量数据（一个设备多个时间戳多个值）写入
+
+* 对应的接口
+	* Session 的 insertBatch
+
+* 总入口: public Integer[] insertBatch(BatchInsertPlan batchInsertPlan)
+	* 找到对应的 StorageGroupProsessor
+	* 根据这批数据的时间以及当前设备落盘的最后时间戳，将这批数据分成小批，分别对应到一个 TsFileProcessor 中
+	* 分别将每小批写入 TsFileProcessor 对应的 memtable 中
+	* 记录写前日志
+	* 根据 memtable 大小，来判断是否触发异步持久化 memtable 操作
+	* 根据当前磁盘 TsFile 的大小，判断是否触发文件关闭操作
+
+
+## 数据访问
+
+* 总入口（StorageEngine）: public QueryDataSource query(SingleSeriesExpression seriesExpression, QueryContext context,
+      QueryFileManager filePathsManager)
+      
+	* 找到所有包含这个时间序列的顺序和乱序的 TsFileResource 进行返回，供查询引擎使用。
+
+## 相关文档
+
+* [写前日志 (WAL)](2-WAL.md)
+
+* [memtable 持久化](3-FlushManager.md)
+
+* [文件合并机制](4-MergeManager.md)

docs/Documentation-CHN/SystemDesign/4-StorageEngine/2-WAL.md

@@ -0,0 +1,26 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 写前日志
+
+## 相关代码
+
+* org.apache.iotdb.db.writelog.*

docs/Documentation-CHN/SystemDesign/4-StorageEngine/3-FlushManager.md

@@ -0,0 +1,84 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# Flush Memtable
+
+## 设计思想
+
+内存缓冲区 memtable 达到一定阈值后，会交给 FlushManager 进行异步的持久化，不阻塞正常写入。持久化的过程采用流水线的方式。
+
+## 相关代码
+
+* org.apache.iotdb.db.engine.flush.FlushManager
+
+	Memtable 的 Flush 任务管理器。
+	
+* org.apache.iotdb.db.engine.flush.MemtableFlushTask
+
+	负责持久化一个 Memtable。
+
+## FlushManager: 持久化管理器
+
+FlushManager 可以接受 memtable 的持久化任务，提交者有两个，第一个是 TsFileProcessor，第二个是持久化子线程 FlushThread。
+
+每个 TsFileProcessor 同一时刻只会有一个 flush 任务执行，一个 TsFileProcessor 可能对应多个需要持久化的 memtable
+
+## MemTableFlushTask: 持久化任务
+
+<img style="width:100%; max-width:800px; max-height:600px; margin-left:auto; margin-right:auto; display:block;" src="https://user-images.githubusercontent.com/19167280/73625254-03fe2680-467f-11ea-8197-115f3a749cbd.png">
+
+背景：每个 memtable 可包含多个 device，每个 device 可包含多个 measurement。
+
+### 三个线程
+
+一个 memtable 的持久化的过程有三个线程，只有当所有任务都完成后，主线程工作才结束。
+
+* MemTableFlushTask 所在线程
+	
+	持久化主线程兼排序线程，负责给每个 measurement 对应的 chunk 排序。
+
+* encodingTask 线程
+
+	编码线程，负责给每个 Chunk 进行编码，编码成字节数组。
+	
+* ioTask 线程
+
+	IO线程，负责将编码好的 Chunk 持久化到磁盘的 TsFile 文件上。
+
+### 两个任务队列
+
+三个线程之间通过两个任务队列交互
+
+* encodingTaskQueue: 排序线程->编码线程，包括三种任务
+	
+	* StartFlushGroupIOTask：开始持久化一个 device (ChunkGroup)， encoding 不处理这个命令，直接发给 IO 线程。
+	
+	* Pair\<TVList, MeasurementSchema\>：编码一个 Chunk
+	
+	* EndChunkGroupIoTask：结束一个 device (ChunkGroup) 的持久化，encoding 不处理这个命令，直接发给 IO 线程。
+
+* ioTaskQueue: 编码线程->IO线程，包括三种任务
+	
+	* StartFlushGroupIOTask：开始持久化一个 device (ChunkGroup)。
+	
+	* IChunkWriter：持久化一个 Chunk 到磁盘上
+	
+	* EndChunkGroupIoTask：结束一个 device (ChunkGroup) 的持久化。

docs/Documentation-CHN/SystemDesign/4-StorageEngine/4-MergeManager.md

@@ -0,0 +1,26 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 文件合并机制
+
+## 相关代码
+
+* org.apache.iotdb.db.engine.merge.*

docs/Documentation-CHN/SystemDesign/5-DataQuery/1-DataQuery.md

@@ -0,0 +1,67 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# 数据查询
+
+* 查询种类
+	* 原始数据查询
+	* 聚合查询
+	* 降采样查询
+	* 单点补空值查询
+	* 最新数据查询等
+	
+## 查询计划
+
+主要包括两种查询框架：
+
+* 不带值过滤条件的（如，select f1, f2, f3 from root.d1）
+	
+	可以按照多路归并的方式进行查询，分别查询三列的结果，最后按照时间进行对齐。
+	
+* 带值过滤条件的（如，select f1, f2, f3 from root.d1 where f1 = 1 and f2 = 2）
+
+	先根据过滤条件生成结果集的时间戳，再根据时间戳查询其他列的值。
+
+## 查询组件
+
+为了实现任意过滤条件下的多种查询，有以下查询组件
+
+* SeriesReaderWithoutValueFilter
+
+	一个时间序列的 reader，不带任何值过滤条件，但是可能带时间过滤条件。
+
+* SeriesReaderWithValueFilter
+
+	一个时间序列的 reader，带值过滤条件（也可能包含时间过滤条件）。
+
+* SeriesReaderByTimestamp
+
+	一个时间序列的 reader，每给一个时间戳，返回此时间戳对应的值，若没有则返回空。
+
+* TimeGenerator
+	* 功能：根据过滤条件构造表达式的树形结构，生成结果集的时间戳，如 f1 = 1 and f2 = 2，构造一棵树 AND（leaf(f1=1), leaf(f2=2)）。
+	* 叶子节点：对应一个 SeriesReaderWithValueFilter，返回满足条件的结果的时间戳。
+	* 中间节点：按照此节点的谓词逻辑对时间戳做计算。
+	* 根节点：返回最终结果集的时间戳。
+
+## AbstractSeriesReader
+
+单个时间序列的基础 Reader 类。输入为一个 QueryDataSource，包括顺序和乱序文件列表。

docs/Documentation/SystemDesign/0-Content.md

@@ -0,0 +1,42 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+
+# Chapter 0: Architecture
+* 1-Architecture
+# Chapter 1: TsFile
+* 1-TsFile
+* 2-Format
+* 3-Write
+* 4-Read
+# Chapter 2: QueryEngine
+* 1-QueryEngine
+* 2-Planner
+* 3-PlanExecutor
+# Chapter 3: SchemaManager
+* 1-SchemaManager
+# Chapter 4: StorageEngine
+* 1-StorageEngine
+* 2-WAL
+* 3-FlushManager
+* 4-MergeManager
+# Chapter 5: DataQuery
+* 1-DataQuery

docs/Documentation/SystemDesign/1-TsFile/1-TsFile.md

@@ -0,0 +1,30 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# TsFile
+
+TsFile is the data file of IoTDB.
+
+## Related
+
+* [File Format](2-Format.md)
+* [Write Process](3-Write.md)
+* [Read Process](4-Read.md)

docs/Documentation/SystemDesign/1-TsFile/2-Format.md

@@ -7,9 +7,9 @@
     to you under the Apache License, Version 2.0 (the
     "License"); you may not use this file except in compliance
     with the License.  You may obtain a copy of the License at
-    
+
         http://www.apache.org/licenses/LICENSE-2.0
-    
+
     Unless required by applicable law or agreed to in writing,
     software distributed under the License is distributed on an
     "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
@@ -19,7 +19,7 @@
 
 -->
 
-# Chapter 7: System Design
+# TsFile Format
 
 ## 1. TsFile Design
 

docs/Documentation/SystemDesign/1-TsFile/3-Write.md

@@ -0,0 +1,22 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# TsFile Write Process

docs/Documentation/SystemDesign/1-TsFile/4-Read.md

@@ -0,0 +1,22 @@
+<!--
+
+    Licensed to the Apache Software Foundation (ASF) under one
+    or more contributor license agreements.  See the NOTICE file
+    distributed with this work for additional information
+    regarding copyright ownership.  The ASF licenses this file
+    to you under the Apache License, Version 2.0 (the
+    "License"); you may not use this file except in compliance
+    with the License.  You may obtain a copy of the License at
+
+        http://www.apache.org/licenses/LICENSE-2.0
+
+    Unless required by applicable law or agreed to in writing,
+    software distributed under the License is distributed on an
+    "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+    KIND, either express or implied.  See the License for the
+    specific language governing permissions and limitations
+    under the License.
+
+-->
+
+# TsFile Read Process

docs/Documentation/UserGuide/0-Content.md

@@ -65,8 +65,7 @@
 * 3-Spark TsFile
 * 4-Spark IoTDB
 * 5-Hive TsFile
-# Chapter 8: System Design (Developer)
-* 1-Hierarchy
-* 2-Files
-* 3-Writing Data on HDFS
-* 4-Shared Nothing Cluster
+# Chapter 8: Architecture
+* 1-Files
+* 2-Writing Data on HDFS
+* 3-Shared Nothing Cluster

pom.xml

@@ -507,6 +507,10 @@
                             <exclude>licenses/*</exclude>
                             <!-- generated by Github -->
                             <exclude>.github/**</exclude>
+                            <!-- figures -->
+                            <exclude>**/.eps</exclude>
+                            <exclude>**/.png</exclude>
+                            <exclude>**/.jpg</exclude>
                             <!--Generated by Apache Release -->
                             <exclude>local-snapshots-dir/**</exclude>
                             <!-- JSON can't contain comments and therefore no Apache header -->