Hive優(yōu)化之一fetch task妆棒。
優(yōu)化場景幻锁，
1川抡、當在hive中執(zhí)行select * from emp全部查詢時及過濾屬性字段
2辐真、當在hive中執(zhí)行分區(qū)查詢時
3、當查詢前10/20筆數(shù)據(jù)這樣的LIMIT
執(zhí)行以上操作時不執(zhí)行MapReduce
優(yōu)化方法
在hive-site.xml中添加 hive.fetch.task.conversion 配置,見下面的描述
<property>
<name>hive.fetch.task.conversion</name>
<value>more</value>
<description>
Some select queries can be converted to single FETCH task minimizing latency.
Currently the query should be single sourced not having any subquery and should not have
any aggregations or distincts (which incurs RS), lateral views and joins.
1. minimal : SELECT STAR, FILTER on partition columns, LIMIT only
2. more : SELECT, FILTER, LIMIT only (TABLESAMPLE, virtual columns)
</description>
</property>
測試優(yōu)化方法
退出hive命令行崖堤，重新進入命令行侍咱，因為hive-site.xml配置文件中添加了屬性配置。
前后比較
沒加配置前
加配置后
其他測試
