內(nèi)容規(guī)劃總共分為三個(gè)章節(jié)來寫次员，分別運(yùn)行環(huán)境構(gòu)建膨处、利用Web應(yīng)用管理索引以及Web應(yīng)用管理數(shù)據(jù)三大塊來說明解幼。
具體有：
一外永、SpringBoot集成Elasticsearch7.4 實(shí)戰(zhàn)（一）：這些操作都是在CentOS下操作的，主要帶大家熟悉下Elasticsearch環(huán)境匙铡。
- 1.1.下載& Linux下ElasticSearch安裝
- 1.2.中文分詞插件IK
- 1.3.索引
- 1.4.如何數(shù)據(jù)管理
二图甜、SpringBoot集成Elasticsearch7.4 實(shí)戰(zhàn)（二）：在Springboot環(huán)境下，利用JAVA環(huán)境操作索引鳖眼。
- 2.1.新增索引
- 2.2.查詢索引
- 2.3.刪除索引
三黑毅、SpringBoot集成Elasticsearch7.4 實(shí)戰(zhàn)（三）：在Springboot環(huán)境下，管理數(shù)據(jù)钦讳。
- 3.1.WEB HTTP提交數(shù)據(jù)<單條提交矿瘦、批量提交>
- 3.2.WEB HTTP方式條件查詢
- 3.3.WEB HTTP刪除數(shù)據(jù)
elasticsearch
1. 目錄
20191205172258.png
2. SpringBoot集成
開發(fā)工具，這里選擇的是IDEA 2019.2蜂厅，構(gòu)建Maven工程等一堆通用操作匪凡，不清楚的自行百度膊畴。
2.1. POM配置
我這邊選擇 elasticsearch-rest-high-level-client 方式來集成掘猿，發(fā)現(xiàn)這有個(gè)坑，開始沒注意唇跨，踩了好久稠通，就是要排除掉 elasticsearch衬衬、elasticsearch-rest-client ，這里沒有選擇 spring-boot-starter-data-elasticsearch 改橘，因?yàn)樽钚掳娴?starter 現(xiàn)在依然是6.x版本號(hào)滋尉，并沒有集成 elasticsearch7.4.0，導(dǎo)致使用過程中有很多版本沖突飞主，讀者在選擇的時(shí)候多加留意狮惜。
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-high-level-client</artifactId>
<version>7.4.0</version>
<exclusions>
<exclusion>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
</exclusion>
<exclusion>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
</exclusion>
</exclusions>
</dependency>
<dependency>
<groupId>org.elasticsearch.client</groupId>
<artifactId>elasticsearch-rest-client</artifactId>
<version>7.4.0</version>
</dependency>
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>7.4.0</version>
</dependency>
2.2. yml配置
server:
port: 9090
spring:
datasource:
name: mysql
type: com.alibaba.druid.pool.DruidDataSource
driver-class-name: com.mysql.cj.jdbc.Driver
url: jdbc:mysql://127.0.0.1:3306/springboot?useUnicode=true&characterEncoding=utf8&useSSL=false&serverTimezone=UTC
username: root
password: 123456
druid:
initial-size: 5
min-idle: 5
max-active: 20
max-wait: 30000
time-between-eviction-runs-millis: 60000
min-evictable-idle-time-millis: 300000
validation-query: select 1
test-while-idle: true
test-on-borrow: false
test-on-return: false
pool-prepared-statements: false
max-pool-prepared-statement-per-connection-size: 20
connectionProperties: druid.stat.mergeSql=true;druid.stat.slowSqlMillis=6000
es:
host: 192.168.147.132
port: 9200
scheme: http
mybatis:
mapperLocations: classpath:mapper/**/*.xml
這里定義 es 節(jié)點(diǎn)下即 elasticsearch 的地址端口信息，修改為自己的即可碌识。
2.3. 核心操作類
為了規(guī)范索引管理碾篡，這里將所有的操作都封裝成一個(gè)基類，實(shí)現(xiàn)對索引的增刪改查筏餐。同時(shí)還集成了對數(shù)據(jù)的單個(gè)以及批量的插入以及刪除开泽。避免針對每個(gè)索引都自己寫一套實(shí)現(xiàn)，杜絕代碼的冗余魁瞪，同時(shí)這樣的集成對代碼的結(jié)構(gòu)本身也是低侵入性。
package xyz.wongs.weathertop.base.dao;
import com.alibaba.fastjson.JSON;
import lombok.extern.slf4j.Slf4j;
import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest;
import org.elasticsearch.action.bulk.BulkRequest;
import org.elasticsearch.action.delete.DeleteRequest;
import org.elasticsearch.action.index.IndexRequest;
import org.elasticsearch.action.search.SearchRequest;
import org.elasticsearch.action.search.SearchResponse;
import org.elasticsearch.action.support.IndicesOptions;
import org.elasticsearch.client.RequestOptions;
import org.elasticsearch.client.RestHighLevelClient;
import org.elasticsearch.client.indices.CreateIndexRequest;
import org.elasticsearch.client.indices.CreateIndexResponse;
import org.elasticsearch.client.indices.GetIndexRequest;
import org.elasticsearch.common.settings.Settings;
import org.elasticsearch.common.xcontent.XContentType;
import org.elasticsearch.index.query.QueryBuilder;
import org.elasticsearch.index.reindex.DeleteByQueryRequest;
import org.elasticsearch.search.SearchHit;
import org.elasticsearch.search.builder.SearchSourceBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import sun.rmi.runtime.Log;
import xyz.wongs.weathertop.base.entiy.ElasticEntity;
import java.util.ArrayList;
import java.util.Collection;
import java.util.List;
@Slf4j
@Component
public class BaseElasticDao {
@Autowired
RestHighLevelClient restHighLevelClient;
/**
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:30
* @param idxName 索引名稱
* @param idxSQL 索引描述
* @return void
* @throws
* @since
*/
public void createIndex(String idxName,String idxSQL){
try {
if (!this.indexExist(idxName)) {
log.error(" idxName={} 已經(jīng)存在,idxSql={}",idxName,idxSQL);
return;
}
CreateIndexRequest request = new CreateIndexRequest(idxName);
buildSetting(request);
request.mapping(idxSQL, XContentType.JSON);
//request.settings() 手工指定Setting
CreateIndexResponse res = restHighLevelClient.indices().create(request, RequestOptions.DEFAULT);
if (!res.isAcknowledged()) {
throw new RuntimeException("初始化失敗");
}
} catch (Exception e) {
e.printStackTrace();
System.exit(0);
}
}
/** 斷某個(gè)index是否存在
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:27
* @param idxName index名
* @return boolean
* @throws
* @since
*/
public boolean indexExist(String idxName) throws Exception {
GetIndexRequest request = new GetIndexRequest(idxName);
request.local(false);
request.humanReadable(true);
request.includeDefaults(false);
request.indicesOptions(IndicesOptions.lenientExpandOpen());
return restHighLevelClient.indices().exists(request, RequestOptions.DEFAULT);
}
/** 設(shè)置分片
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 19:27
* @param request
* @return void
* @throws
* @since
*/
public void buildSetting(CreateIndexRequest request){
request.settings(Settings.builder().put("index.number_of_shards",3)
.put("index.number_of_replicas",2));
}
/**
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:27
* @param idxName index
* @param entity 對象
* @return void
* @throws
* @since
*/
public void insertOrUpdateOne(String idxName, ElasticEntity entity) {
IndexRequest request = new IndexRequest(idxName);
request.id(entity.getId());
request.source(JSON.toJSONString(entity.getData()), XContentType.JSON);
try {
restHighLevelClient.index(request, RequestOptions.DEFAULT);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/** 批量插入數(shù)據(jù)
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:26
* @param idxName index
* @param list 帶插入列表
* @return void
* @throws
* @since
*/
public void insertBatch(String idxName, List<ElasticEntity> list) {
BulkRequest request = new BulkRequest();
list.forEach(item -> request.add(new IndexRequest(idxName).id(item.getId())
.source(JSON.toJSONString(item.getData()), XContentType.JSON)));
try {
restHighLevelClient.bulk(request, RequestOptions.DEFAULT);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/** 批量刪除
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:14
* @param idxName index
* @param idList 待刪除列表
* @return void
* @throws
* @since
*/
public <T> void deleteBatch(String idxName, Collection<T> idList) {
BulkRequest request = new BulkRequest();
idList.forEach(item -> request.add(new DeleteRequest(idxName, item.toString())));
try {
restHighLevelClient.bulk(request, RequestOptions.DEFAULT);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/**
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:14
* @param idxName index
* @param builder 查詢參數(shù)
* @param c 結(jié)果類對象
* @return java.util.List<T>
* @throws
* @since
*/
public <T> List<T> search(String idxName, SearchSourceBuilder builder, Class<T> c) {
SearchRequest request = new SearchRequest(idxName);
request.source(builder);
try {
SearchResponse response = restHighLevelClient.search(request, RequestOptions.DEFAULT);
SearchHit[] hits = response.getHits().getHits();
List<T> res = new ArrayList<>(hits.length);
for (SearchHit hit : hits) {
res.add(JSON.parseObject(hit.getSourceAsString(), c));
}
return res;
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/** 刪除index
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:13
* @param idxName
* @return void
* @throws
* @since
*/
public void deleteIndex(String idxName) {
try {
if (!this.indexExist(idxName)) {
log.error(" idxName={} 已經(jīng)存在",idxName);
return;
}
restHighLevelClient.indices().delete(new DeleteIndexRequest(idxName), RequestOptions.DEFAULT);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
/**
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:13
* @param idxName
* @param builder
* @return void
* @throws
* @since
*/
public void deleteByQuery(String idxName, QueryBuilder builder) {
DeleteByQueryRequest request = new DeleteByQueryRequest(idxName);
request.setQuery(builder);
//設(shè)置批量操作數(shù)量,最大為10000
request.setBatchSize(10000);
request.setConflicts("proceed");
try {
restHighLevelClient.deleteByQuery(request, RequestOptions.DEFAULT);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
}
3. 實(shí)戰(zhàn)
通過以上的集成，我們看到完成在項(xiàng)目中對 elasticsearch 的集成恳守，同時(shí)也用基類驳概，將所有可能的操作都封裝起來。下來我們通過對基類的講解旅薄，來逐個(gè)說明贡歧！
3.1. 索引管理
由于在BaseElasticDao類中createIndex方法，我在Controller層將索引名稱和索引SQL封裝過赋秀，詳細(xì)見Github演示源碼 中xyz.wongs.weathertop.palant.vo.IdxVo
3.1.1. 創(chuàng)建索引
我們在創(chuàng)建索引過程中需要先判斷是否有這個(gè)索引利朵，否則不允許創(chuàng)建，由于我案例采用的是手動(dòng)指定indexName和Settings猎莲，大家看的過程中要特別注意下绍弟，而且還有一點(diǎn)indexName必須是小寫，如果是大寫在創(chuàng)建過程中會(huì)有錯(cuò)誤
官方索引創(chuàng)建說明
索引名大寫
著洼。詳細(xì)的代碼實(shí)現(xiàn)見如下：
/**
* @Description 創(chuàng)建Elastic索引
* @param idxVo
* @return xyz.wongs.weathertop.base.message.response.ResponseResult
* @throws
* @date 2019/11/19 11:07
*/
@RequestMapping(value = "/createIndex",method = RequestMethod.POST)
public ResponseResult createIndex(@RequestBody IdxVo idxVo){
ResponseResult response = new ResponseResult();
try {
//索引不存在樟遣，再創(chuàng)建，否則不允許創(chuàng)建
if(!baseElasticDao.indexExist(idxVo.getIdxName())){
String idxSql = JSONObject.toJSONString(idxVo.getIdxSql());
log.warn(" idxName={}, idxSql={}",idxVo.getIdxName(),idxSql);
baseElasticDao.createIndex(idxVo.getIdxName(),idxSql);
} else{
response.setStatus(false);
response.setCode(ResponseCode.DUPLICATEKEY_ERROR_CODE.getCode());
response.setMsg("索引已經(jīng)存在身笤，不允許創(chuàng)建");
}
} catch (Exception e) {
response.setStatus(false);
response.setCode(ResponseCode.ERROR.getCode());
response.setMsg(ResponseCode.ERROR.getMsg());
}
return response;
}
創(chuàng)建索引需要設(shè)置分片豹悬，這里采用Settings.Builder方式，當(dāng)然也可以JSON自定義方式液荸，本文篇幅有限瞻佛，不做演示。查看xyz.wongs.weathertop.base.service.BaseElasticService.buildSetting方法，這里是默認(rèn)值伤柄。
index.number_of_shards：分片數(shù)
number_of_replicas：副本數(shù)
/** 設(shè)置分片
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 19:27
* @param request
* @return void
* @throws
* @since
*/
public void buildSetting(CreateIndexRequest request){
request.settings(Settings.builder().put("index.number_of_shards",3)
.put("index.number_of_replicas",2));
這時(shí)候我們通過Postman工具調(diào)用Controller绊困，發(fā)現(xiàn)創(chuàng)建索引成功。
新增索引
再命令行執(zhí)行curl -H "Content-Type: application/json" -X GET "http://localhost:9200/_cat/indices?v"适刀，效果如圖：
[elastic@localhost elastic]$ curl -H "Content-Type: application/json" -X GET "http://localhost:9200/_cat/indices?v"
health status index uuid pri rep docs.count docs.deleted store.size pri.store.size
yellow open twitter scSSD1SfRCio4F77Hh8aqQ 3 2 2 0 8.3kb 8.3kb
yellow open idx_location _BJ_pOv0SkS4tv-EC3xDig 3 2 1 0 4kb 4kb
yellow open wongs uT13XiyjSW-VOS3GCqao8w 3 2 1 0 3.4kb 3.4kb
yellow open idx_locat Kr3wGU7JT_OUrRJkyFSGDw 3 2 3 0 13.2kb 13.2kb
yellow open idx_copy_to HouC9s6LSjiwrJtDicgY3Q 3 2 1 0 4kb 4kb
說明創(chuàng)建成功秤朗，這里總是通過命令行來驗(yàn)證，有點(diǎn)繁瑣笔喉，既然我都有WEB服務(wù)取视，為什么不直接通過HTTP驗(yàn)證了？
3.1.2. 查看索引
我們寫一個(gè)對外以HTTP+GET方式對外提供查詢的服務(wù)常挚。存在為TRUE贫途，否則False.
/**
* @Description 判斷索引是否存在；存在-TRUE待侵，否則-FALSE
* @param index
* @return xyz.wongs.weathertop.base.message.response.ResponseResult
* @throws
* @date 2019/11/19 18:48
*/
@RequestMapping(value = "/exist/{index}")
public ResponseResult indexExist(@PathVariable(value = "index") String index){
ResponseResult response = new ResponseResult();
try {
if(!baseElasticDao.isExistsIndex(index)){
log.error("index={},不存在",index);
response.setCode(ResponseCode.RESOURCE_NOT_EXIST.getCode());
response.setMsg(ResponseCode.RESOURCE_NOT_EXIST.getMsg());
} else {
response.setMsg(" 索引已經(jīng)存在, " + index);
}
} catch (Exception e) {
response.setCode(ResponseCode.NETWORK_ERROR.getCode());
response.setMsg(" 調(diào)用ElasticSearch 失敹纭！");
response.setStatus(false);
}
return response;
}
3.1.3. 刪除索引
刪除的邏輯就比較簡單秧倾，這里就不多說怨酝。
/** 刪除index
* @author WCNGS@QQ.COM
* @See
* @date 2019/10/17 17:13
* @param idxName
* @return void
* @throws
* @since
*/
public void deleteIndex(String idxName) {
try {
if (!this.indexExist(idxName)) {
log.error(" idxName={} 已經(jīng)存在",idxName);
return;
}
restHighLevelClient.indices().delete(new DeleteIndexRequest(idxName), RequestOptions.DEFAULT);
} catch (Exception e) {
throw new RuntimeException(e);
}
}
4. 源碼
Github演示源碼 ，記得給Star
Gitee演示源碼那先，記得給Star
5 相關(guān)章節(jié)
一农猬、SpringBoot集成Elasticsearch7.4 實(shí)戰(zhàn)（一）
二、SpringBoot集成Elasticsearch7.4 實(shí)戰(zhàn)（二）
三售淡、SpringBoot集成Elasticsearch7.4 實(shí)戰(zhàn)（三）
