peihaoyu/Echo

Fork 0

Veal98 db263d3925 Complete search funtion

2021-01-29 16:59:46 +08:00

20 KiB

Raw Blame History

搜索

想要在 Elasticsearch 中搜索数据，需要把数据在 Elasticsearch 中再存一份

一个索引对应一个数据库（7.0 中索引对应表），一个类型对应一张表（7.0 已废弃该属性），一个文档对应表中的一行，一个字段对应表中的一列

ElasticSearch 下载安装

注意，下载 ElasticSearch 版本一定要与你的 SpringBoot 版本内部规定的一致，我的是 SpringBoot 2.1.5

Elasticsearch 6.4.3 下载地址

解压完毕后，需要配置一下：config/elasticsearch.yml

配到环境变量中去：

还需要安装一个中文分词插件（Elasticsearch 自带一个英文分词插件）elasticsearch-analysis-ik 6.4.3 下载地址

注意：必须解压到你的 elasticsearch 安装目录的 plugins/ik 文件夹下（D:\elasticsearch-6.4.3\plugins\ik）

启动 elasticsearch

常用命令：

curl -X PUT "localhost:9200/test" 创建索引
	
curl -X GET "localhost:9200/_cat/indices?v" 查看索引

curl -X DELETE "localhost:9200/test" 删除索引

可以使用 Postman 简化命令行的操作，比如创建索引：

增加一条数据：

查询：

删除：

搜索：搜索 title 中包含“互联网”的内容

复杂搜索：搜索 title 或 content 中包含 “互联网” 的内容

Spring Boot 整合 Elasticsearch

引入依赖

<!--Elasticsearch-->
<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-data-elasticsearch</artifactId>
</dependency>

配置 Elasticsearch

cluster-name
cluster-nodes

# Elasticsearch
# 该字段见 Elasticsearch 安装包中的 elasticsearch.yml，可自行修改
spring.data.elasticsearch.cluster-name = community
spring.data.elasticsearch.cluster-nodes = 127.0.0.1:9300

解决 Elasticsearch 和 Redis 底层的 Netty 启动冲突问题

@SpringBootApplication
public class CommunityApplication {

    /**
     * 解决 Elasticsearch 和 Redis 底层的 Netty 启动冲突问题
     */
    @PostConstruct
    public void init() {
        System.setProperty("es.set.netty.runtime.available.processors", "false");
    }

    public static void main(String[] args) {
        SpringApplication.run(CommunityApplication.class, args);
    }

}

Spring Data Elasticsearch

ElasticsearchTemplate
ElasticsearchRepository（推荐）

首先，将实体类和 Elasticsearch 之间建立联系：

@Document(indexName = "discusspost", type = "_doc", shards = 6, replicas = 3)
public class DiscussPost {

type 类型在 elasticsearch 7 版本中已完全抛弃，6 也不太建议使用，所以我们就固定为 _doc，不用去管它

@Document(indexName = "discusspost", type = "_doc", shards = 6, replicas = 3)
public class DiscussPost {

    @Id
    private int id;

    @Field(type = FieldType.Integer)
    private int userId;

    @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String title;

    @Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")
    private String content;

    @Field(type = FieldType.Integer)
    private int type;

    @Field(type = FieldType.Integer)
    private int status;

    @Field(type = FieldType.Date)
    private Date createTime;

    @Field(type = FieldType.Integer)
    private int commentCount;

    @Field(type = FieldType.Double)
    private double score;

测试增删改查：

@RunWith(SpringRunner.class)
@ContextConfiguration(classes = CommunityApplication.class)
@SpringBootTest
public class ElasticsearchTests {

    @Autowired
    private DiscussPostMapper discussPostMapper;

    @Autowired
    private DiscussPostRepository discussPostRepository;

    @Autowired
    private ElasticsearchTemplate elasticsearchTemplate;

    /**
     * 测试插入数据
     */
    @Test
    public void testInsert() {
        discussPostRepository.save(discussPostMapper.selectDiscussPostById(241));
        discussPostRepository.save(discussPostMapper.selectDiscussPostById(242));
        discussPostRepository.save(discussPostMapper.selectDiscussPostById(243));
    }

    /**
     * 测试批量插入数据
     */
    @Test
    public void testInsetList() {
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(101, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(102, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(103, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(111, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(112, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(131, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(132, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(133, 0, 100));
        discussPostRepository.saveAll(discussPostMapper.selectDiscussPosts(134, 0, 100));
    }

    /**
     * 测试修改数据
     */
    @Test
    public void testUpdate() {
        DiscussPost discussPost = discussPostMapper.selectDiscussPostById(231);
        discussPost.setContent("Great Elasticsearch");
        discussPostRepository.save(discussPost);
    }

    /**
     * 测试删除数据（注意数据库中的数据并未被删除）
     */
    @Test
    public void testDelete() {
        discussPostRepository.deleteById(231);
        // discussPostRepository.deleteAll();
    }

    /**
     * 测试使用 ElasticsearchRepository 进行搜索
     */
    @Test
    public void testSearchByRepository() {
        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(QueryBuilders.multiMatchQuery("互联网寒冬", "title", "content"))
                .withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC))
                .withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC))
                .withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC))
                .withPageable(PageRequest.of(0, 10))
                .withHighlightFields(
                        new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),
                        new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")
                ).build();

        // elasticsearchTemplate.queryForPage(searchQuery, class, SearchResultMapper);
        // 底层获取到了高亮显示的值，但是没有做处理（所以想要更加完善的话需要使用 ElasticsearchTemplate)

        Page<DiscussPost> page = discussPostRepository.search(searchQuery);

        System.out.println(page.getTotalElements());
        System.out.println(page.getTotalPages());
        System.out.println(page.getNumber());
        System.out.println(page.getSize());
        for (DiscussPost post : page) {
            System.out.println(post);
        }
    }

    /**
     * 测试使用 ElasticsearchTemplate 进行搜索
     */
    @Test
    public void testSearchTemplate() {
        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(QueryBuilders.multiMatchQuery("互联网寒冬", "title", "content"))
                .withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC))
                .withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC))
                .withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC))
                .withPageable(PageRequest.of(0, 10))
                .withHighlightFields(
                        new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),
                        new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")
                ).build();

        Page<DiscussPost> page = elasticsearchTemplate.queryForPage(searchQuery, DiscussPost.class, new SearchResultMapper() {
            @Override
            public <T> AggregatedPage<T> mapResults(SearchResponse searchResponse, Class<T> aClass, Pageable pageable) {
                SearchHits hits = searchResponse.getHits();
                if (hits.getTotalHits() <= 0) {
                    return null;
                }

                List<DiscussPost> list = new ArrayList<>();

                for (SearchHit hit : hits) {
                    DiscussPost post = new DiscussPost();

                    String id = hit.getSourceAsMap().get("id").toString();
                    post.setId(Integer.valueOf(id));

                    String userId = hit.getSourceAsMap().get("userId").toString();
                    post.setUserId(Integer.valueOf(userId));

                    String title = hit.getSourceAsMap().get("title").toString();
                    post.setTitle(title);

                    String content = hit.getSourceAsMap().get("content").toString();
                    post.setContent(content);

                    String status = hit.getSourceAsMap().get("status").toString();
                    post.setStatus(Integer.valueOf(status));

                    String createTime = hit.getSourceAsMap().get("createTime").toString();
                    post.setCreateTime(new Date(Long.valueOf(createTime)));

                    String commentCount = hit.getSourceAsMap().get("commentCount").toString();
                    post.setCommentCount(Integer.valueOf(commentCount));

                    // 处理高亮显示的内容
                    HighlightField titleField = hit.getHighlightFields().get("title");
                    if (titleField != null) {
                        post.setTitle(titleField.getFragments()[0].toString());
                    }

                    HighlightField contentField = hit.getHighlightFields().get("content");
                    if (contentField != null) {
                        post.setContent(contentField.getFragments()[0].toString());
                    }

                    list.add(post);
                }

                return new AggregatedPageImpl(list, pageable,
                        hits.getTotalHits(), searchResponse.getAggregations(), searchResponse.getScrollId(), hits.getMaxScore());
    }
});

        System.out.println(page.getTotalElements());
        System.out.println(page.getTotalPages());
        System.out.println(page.getNumber());
        System.out.println(page.getSize());
        for (DiscussPost post : page) {
            System.out.println(post);
        }
    }

}

🚨 注意这里面的 Page 是 Spring 提供的，而非我们自己开发的那个，它的 current 当前页码是从 0 开始计数的，而我们自己开发的那个 Page 是从 1 开始的，所以后续编码的时候记得稍微处理一下。

开发社区搜索功能

使用消息队列异步地（提高性能）将帖子提交到 Elasticsearch 提交到服务器

Service

/**
 * 搜索相关
 */
@Service
public class ElasticsearchService {

    @Autowired
    private DiscussPostRepository discussPostRepository;

    @Autowired
    private ElasticsearchTemplate elasticsearchTemplate;

    /**
     * 将数据插入 Elasticsearch 服务器
     * @param post
     */
    public void saveDiscusspost(DiscussPost post) {
        discussPostRepository.save(post);
    }

    /**
     * 将数据从 Elasticsearch 服务器中删除
     * @param id
     */
    public void deleteDiscusspost(int id) {
        discussPostRepository.deleteById(id);
    }

    /**
     * 搜索
     * @param keyword 搜索的关键词
     * @param current 当前页码（这里的 Page 是 Spring 提供的，而非我们自己实现的那个）
     * @param limit 每页显示多少条数据
     * @return
     */
    public Page<DiscussPost> searchDiscussPost(String keyword, int current, int limit) {
        SearchQuery searchQuery = new NativeSearchQueryBuilder()
                .withQuery(QueryBuilders.multiMatchQuery(keyword, "title", "content"))
                .withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC))
                .withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC))
                .withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC))
                .withPageable(PageRequest.of(current, limit))
                .withHighlightFields(
                        new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),
                        new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")
                ).build();

        return elasticsearchTemplate.queryForPage(searchQuery, DiscussPost.class, new SearchResultMapper() {
            @Override
            public <T> AggregatedPage<T> mapResults(SearchResponse searchResponse, Class<T> aClass, Pageable pageable) {
                // 获取命中的数据
                SearchHits hits = searchResponse.getHits();
                if (hits.getTotalHits() <= 0) {
                    return null;
                }

                // 处理命中的数据
                List<DiscussPost> list = new ArrayList<>();
                for (SearchHit hit : hits) {
                    DiscussPost post = new DiscussPost();

                    String id = hit.getSourceAsMap().get("id").toString();
                    post.setId(Integer.valueOf(id));

                    String userId = hit.getSourceAsMap().get("userId").toString();
                    post.setUserId(Integer.valueOf(userId));

                    String title = hit.getSourceAsMap().get("title").toString();
                    post.setTitle(title);

                    String content = hit.getSourceAsMap().get("content").toString();
                    post.setContent(content);

                    String status = hit.getSourceAsMap().get("status").toString();
                    post.setStatus(Integer.valueOf(status));

                    String createTime = hit.getSourceAsMap().get("createTime").toString();
                    post.setCreateTime(new Date(Long.valueOf(createTime)));

                    String commentCount = hit.getSourceAsMap().get("commentCount").toString();
                    post.setCommentCount(Integer.valueOf(commentCount));

                    // 处理高亮显示的内容
                    HighlightField titleField = hit.getHighlightFields().get("title");
                    if (titleField != null) {
                        post.setTitle(titleField.getFragments()[0].toString());
                    }

                    HighlightField contentField = hit.getHighlightFields().get("content");
                    if (contentField != null) {
                        post.setContent(contentField.getFragments()[0].toString());
                    }

                    list.add(post);
                }

                return new AggregatedPageImpl(list, pageable,
                        hits.getTotalHits(), searchResponse.getAggregations(), searchResponse.getScrollId(), hits.getMaxScore());
            }
        });

    }

}

Controller

DiscusspostController

发帖成功后，通过消息队列将该帖子存入 Elasticsearch 服务器
CommentController

对帖子添加评论成功后，通过消息队列将该帖子存入 Elasticsearch 服务器（不懂为啥要这样做，又不用查询评论）

在消息队列消费者中添加一个消费发帖事件的方法

/**
 * 消费发帖事件
 */
@KafkaListener(topics = {TOPIC_PUBLISH})
public void handlePublishMessage(ConsumerRecord record) {
    if (record == null || record.value() == null) {
        logger.error("消息的内容为空");
        return ;
    }
    Event event = JSONObject.parseObject(record.value().toString(), Event.class);
    if (event == null) {
        logger.error("消息格式错误");
        return ;
    }

    DiscussPost post = discussPostSerivce.findDiscussPostById(event.getEntityId());
    elasticsearchService.saveDiscusspost(post);

}

SearchController

/**
 * 搜索
 */
@Controller
public class SearchController implements CommunityConstant {

    @Autowired
    private ElasticsearchService elasticsearchService;

    @Autowired
    private UserService userService;

    @Autowired
    private DiscussPostSerivce discussPostSerivce;

    @Autowired
    private LikeService likeService;

    // search?keword=xxx
    @GetMapping("/search")
    public String search(String keyword, Page page, Model model) {
        // 搜索帖子 (Spring 提供的 Page 当前页码从 0 开始计数)
        org.springframework.data.domain.Page<DiscussPost> searchResult =
            elasticsearchService.searchDiscussPost(keyword, page.getCurrent()-1, page.getLimit());
        // 聚合数据
        List<Map<String, Object>> discussPosts = new ArrayList<>();
        if (searchResult != null) {
            for (DiscussPost post : searchResult) {
                Map<String, Object> map = new HashMap<>();
                // 帖子
                map.put("post", post);
                // 作者
                map.put("user", userService.findUserById(post.getUserId()));
                // 点赞数量
                map.put("likeCount", likeService.findEntityLikeCount(ENTITY_TYPE_POST, post.getId()));

                discussPosts.add(map);
            }
        }

        model.addAttribute("discussPosts", discussPosts);
        model.addAttribute("keyword", keyword);

        // 设置分页
        page.setPath("/search?keyword="+ keyword);
        page.setRows(searchResult == null ? 0 : (int) searchResult.getTotalElements());

        return "/site/search";
    }

}

前端

index.html 搜索框

<form method="get" th:action="@{/search}">
   <input name="keyword" th:value="${keyword}" />
   <button type="submit"> 搜索</button>
</form>

name = "keyword" 和 Controller 中参数的名称要一致

search.html 搜索详情页

<li th:each="map:${discussPosts}">
   <img th:src="${map.user.headerUrl}" alt="用户头像">
      <a th:href="@{|/discuss/detail/${map.post.id}|}" th:utext="${map.post.title}"></a>
      <div th:utext="${map.post.content}"></div>
      <u th:utext="${map.user.username}"></u>
      发布于 <b th:text="${#dates.format(map.post.createTime, 'yyyy-MM-dd HH:mm:ss')}"></b>
      <ul>
        <li>赞 <i th:text="${map.likeCount}"></i></li>
        <li>|</li>
        <li>回复 <i th:text="${map.post.commentCount}"></i></li>
     </ul>
</li>

20 KiB Raw Blame History Unescape Escape

搜索