gogoWebsite

es series navigation

Elasticsearch first article: Basic introduction
Elasticsearch Chapter 2: es version comparison
Elasticsearch Chapter 3: Comparison of the three pagination methods of es and their advantages and disadvantages
Elasticsearch Chapter 4: Java implements Scroll paging, and contains code copying ready-to-use

1. Preface

When learning Elasticsearch, I was not sure about the problems of each version, which was very troublesome. The official also gave the updates of each version. However, it was in the English version, and the version update information was very large. Recently, I have read a lot of information, but none of them were sorted out clearly. Then I sorted it out myself. First, I declare that the following sortings are all important points for each version, because there are too many contents for each major version, and I cannot give examples one by one. For details, you need to refer to the official document. There is a link at the bottom of the article. I also want to understand the iterations of various versions of Elasticsearch as a whole, so that I can better understand and use Elasticsearch products. So I have this article.

2. Version number

Initial version 0.7

Released on May 14, 2010, the first version that can query the release information, important features:

• Zen Discovery Automatic Discovery Module
• Groovy Client Support
• Simple plug-in management mechanism
• Better support for ICU word segmentation
• More management APIs
• I won’t introduce the initialized version for the time being, so I will do it for the first time.

Upgrade version 1.0.0

Release on February 14, 2014, Important Features: -Snapshot/Restore API Backup and Restore API

• Support Aggregations
• CAT API Support
• Support alliance query
• Circuit breaker support
• Doc values ​​introduction

Version 2.0.0

Released on October 28, 2015, Important Features:

• Added pipeline Aggregations
• query/filter query merge, all merge into query, and different queries are executed according to different contexts.
• Storage compression configurable
• The Rivers module was removed
• Multicast multicast discovery is removed and becomes a plug-in. The production environment must configure a unicast address.

New Feature Version 5.0.0

Released on October 26, 2016, Important Features:

• Lucene supports half of disk space; half of index time; 25% increase in query performance; IPV6 support.
• Internal engine level removes competition locks to avoid concurrent updates of the same document, bringing a 15%-20% performance improvement
• Shrink API, it can shrink the number of shards into its factors. For example, before you had 15 shards, you can shrink into 5, 3 or 1. Then we can imagine it as a scenario where in the collection stage with very high writing pressure, set enough indexes to make full use of shard's parallel writing ability, shrink into fewer shards after the index is written, improve query performance.
• Provides the first Java native REST client SDK
  IngestNode, if the data was processed before, it was processed before the index. For example, logstash can structure and convert the logs, and now it can be processed directly in es.
• Provides Painless scripts instead of Groovy scripts
• Remove site plugins, which means that head and bigdesk cannot be installed directly in es, but you can deploy independent sites (all static files anyway) or develop kibana plug-in
• A new Sliced ​​Scroll type has been added. Now the Scroll interface can be used to traverse data concurrently. Each Scroll request can be divided into multiple Slice requests, which can be understood as slices. Each Slice is in parallel independently, and it is many times faster to reconstruct or traverse using Scroll.
• Added Profile API
• Added Rollover API
• Added Reindex
• Provides the first Java native REST client SDK. The HTTP protocol-based client decoupling of Elasticsearch dependency, without jar packet conflicts, provides automatic discovery of cluster nodes, log processing, and automatic request polling for node request failure, fully exerting the high availability capabilities of Elasticsearch
• Introduce a new field type Text/Keyword to replace String
• Limit the index request size to avoid overwhelming a large number of concurrent requests
• Limit the number of shards for a single request, default to 1000

New Feature Version 6.0.0

Released on August 31, 2017, Important Features:

• Sparseness Doc Values ​​support
• Index sorting, that is, the sorting of the index stage.
• Support for sequence numbers, each es operation has a sequence number (similar to incremental design)
  Seamless scrolling upgrade
• Removal of types, in 6.0, there are multiple types in one index at the beginning.
• Index-template inheritance, the inheritance of the index version. Currently, all matching index templates will be merged, which will cause some conflict problems in the index template. 6.0 will only match one, and it will also be verified when the index is created.
• Load aware shard routing, load-based request routing, current search requests are full-node polling, so the slowest nodes often cause overall delay increase. The new implementation method will automatically adjust the queue length based on the time-consuming queue of queues. The queue length of nodes with high load will be reduced, allowing other nodes to share more pressure, and search and index will be based on this mechanism.
• The closed index will also support the automatic processing of replica to ensure the data is reliable.

New Feature Version 7.0.0

Released on April 10, 2019, Important Features:

• Cluster connection changes: TransportClient is abandoned so that es7's java code can only use restclient. Then, I have summarized it. For Java programming, it is recommended to use the High-level-rest-client method to operate the ES cluster.

• ES packages are packaged with jdk by default: so that the package size of the version is suddenly 300MB+. Compared to the comparison, it is found that the package is 200MB+, which is exactly the size of the JDK

• Lucene9.0

• Major improvements - officially abolishing the support for multiple types under a single index. When es6 was es6, the official mentioned that es7 would delete the type, and when es6, it stipulated that each index could only have one type. Use the default _doc as type in es7, and the official said that the type will be completely removed in the version. Api request method also sends changes, such as obtaining a document with an ID of a certain index: GET index/_doc/id where index and id are specific values

• Starting from 7.1, Security function is free to use

• ECK-ElasticSearch Operator on Kubernetes

• Introduced a true memory circuit breaker that detects unprocessed requests more accurately and prevents them from making individual nodes unstable

• Zen2 is Elasticsearch's new cluster coordination layer that improves reliability, performance and user experience, making it faster, safer, and easier to use
  New Features
  item
  Cluster coordination
  - Complete High Level REST Client
  Score Query

• Performance optimization
  -AND algorithm improves query performance
  2. The default Primary Shared number is changed from 5 to 1 to avoid Over Sharding
  3. Faster first k queries
  4. Intervals queries Some search use cases (e.g., legal and patent searches) introduce the need to find records where words or phrases are a distance away from each other. Interval queries in Elasticsearch 7.0 introduce a completely new way to build such queries, which is easier to use and define than previous methods (span queries). Compared with span query, interval query is more adaptable to edge cases.

Author: Flying Stone
Source:/flyrock/