Elasticsearch Reference
  • Introduction
  • 入门
    • 基本概念
    • 安装
    • 探索集群
      • 集群健康
      • 获取所有索引库
      • 创建一个索引库
      • 索引文档创建与查询
      • 删除索引
    • 修改数据
      • 更新文档
      • 删除文档
      • 批量处理
    • 探索您的数据
      • 搜索API
      • 查询语言介绍
      • 执行搜索
      • 执行过滤
      • 执行聚合
    • 总结
  • Elasticsearch设置
    • 安装Elasticsearch
      • .zip或.tar.gz文件的安装方式
      • Debian软件包安装方式
      • RPM安装方式
      • Windows安装方式
      • Docker安装方式
    • 配置Elasticsearch
    • 重要的Elasticsearch配置
    • 安全配置
    • 启动前检查
      • 堆大小检查
      • 文件描述符检查
      • 内存锁定检查
      • 最大线程数检查
      • 最大虚拟内存检查
      • 最大map数检查
      • JVM检查
      • 串行收集使用检查
      • 系统调用过滤检查
      • OnError与OnOutOfMemroyError检查
      • G1GC检查
    • 重要的系统配置
      • 系统设置
      • 在jvm.options中设置JVM堆大小
      • 禁用swapping
      • 文件描述符
      • 虚拟内存
      • 线程数
    • Elasticsearch升级
      • 滚动升级
      • 全集群重启升级
      • 索引重建升级
    • Elasticsearch停机
  • 重大变更
    • 5.3重大变更
    • 5.2重大变更
      • 删除Shadow副本
    • 5.1重大变更
    • 5.0重大变更
      • 搜索与查询DSL变化
      • Mapping变化
      • Percolator变化
      • Suggester变化
      • Index API变化
      • Document API变化
      • Settings变化
      • Allocation变化
      • HTTP变化
      • REST API变化
      • CAT API变化
      • JAVA API变化
      • 安装包
      • 插件变化
      • 文件系统相关变化
      • 磁盘数据文件路径
      • Aggregation变化
      • 脚本相关变化
  • API约定
    • 多索引语法
    • 索引库名称的日期运算
    • 常用选项
    • URL-based访问控制
  • 文档API
    • 读写文档
    • Index API
    • Get API
    • Delete API
    • Delete By Query API
    • Update API
    • Update By Query API
    • Multi Get API
    • Bulk API
    • Reindex API
    • Term Vectors
    • Multi termvectors API
    • refresh
  • 搜索API
    • Search
    • URI Search
    • Request Body Search
      • Query
      • From / Size
      • Sort
      • Source filtering
      • Fields
      • Script Fields
      • Doc value Fields
      • Post filter
      • Highlighting
      • Rescoring
      • Search Type
      • Scroll
      • Preference
      • Explain
      • Version
      • Index Boost
      • min_score
      • Named Queries
      • Inner hits
      • Field Collapsing
      • Search After
    • Search Template
    • Multi Search Template
    • Search Shards API
    • Suggesters
      • Term suggester
      • Phrase Suggester
      • Completion Suggester
      • Context Suggester
    • Multi Search API
    • Count API
    • Validate API
    • Explain API
    • Profile API
      • Profiling Queries
      • Profiling Aggregations
      • Profiling Considerations
    • Percolator
    • Field stats API
  • 聚合
    • Metrics Aggregations
      • Avg Aggregation
      • Cardinality Aggregation
      • Extended Stats Aggregation
      • Geo Bounds Aggregation
      • Geo Centroid Aggregation
      • Max Aggregation
      • Min Aggregation
      • Percentiles Aggregation
      • Percentile Ranks Aggregation
      • Scripted Metric Aggregation
      • Stats Aggregation
      • Sum Aggregation
      • Top hits Aggregation
      • Value Count Aggregation
    • Bucket Aggregations
      • Adjacency Matrix Aggregation
      • Children Aggregation
      • Date Histogram Aggregation
      • Date Range Aggregation
      • Diversified Sampler Aggregation
      • Filter Aggregation
      • Filters Aggregation
      • Geo Distance Aggregation
      • GeoHash grid Aggregation
      • Global Aggregation
      • Histogram Aggregation
      • IP Range Aggregation
      • Missing Aggregation
      • Nested Aggregation
      • Range Aggregation
      • Reverse nested Aggregation
      • Sampler Aggregation
      • Significant Terms Aggregation
      • Terms Aggregation
    • Pipeline Aggregations
      • Avg Bucket Aggregation
      • Derivative Aggregation
      • Max Bucket Aggregation
      • Min Bucket Aggregation
      • Sum Bucket Aggregation
      • Stats Bucket Aggregation
      • Extended Stats Bucket Aggregation
      • Percentiles Bucket Aggregation
      • Moving Average Aggregation
      • Cumulative Sum Aggregation
      • Bucket Script Aggregation
      • Bucket Selector Aggregation
      • Serial Differencing Aggregation
    • Matrix Aggregations
      • Matrix Stats
    • Caching heavy aggregations
    • Returning only aggregation results
    • Aggregation Metadata
  • 索引库API
    • Create Index
    • Delete Index
    • Get Index
    • Indices Exists
    • Open / Close Index API
    • Shrink Index
    • Rollover Index
    • Put Mapping
    • Get Mapping
    • Get Field Mapping
    • Types Exists
    • Index Aliases
    • Update Indices Settings
    • Get Settings
    • Analyze
      • Explain Analyze
    • Index Templates
    • Shadow replica indices
      • Node level settings related to shadow replicas
    • Indices Stats
    • Indices Segments
    • Indices Recovery
    • Indices Shard Stores
    • Clear Cache
    • Flush
      • Synced Flush
    • Refresh
    • Force Merge
  • cat API
    • cat aliases
    • cat allocation
    • cat count
    • cat fielddata
    • cat health
    • cat indices
    • cat master
    • cat nodeattrs
    • cat nodes
    • cat pending tasks
    • cat plugins
    • cat recovery
    • cat repositories
    • cat thread pool
    • cat shards
    • cat segments
    • cat snapshots
    • cat templates
  • 集群API
    • Cluster Health
    • Cluster State
    • Cluster Stats
    • Pending cluster tasks
    • Cluster Reroute
    • Cluster Update Settings
    • Nodes Stats
    • Nodes Info
    • Task Management API
    • Nodes hot_threads
    • Cluster Allocation Explain API
  • DSL查询
    • Query and filter context
    • Match All Query
    • Full text queries
      • Match Query
      • Match Phrase Query
      • Match Phrase Prefix Query
      • Multi Match Query
      • Common Terms Query
      • Query String Query
      • Simple Query String Query
    • Term level queries
      • Term Query
      • Terms Query
      • Range Query
      • Exists Query
      • Prefix Query
      • Wildcard Query
      • Regexp Query
      • Fuzzy Query
      • Type Query
      • Ids Query
    • Compound queries
      • Constant Score Query
      • Bool Query
      • Dis Max Query
      • Function Score Query
      • Boosting Query
      • Indices Query
    • Joining queries
      • Nested Query
      • Has Child Query
      • Has Parent Query
      • Parent Id Query
    • Geo queries
      • GeoShape Query
      • Geo Bounding Box Query
      • Geo Distance Query
      • Geo Distance Range Query
      • Geo Polygon Query
    • Specialized queries
      • More Like This Query
      • Template Query
      • Script Query
      • Percolate Query
    • Span queries
      • Span Term Query
      • Span Multi Term Query
      • Span First Query
      • Span Near Query
      • Span Or Query
      • Span Not Query
      • Span Containing Query
      • Span Within Query
      • Span Field Masking Query
    • Minimum Should Match
    • Multi Term Query Rewrite
  • Mapping
    • Field datatypes
      • Array datatype
      • Binary datatype
      • Range datatypes
      • Boolean datatype
      • Date datatype
      • Geo-point datatype
      • Geo-Shape datatype
      • IP datatype
      • Keyword datatype
      • Nested datatype
      • Numeric datatypes
      • Object datatype
      • String datatype
      • Text datatype
      • Token count datatype
      • Percolator type
    • Meta-Fields
      • _all field
      • _field_names field
      • _id field
      • _index field
      • _meta field
      • _parent field
      • _routing field
      • _source field
      • _type field
      • _uid field
    • Mapping parameters
      • analyzer
      • normalizer
      • boost
      • coerce
      • copy_to
      • doc_values
      • dynamic
      • enabled
      • fielddata
      • format
      • ignore_above
      • ignore_malformed
      • include_in_all
      • index
      • index_options
      • fields
      • norms
      • null_value
      • position_increment_gap
      • properties
      • search_analyzer
      • similarity
      • store
      • term_vector
    • Dynamic Mapping
      • default mapping
      • Dynamic field mapping
      • Dynamic templates
      • Override default template
  • Analysis
    • Anatomy of an analyzer
    • Testing analyzers
    • Analyzers
      • Configuring built-in analyzers
      • Standard Analyzer
      • Simple Analyzer
      • Whitespace Analyzer
      • Stop Analyzer
      • Keyword Analyzer
      • Pattern Analyzer
      • Language Analyzers
      • Fingerprint Analyzer
      • Custom Analyzer
      • Normalizers
    • Tokenizers
      • Standard Tokenizer
      • Letter Tokenizer
      • Lowercase Tokenizer
      • Whitespace Tokenizer
      • UAX URL Email Tokenizer
      • Classic Tokenizer
      • Thai Tokenizer
      • NGram Tokenizer
      • Edge NGram Tokenizer
      • Keyword Tokenizer
      • Pattern Tokenizer
      • Path Hierarchy Tokenizer
    • Token Filters
      • Standard Token Filter
      • ASCII Folding Token Filter
      • Flatten Graph Token Filter
      • Length Token Filter
      • Lowercase Token Filter
      • Uppercase Token Filter
      • NGram Token Filter
      • Edge NGram Token Filter
      • Porter Stem Token Filter
      • Shingle Token Filter
      • Stop Token Filter
      • Word Delimiter Token Filter
      • Stemmer Token Filter
      • Stemmer Override Token Filter
      • Keyword Marker Token Filter
      • Keyword Repeat Token Filter
      • KStem Token Filter
      • Snowball Token Filter
      • Phonetic Token Filter
      • Synonym Token Filter
      • Synonym Graph Token Filter
      • Compound Word Token Filter
      • Reverse Token Filter
      • Elision Token Filter
      • Truncate Token Filter
      • Unique Token Filter
      • Pattern Capture Token Filter
      • Pattern Replace Token Filter
      • Trim Token Filter
      • Limit Token Count Token Filter
      • Hunspell Token Filter
      • Common Grams Token Filter
      • Normalization Token Filter
      • CJK Width Token Filter
      • CJK Bigram Token Filter
      • Delimited Payload Token Filter
      • Keep Words Token Filter
      • Keep Types Token Filter
      • Classic Token Filter
      • Apostrophe Token Filter
      • Decimal Digit Token Filter
      • Fingerprint Token Filter
      • Minhash Token Filter
    • Character Filters
      • HTML Strip Char Filter
      • Mapping Char Filter
      • Pattern Replace Char Filter
  • 模块
    • 集群
      • 集群的分片分配
      • 基于磁盘的分片分配
      • 了解分片分配
      • 分片分配过滤
      • 其他群集设置
    • Discovery
      • Azure Classic Discovery
      • EC2 Discovery
      • Google Compute Engine Discovery
      • Zen Discovery
    • 本地网关
    • HTTP
    • 索引库
      • 熔断
      • Fielddata
      • 节点查询缓存
      • 索引缓冲
      • 分片请求缓存
      • 索引库恢复
    • 网络设置
    • 节点
    • 插件
    • 脚本
      • 如何使用脚本
      • 文档中字段的访问以及一些特殊变量
      • 脚本与安全
      • Groovy脚本语言
      • PainLess脚本语言
      • PainLess语法
      • PainLess调试
      • Lucene表达式语言
      • 原生(java)脚本
      • 高级文本评分脚本
    • 快照与还原
    • 线程池
    • 传输
    • Tribe节点
    • 跨集群搜索
  • Index模块
    • Analysis
    • Index Shard Allocation
      • Shard Allocation Filtering
      • Delaying allocation when a node leaves
      • Index recovery prioritization
      • Total Shards Per Node
    • Mapper
    • Merge
    • Similarity module
    • Slow Log
    • Store
      • Pre-loading data into the file system cache
    • Translog
  • Ingest节点
    • Pipeline Definition
    • Ingest APIs
      • Put Pipeline API
      • Get Pipeline API
      • Delete Pipeline API
      • Simulate Pipeline API
    • Accessing Data in Pipelines
    • Handling Failures in Pipelines
    • Processors
      • Append Processor
      • Convert Processor
      • Date Processor
      • Date Index Name Processor
      • Fail Processor
      • Foreach Processor
      • Grok Processor
      • Gsub Processor
      • Join Processor
      • JSON Processor
      • KV Processor
      • Lowercase Processor
      • Remove Processor
      • Rename Processor
      • Script Processor
      • Set Processor
      • Split Processor
      • Sort Processor
      • Trim Processor
      • Uppercase Processor
      • Dot Expander Processor
  • How To
    • General recommendations
    • Recipes
    • Tune for indexing speed
    • Tune for search speed
    • Tune for disk usage
  • Testing
    • Java Testing Framework
      • why randomized testing?
      • Using the elasticsearch test classes
      • unit tests
      • integration tests
      • Randomized testing
      • Assertions
  • 术语表
  • 发布说明
    • 5.3.0 Release Notes
    • 5.2.2 Release Notes
    • 5.2.1 Release Notes
    • 5.2.0 Release Notes
    • 5.1.2 Release Notes
    • 5.1.1 Release Notes
    • 5.1.0 Release Notes
    • 5.0.2 Release Notes
    • 5.0.1 Release Notes
    • 5.0.0 Combined Release Notes
    • 5.0.0 GA Release Notes
    • 5.0.0-rc1 Release Notes
    • 5.0.0-beta1 Release Notes
    • 5.0.0-alpha5 Release Notes
    • 5.0.0-alpha4 Release Notes
    • 5.0.0-alpha3 Release Notes
    • 5.0.0-alpha2 Release Notes
    • 5.0.0-alpha1 Release Notes
    • 5.0.0-alpha1 Release Notes (Changes previously released in 2.x)
  • Painless API参考文档
Powered by GitBook
On this page
  • 版本控制
  • 路由
  • Parent
  • 等待活动分片
  • 冲刷
  • 更新
  • 安全

Was this helpful?

  1. 文档API

Bulk API

PreviousMulti Get APINextReindex API

Last updated 4 years ago

Was this helpful?

Bulk API可以在单个API调用中执行多个创建索引/删除的操作。这可以大大提高索引速度。

批量请求客户端支持

一些官方支持的客户端提供帮助来协助从一个索引到另一个索引的批量请求和重新索引:

Perl

  参见与

Python

  参见

REST API端点是/_bulk,它期望是以换行符分隔JSON(NDJSON)的结构:

action_and_meta_data\n
optional_source\n
action_and_meta_data\n
optional_source\n
....
action_and_meta_data\n
optional_source\n

注意:最后一行数据必须以换行符\n结尾。每个换行符字符之前都可以回车\r。当向该端点发送请求时,应将Content-Type头设置为application/x-ndjson。

可能的操作是index、create、delete和update。index和create期望下一行的源,并且具有与标准索引API的op_type参数相同的语义(即:如果具有相同索引和类型的文档已经存在,则create将失败,而有必要时索引将添加或替换文档)。delete并不期望下列行的源,并且具有与标准delete API相同的语义。 update需要在下一行指定部分文档,upsert和script及其选项。

如果要提供文本文件输入到curl,则必须使用--data-binary标志替代-d的文本。后者不需要保留换行符。例:

$ cat requests
{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
$ curl -s -H "Content-Type: application/x-ndjson" -XPOST localhost:9200/_bulk --data-binary "@requests"; echo
{"took":7, "errors": false, "items":[{"index":{"_index":"test","_type":"type1","_id":"1","_version":1,"result":"created","forced_refresh":false}}]}

因为此格式使用文字\n作为分隔符,请确保JSON操作和源文档不是格式化的打印。以下是批量命令正确序列的示例:

POST _bulk
{ "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } }
{ "field1" : "value1" }
{ "delete" : { "_index" : "test", "_type" : "type1", "_id" : "2" } }
{ "create" : { "_index" : "test", "_type" : "type1", "_id" : "3" } }
{ "field1" : "value3" }
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "test"} }
{ "doc" : {"field2" : "value2"} }

批量操作的结果如下:

{
   "took": 30,
   "errors": false,
   "items": [
      {
         "index": {
            "_index": "test",
            "_type": "type1",
            "_id": "1",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "created": true,
            "status": 201
         }
      },
      {
         "delete": {
            "found": false,
            "_index": "test",
            "_type": "type1",
            "_id": "2",
            "_version": 1,
            "result": "not_found",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "status": 404
         }
      },
      {
         "create": {
            "_index": "test",
            "_type": "type1",
            "_id": "3",
            "_version": 1,
            "result": "created",
            "_shards": {
               "total": 2,
               "successful": 1,
               "failed": 0
            },
            "created": true,
            "status": 201
         }
      },
      {
         "update": {
            "_index": "test",
            "_type": "type1",
            "_id": "1",
            "_version": 2,
            "result": "updated",
            "_shards": {
                "total": 2,
                "successful": 1,
                "failed": 0
            },
            "status": 200
         }
      }
   ]
}

端点是/_bulk、/{index}/_bulk和{index}/{type}/_bulk。当提供索引或索引/类型时,它们将作为批量操作的条目的默认值使用、不会作为明确声明的条目使用。

关于格式的注意。这里的想法是尽可能快地处理这个问题。由于某些操作将被重定向到其他节点上的其他分片,因此在接收节点侧仅解析action_meta_data。

使用此协议的客户端库应尽可能尝试在客户端执行类似操作,并尽可能减少缓冲。

对批量操作的响应是一个大的JSON结构,其中包含了执行每个操作的各个结果。一个动作的失败不会影响剩余的动作。

单次bulk调用没有一个“正确”的操作执行数量。您应该尝试使用不同的设置来查找特定工作负载的最佳大小。

如果使用HTTP API,请确保客户端不发送HTTP块,因为这会减慢事情。

版本控制

路由

每个bulk条目可以使用_routing与routing字段包括路由值。它基于映射的_routing来自动跟踪索引与删除操作的行为。

Parent

每个bulk条目可以使用_parent与parent字段来包含父值。它基于映射的_parent与_routing来自动跟踪索引/删除操作的行为。

等待活动分片

冲刷

更新

当使用update操作时,_retry_on_conflict可以用作动作本身的字段(而不是额外的数据行),可以指定在版本冲突的情况下应重试更新的次数。

POST _bulk
{ "update" : {"_id" : "1", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"} }
{ "update" : { "_id" : "0", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "script" : { "inline": "ctx._source.counter += params.param1", "lang" : "painless", "params" : {"param1" : 1}}, "upsert" : {"counter" : 1}}
{ "update" : {"_id" : "2", "_type" : "type1", "_index" : "index1", "_retry_on_conflict" : 3} }
{ "doc" : {"field" : "value"}, "doc_as_upsert" : true }
{ "update" : {"_id" : "3", "_type" : "type1", "_index" : "index1", "_source" : true} }
{ "doc" : {"field" : "value"} }
{ "update" : {"_id" : "4", "_type" : "type1", "_index" : "index1"} }
{ "doc" : {"field" : "value"}, "_source": true}

安全

每个bulk条目可以使用_version与version字段包含版本值。它基于_version映射自动跟踪索引与删除操作的行为。它还支持version_typ / _version_type(请参阅)。

进行批量调用时,您可以设置wait_for_active_shards参数,以便在开始处理批量请求之前要求最小数量的分片副本处于活动状态。有关详细信息和使用示例,请参阅。

用来控制本次的修改能够被搜索可见。参见:。

更新操作数据行支持以下选项:doc(部分文档)、upsert、doc_as_upsert、scirpt、params(与脚本结合使用)、lang(与脚本结合使用)和_source。有关选项的详细信息,请参阅。更新操作的示例:

参见

Search::Elasticsearch::Bulk
Search::Elasticsearch::Scroll
elasticsearch.helpers.*
refresh
更新操作文档
基于URL的访问控制
版本控制
此处