ASCII folding token filter

基本ラテンUnicodeブロック（最初の127 ASCII文字）に含まれないアルファベット、数字、記号の文字を、存在する場合はそのASCII等価物に変換します。たとえば、このフィルターはàをaに変更します。

このフィルターはLuceneのASCIIFoldingFilterを使用します。

Example

次のanalyze APIリクエストは、asciifoldingフィルターを使用してaçaí à la carteのダイアクリティカルマークを削除します:

Python

  1. resp = client.indices.analyze(
  2.    tokenizer="standard",
  3.    filter=[
  4.    "asciifolding"
  5.    ],
  6.    text="açaí à la carte",
  7. )
  8. print(resp)

Ruby

  1. response = client.indices.analyze(
  2.   body: {
  3.    tokenizer: 'standard',
  4.    filter: [
  5.    'asciifolding'
  6.    ],
  7.    text: 'açaí à la carte'
  8.   }
  9. )
  10. puts response

Js

  1. const response = await client.indices.analyze({
  2.   tokenizer: "standard",
  3.   filter: ["asciifolding"],
  4.   text: "açaí à la carte",
  5. });
  6. console.log(response);

Console

  1. GET /_analyze
  2. {
  3.   "tokenizer" : "standard",
  4.   "filter" : ["asciifolding"],
  5.   "text" : "açaí à la carte"
  6. }

フィルターは次のトークンを生成します:

Text

  1. [ acai, a, la, carte ]

Add to an analyzer

次のcreate index APIリクエストは、asciifoldingフィルターを使用して新しいcustom analyzerを構成します。

Python

  1. resp = client.indices.create(
  2.    index="asciifold_example",
  3.    settings={
  4.    "analysis": {
  5.    "analyzer": {
  6.    "standard_asciifolding": {
  7.    "tokenizer": "standard",
  8.    "filter": [
  9.    "asciifolding"
  10.    ]
  11.    }
  12.    }
  13.    }
  14.    },
  15. )
  16. print(resp)

Ruby

  1. response = client.indices.create(
  2.   index: 'asciifold_example',
  3.   body: {
  4.    settings: {
  5.    analysis: {
  6.    analyzer: {
  7.    standard_asciifolding: {
  8.    tokenizer: 'standard',
  9.    filter: [
  10.    'asciifolding'
  11.    ]
  12.    }
  13.    }
  14.    }
  15.    }
  16.   }
  17. )
  18. puts response

Js

  1. const response = await client.indices.create({
  2.   index: "asciifold_example",
  3.   settings: {
  4.    analysis: {
  5.    analyzer: {
  6.    standard_asciifolding: {
  7.    tokenizer: "standard",
  8.    filter: ["asciifolding"],
  9.    },
  10.    },
  11.    },
  12.   },
  13. });
  14. console.log(response);

Console

  1. PUT /asciifold_example
  2. {
  3.   "settings": {
  4.    "analysis": {
  5.    "analyzer": {
  6.    "standard_asciifolding": {
  7.    "tokenizer": "standard",
  8.    "filter": [ "asciifolding" ]
  9.    }
  10.    }
  11.    }
  12.   }
  13. }

Configurable parameters

  • preserve_original
  • （オプション、ブール値）trueの場合、元のトークンと折りたたまれたトークンの両方を出力します。デフォルトはfalseです。

Customize

  2. たとえば、次のリクエストは`````asciifolding`````フィルターを作成し、`````preserve_original`````をtrueに設定します:
  5. #### Python
  7. ``````python
  8. resp = client.indices.create(
  9.    index="asciifold_example",
  10.    settings={
  11.    "analysis": {
  12.    "analyzer": {
  13.    "standard_asciifolding": {
  14.    "tokenizer": "standard",
  15.    "filter": [
  16.    "my_ascii_folding"
  17.    ]
  18.    }
  19.    },
  20.    "filter": {
  21.    "my_ascii_folding": {
  22.    "type": "asciifolding",
  23.    "preserve_original": True
  24.    }
  25.    }
  26.    }
  27.    },
  28. )
  29. print(resp)
  30. `

Ruby

  1. response = client.indices.create(
  2.   index: 'asciifold_example',
  3.   body: {
  4.    settings: {
  5.    analysis: {
  6.    analyzer: {
  7.    standard_asciifolding: {
  8.    tokenizer: 'standard',
  9.    filter: [
  10.    'my_ascii_folding'
  11.    ]
  12.    }
  13.    },
  14.    filter: {
  15.    my_ascii_folding: {
  16.    type: 'asciifolding',
  17.    preserve_original: true
  18.    }
  19.    }
  20.    }
  21.    }
  22.   }
  23. )
  24. puts response

Js

  1. const response = await client.indices.create({
  2.   index: "asciifold_example",
  3.   settings: {
  4.    analysis: {
  5.    analyzer: {
  6.    standard_asciifolding: {
  7.    tokenizer: "standard",
  8.    filter: ["my_ascii_folding"],
  9.    },
  10.    },
  11.    filter: {
  12.    my_ascii_folding: {
  13.    type: "asciifolding",
  14.    preserve_original: true,
  15.    },
  16.    },
  17.    },
  18.   },
  19. });
  20. console.log(response);

Console

  1. PUT /asciifold_example
  2. {
  3.   "settings": {
  4.    "analysis": {
  5.    "analyzer": {
  6.    "standard_asciifolding": {
  7.    "tokenizer": "standard",
  8.    "filter": [ "my_ascii_folding" ]
  9.    }
  10.    },
  11.    "filter": {
  12.    "my_ascii_folding": {
  13.    "type": "asciifolding",
  14.    "preserve_original": true
  15.    }
  16.    }
  17.    }
  18.   }
  19. }