# Provided Text Chunking Example fails with Neural Sparse!

**URL:** <https://forum.opensearch.org/t/provided-text-chunking-example-fails-with-neural-sparse/24314>\
**Category:** OpenSearch\
**Created:** [May 9, 2025, 7:05am UTC](https://forum.opensearch.org/t/provided-text-chunking-example-fails-with-neural-sparse/24314 "2025-05-09T07:05:02Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![asfoorial](https://avatars.discourse-cdn.com/v4/letter/a/3da27b/32.png) [@asfoorial](https://forum.opensearch.org/u/asfoorial)\
**Post date:** [May 9, 2025, 7:05am UTC](https://forum.opensearch.org/t/provided-text-chunking-example-fails-with-neural-sparse/24314/1 "2025-05-09T07:05:02Z")

</div>

OS 3.0.0/ Windows 11

Hi all,

I have tried the example here [Text chunking - OpenSearch Documentation](https://docs.opensearch.org/docs/latest/vector-search/ingesting-data/text-chunking/) and it works very well with knn dense vectors. However when I change the embedding type to neural sparse it fails. Below is the full example.

I highly appreciate your help/suggestions.

```json
POST /_plugins/_ml/models/_register?deploy=true
{
  "name": "amazon/neural-sparse/opensearch-neural-sparse-encoding-v2-distill",
  "version": "1.0.0",
  "model_format": "TORCH_SCRIPT"
}

```

```json
PUT _ingest/pipeline/text-chunking-embedding-ingest-pipeline
{
  "description": "A text chunking and embedding ingest pipeline",
  "processors": [
    {
      "text_chunking": {
        "algorithm": {
          "fixed_token_length": {
            "token_limit": 10,
            "overlap_rate": 0.2,
            "tokenizer": "standard"
          }
        },
        "field_map": {
          "passage_text": "passage_chunk"
        }
      }
    },
    {
      "text_embedding": {
        "model_id": "B973sJYBVv-6fOo2U1W1",
        "field_map": {
          "passage_chunk": "passage_chunk_embedding"
        }
      }
    }
  ]
}

```

```json
PUT testindex
{
  "settings": {
    "index": {
    }
  },
  "mappings": {
    "properties": {
      "text": {
        "type": "text"
      },
      "passage_chunk_embedding": {
        "type": "nested",
        "properties": {
          "knn": {
            "type": "rank_features"
          }
        }
      }
    }
  }
}

```

```json
POST testindex/_doc?pipeline=text-chunking-embedding-ingest-pipeline
{
  "passage_text": "This is an example document to be chunked. The document contains a single paragraph, two sentences and 24 tokens by standard tokenizer in OpenSearch."
}

```

It fails to ingest and gives the below error:

```json
{
  "error": {
    "root_cause": [
      {
        "type": "null_pointer_exception",
        "reason": "Cannot invoke \"java.util.List.iterator()\" because \"tensorsList\" is null"
      }
    ],
    "type": "null_pointer_exception",
    "reason": "Cannot invoke \"java.util.List.iterator()\" because \"tensorsList\" is null"
  },
  "status": 500
}

```

---

<div class="post-metadata">

**Author:** ![yuye-aws](https://avatars.discourse-cdn.com/v4/letter/y/13edae/32.png) [@yuye-aws](https://forum.opensearch.org/u/yuye-aws)\
**Post date:** [November 18, 2025, 10:17am UTC](https://forum.opensearch.org/t/provided-text-chunking-example-fails-with-neural-sparse/24314/2 "2025-11-18T10:17:08Z")

</div>

You can try to use this index mapping:

```auto
“passage_chunk_embedding”: {
  “type”: “nested”,
  “properties”: {
  “sparse_encoding”: {
    “type”: “rank_features”
  }
}

```

Meanwhile, you should use the `sparse_encoding` processor, instead of the `text_embedding`. processor in the ingestion pipeline. This is an example:

```auto
PUT _ingest/pipeline/text-chunking-embedding-ingest-pipeline
{
  “description”: “A text chunking and embedding ingest pipeline”,
  “processors”: [
    {
      “text_chunking”: {
        “algorithm”: {
          “fixed_token_length”: {
            “token_limit”: 10,
            “overlap_rate”: 0.2,
            “tokenizer”: “standard”
          }
        },
        “field_map”: {
          “passage_text”: “passage_chunk”
        }
      }
    },
    {
      “sparse_encoding”: {
        “model_id”: “XrRvlpoBgZd8yJbf_sJH”,
        “field_map”: {
          “passage_chunk”: “passage_chunk_embedding”
        }
      }
    }
  ]
}

```