Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-61192

Return Child Mapping as a JSON in FTS Result

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • feature-backlog
    • 7.6.0
    • fts
    • 0

    Description

      Currently, in FTS, while returning a child mapping, it always returns fields using the child_mapping.field notation. 

      In LLM frameworks like LangChain, the document structures are quite dynamic like the following:

      {

        "text": "this is a sample document",

         "embedding": [vector embeddings],

          "metadata":

      {     "source": "../../modules/state_of_the_union.txt",     "date": "2015-01-01",     "rating": 1,     "author": "Jane Doe"   }

      }

      Here, the "metadata" is dynamic and varies depending on the data loader being used. 

      We can index"metadata" as a dynamic mapping and even return all the fields using "store dynamic fields". However, we do not get the original document structure back.

      Currently, we handle it in the Couchbase Vector Store integration by parsing the metadata fields & removing the "metadata." from the field returned by FTS.

      Would it be possible to support the option to return the entire child mapping as a JSON like in the original document? I believe Elastic is similar to us in terms of querying (like metadata.source) but they return the entire JSON in the response.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            People

              rahul.rampure Rahul Rampure
              nithish.raghunandanan Nithish Raghunandanan
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty