Uploaded image for project: 'Couchbase Python Client Library'
  1. Couchbase Python Client Library
  2. PYCBC-1588

Add support for importing FTS index from JSON

    XMLWordPrintable

Details

    • Improvement
    • Resolution: Unresolved
    • Major
    • None
    • 4.2.0
    • None
    • None
    • 0

    Description

      Use Case

      I want to share the FTS index definition exported from the Server UI in a file so that others can import this index via the SDK Management API. 

      There are a couple of issues with the current implementation of importing the FTS index definition from the Couchbase server via the Python SDK. 

      1. We need to explicitly pass in all the components like `name`, `params`, `source_name` of the Index definition from the file. 

           scope_index_manager.upsert_index( SearchIndex( index_definition["name"], params=index_definition["params"], source_name=index_definition["sourceName"], ), ) 

      {{2. While testing out while passing the entire exported JSON as `params`, I got a completely cryptic error (not in the JSON) back from the Server. }}

      "err: invalid character \'R\' looking for beginning of value"

       

           The error felt like the index_name was being used as the bucket.

      {{InternalServerFailureException(<ec=5, category=couchbase.common, message=internal_server_failure (5), context=HTTPErrorContext({'client_context_id': '046994-f1ca-a847-b245-bf38c18dea2b79', 'method': 'PUT', 'path': '/api/bucket/pdf-docs/scope/shared/index/pdf-docs', 'http_status': 400, 'http_body': '{"error":"rest_create_index: error creating index: , err: manager_api: CreateIndex, Prepare failed, err: PrepareIndex, changing a scoped index\'s bucket/scope name is NOT allowed","request":{"name":"pdf-docs","params":{"name":"pdf_search1","params":{"doc_config":

      Unknown macro: {"docid_prefix_delim"}

      ,"mapping":{"default_analyzer":"standard","default_datetime_parser":"dateTimeOptional","default_field":"_all","default_mapping":

      Unknown macro: {"dynamic"}

      ,"default_type":"_default","docvalues_dynamic":false,"index_dynamic":true,"store_dynamic":false,"type_field":"_type","types":{"shared.docs":{"dynamic":true,"enabled":true,"properties":{"embedding":{"dynamic":false,"enabled":true,"fields":[

      Unknown macro: {"dims"}

      ]},"text":{"dynamic":false,"enabled":true,"fields":[

      Unknown macro: {"index"}

      ]}}}}},"store":{"indexType":"scorch","segmentVersion":16}},"planParams":{"indexPartitions":16,"maxPartitionsPerPIndex":64,"numReplicas":0},"sourceName":"pdf-docs","sourceParams":{},"sourceType":"gocbcore","type":"fulltext-index"},"sourceType":"couchbase","type":"fulltext-index"},"status":"fail"}\n'}), C Source=/Users/couchbase/jenkins/workspace/python/sdk/python-packaging-pipeline/py-client/src/management/search_index_management.cxx:359>)}}

      {{1 & 2 can be solved by using the `SearchIndex.from_server()`. }}

      3. There is however another issue with the arguments being used in the SDK being Snake case while the export from the server is Camel case. This means that the user will have to handle this conversion on their side (manually editing the server export or parsing the individual parts of the JSON to the Constructor). 
      {{}}

      Suggestion

      Have a method that creates the SearchIndex from the Server export handling both the camel to snake case conversion. It could be taking the JSON string like in Java or Python Dictionary.

      Java API

      SearchIndex newSearchIdx = SearchIndex.fromJson(jsonstring); scoped_search_manager.upsertIndex(newSearchIdx);

       

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          People

            jared.casey Jared Casey
            nithish.raghunandanan Nithish Raghunandanan
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty