Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-52522

Internal error while querying external dataset when other file formats are also present in same S3 bucket

    XMLWordPrintable

Details

    Description

      Steps to reproduce :
      1. Have a 2 node cluster - kv and cbas
      2. Make sure AWS S3 bucket has few json files in the same path as parquet files.
      2. Create external link to a AWS S3 bucket.
      3. Create dataset on parquet files using external link.
      4. Try to run select count(*) from dataset query.
      5. Internal error is observed.
       
      A better message should be displayed, letting user know that parsing failed because there were file formats apart from parquet.

      Attachments

        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.

          Activity

            umang.agrawal Umang added a comment -

            Equivalent json for above parquet :
            {
            "coffee": {
            "region": [

            {"id":1,"name":"John Doe"}

            ,

            {"id":2,"name":"Don Joeh"}

            ],
            "country":

            {"id":2,"company":"ACME"}

            },
            "brewing": {
            "region": [

            {"id":1,"name":"John Doe"}

            ,

            {"id":2,"name":"Don Joeh"}

            ],
            "country":

            {"id":2,"company":"ACME"}

            }
            }

            umang.agrawal Umang added a comment - Equivalent json for above parquet : { "coffee": { "region": [ {"id":1,"name":"John Doe"} , {"id":2,"name":"Don Joeh"} ], "country": {"id":2,"company":"ACME"} }, "brewing": { "region": [ {"id":1,"name":"John Doe"} , {"id":2,"name":"Don Joeh"} ], "country": {"id":2,"company":"ACME"} } }

            Build couchbase-server-7.1.2-3315 contains cbas-core commit bac9882 with commit message:
            MB-52522: Handle invalid Parquet file error

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.1.2-3315 contains cbas-core commit bac9882 with commit message: MB-52522 : Handle invalid Parquet file error

            Build couchbase-server-7.2.0-1516 contains cbas-core commit bac9882 with commit message:
            MB-52522: Handle invalid Parquet file error

            build-team Couchbase Build Team added a comment - Build couchbase-server-7.2.0-1516 contains cbas-core commit bac9882 with commit message: MB-52522 : Handle invalid Parquet file error

            Build couchbase-server-8.0.0-1041 contains cbas-core commit bac9882 with commit message:
            MB-52522: Handle invalid Parquet file error

            build-team Couchbase Build Team added a comment - Build couchbase-server-8.0.0-1041 contains cbas-core commit bac9882 with commit message: MB-52522 : Handle invalid Parquet file error
            umang.agrawal Umang added a comment -

            Verified with Enterprise Edition 7.1.2 build 3349

            Following error is observed -

            Invalid Parquet file: s3a://umang-test-1/daily.json. Reason: not a Parquet file
            

            umang.agrawal Umang added a comment - Verified with Enterprise Edition 7.1.2 build 3349 Following error is observed - Invalid Parquet file: s3a://umang-test-1/daily.json. Reason: not a Parquet file

            People

              umang.agrawal Umang
              umang.agrawal Umang
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Gerrit Reviews

                  There are no open Gerrit changes

                  PagerDuty