Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-54575

Analytics Local Links API doesn't send isActive field

    XMLWordPrintable

Details

    • Bug
    • Status: Closed
    • Critical
    • Resolution: Not a Bug
    • 7.1.3
    • None
    • analytics
    • Capella pre-prod
    • Untriaged
    • 0
    • Unknown
    • Analytics Sprint 8

    Description

      Hi team,

       

      We were validating Analytics Links on the Capella UI and found that the API call to retrieve the Analytics Links has a change where the "IsActive" field is not sent in the API response anymore.

       

      Here is a sample call to the API and the response on older RC build and the latest RC build.

      API : 

      GET _p/cbas/query/service

       

       

      Older RC Build Response: (RC8)

      {
      2	"requestID": "d575f0c7-509e-426c-83f4-3db38e5ae6a0",
      3	"signature": {
      4		"*": "*"
      5	},
      6	"results": [ { "DataverseName": "Default", "isDataverse": true, "links": [ { "Name": "Local" } ] }
      7, { "DataverseName": "Default", "Name": "Local", "isLink": true, "IsActive": true }
      8 ]
      9	,
      10	"plans":{},
      11	"status": "success",
      12	"metrics": {
      13		"elapsedTime": "351.532614ms",
      14		"executionTime": "278.989049ms",
      15		"resultCount": 2,
      16		"resultSize": 168,
      17		"processedObjects": 119
      18	}
      19} 

      (IsActive: true set for the default local link)

      Latest RC Build 3479:

      {
      2	"requestID": "e386a94d-d54d-4acf-8207-dae577022a09",
      3	"signature": {
      4		"*": "*"
      5	},
      6	"results": [ { "DataverseName": "Default", "isDataverse": true, "links": [ { "Name": "Local" } ] }
      7, { "DataverseName": "Default", "Name": "Local", "isLink": true }
      8 ]
      9	,
      10	"plans":{},
      11	"status": "success",
      12	"metrics": {
      13		"elapsedTime": "66.921004ms",
      14		"executionTime": "61.343781ms",
      15		"resultCount": 2,
      16		"resultSize": 150,
      17		"processedObjects": 41
      18	}
      19} 

      IsActive is not sent in the API response anymore.

       

      This is breaking the UI on Capella since we expect that field.

       

      Note: Please also confirm if this field is sent for new S3 links that are created.

       

      Can you confirm if this is by design and we should not expect this field anymore or this needs a fix on the server side? Thank you!

      Attachments

        No reviews matched the request. Check your Options in the drop-down menu of this sections header.

        Activity

          Hi Murtadha Hubail Capella UI makes this call.

           

          Please note, this is a proxy API, which simply forwards the API payload to the server API mentioned after /proxy path & returns the response received from the server API as is to the UI.

           

          API: POST https://api.sbx-8.sandbox.nonprod-project-avengers.com/v2/databases/5a9a785a-0e1b-42e4-b5d3-e14c1c5e1b8c/proxy/_p/cbas/query/service

           

          API Payload

           

          {
              "statement": "SELECT meta.*\nFROM (\n  SELECT ds.DataverseName,\n         ds.DataverseName || '.' || ds.DatasetName AS datasetFullyQualifiedName,\n         ds.DatasetName AS id,\n         TRUE AS isDataset,\n         ds.BucketName AS bucketName,\n         ds.ScopeName AS scopeName,\n         ds.CollectionName AS collectionName,\n         ds.BucketDataverseName AS linkDataverseName,\n         ds.`Filter` AS `filter`,\n         ds.LinkName,\n         ds.DatasetType,\n         concat2(', ', (\n             SELECT VALUE FieldName || ' ' || ( CASE WHEN LOWER(FieldType) = 'int64' THEN 'BIGINT' ELSE UPPER(FieldType) END) || ( CASE WHEN NOT IsNullable\n                 AND NOT IsMissable THEN ' IS NOT UNKNOWN' ELSE '' END)\n             FROM t.Derived.Record.Fields)) AS TypeString,\n         (\n           SELECT idx.IndexName,\n                  idx.SearchKey,\n                  idx.SearchKeyType\n           FROM Metadata.`Index` AS idx\n           WHERE idx.IsPrimary = FALSE\n             AND idx.DatasetName = ds.DatasetName\n             AND idx.DataverseName = ds.DataverseName) AS indexes,\n         ds.ExternalDetails.Properties AS externalDetails\n  FROM Metadata.`Dataset` AS ds LEFT\n    JOIN Metadata.Datatype t ON ds.DataverseName = t.DataverseName\n    AND t.DatatypeName = ds.DatatypeName\n  WHERE (ds.BucketName IS NOT MISSING\n      OR ds.DatasetType = 'EXTERNAL')\n  UNION ALL\n  SELECT dv.DataverseName,\n         TRUE AS isDataverse,\n         (\n           SELECT l.Name\n           FROM Metadata.`Link` AS l\n           WHERE l.DataverseName = dv.DataverseName) AS links\n  FROM Metadata.`Dataverse` AS dv\n  WHERE dv.DataverseName != 'Metadata'\n  UNION ALL\n  SELECT DataverseName,\n         Name,\n         IsActive,\n         `Type` AS LinkType,\n         TRUE AS isLink\n  FROM Metadata.`Link`) meta\nORDER BY meta.isDataverse DESC,\n         meta.isLink DESC;"
          }

           

           

          API Response:

          {
              "requestID": "cd9b7704-5e7d-465d-a6a9-82b276904948",
              "signature": {
                  "*": "*"
              },
              "results": [ { "DataverseName": "Default", "Name": "Local", "isLink": true }
          , { "DataverseName": "Default", "isDataverse": true, "links": [ { "Name": "Local" } ] }
           ]
              ,
              "plans":{},
              "status": "success",
              "metrics": {
                  "elapsedTime": "152.500956ms",
                  "executionTime": "147.085705ms",
                  "resultCount": 2,
                  "resultSize": 150,
                  "processedObjects": 41
              }
          }
           

           

           

          talina.shrotriya Talina Shrotriya added a comment - Hi Murtadha Hubail Capella UI makes this call.   Please note, this is a proxy API, which simply forwards the API payload to the server API mentioned after /proxy path & returns the response received from the server API as is to the UI.   API: POST https://api.sbx-8.sandbox.nonprod-project-avengers.com/v2/databases/5a9a785a-0e1b-42e4-b5d3-e14c1c5e1b8c/proxy/_p/cbas/query/service   API Payload   {     "statement" : "SELECT meta.*\nFROM (\n  SELECT ds.DataverseName,\n         ds.DataverseName || '.' || ds.DatasetName AS datasetFullyQualifiedName,\n         ds.DatasetName AS id,\n         TRUE AS isDataset,\n         ds.BucketName AS bucketName,\n         ds.ScopeName AS scopeName,\n         ds.CollectionName AS collectionName,\n         ds.BucketDataverseName AS linkDataverseName,\n         ds.`Filter` AS `filter`,\n         ds.LinkName,\n         ds.DatasetType,\n         concat2(', ', (\n             SELECT VALUE FieldName || ' ' || ( CASE WHEN LOWER(FieldType) = 'int64' THEN 'BIGINT' ELSE UPPER(FieldType) END) || ( CASE WHEN NOT IsNullable\n                 AND NOT IsMissable THEN ' IS NOT UNKNOWN' ELSE '' END)\n             FROM t.Derived.Record.Fields)) AS TypeString,\n         (\n           SELECT idx.IndexName,\n                  idx.SearchKey,\n                  idx.SearchKeyType\n           FROM Metadata.`Index` AS idx\n           WHERE idx.IsPrimary = FALSE\n             AND idx.DatasetName = ds.DatasetName\n             AND idx.DataverseName = ds.DataverseName) AS indexes,\n         ds.ExternalDetails.Properties AS externalDetails\n  FROM Metadata.`Dataset` AS ds LEFT\n    JOIN Metadata.Datatype t ON ds.DataverseName = t.DataverseName\n    AND t.DatatypeName = ds.DatatypeName\n  WHERE (ds.BucketName IS NOT MISSING\n      OR ds.DatasetType = 'EXTERNAL')\n  UNION ALL\n  SELECT dv.DataverseName,\n         TRUE AS isDataverse,\n         (\n           SELECT l.Name\n           FROM Metadata.`Link` AS l\n           WHERE l.DataverseName = dv.DataverseName) AS links\n  FROM Metadata.`Dataverse` AS dv\n  WHERE dv.DataverseName != 'Metadata'\n  UNION ALL\n  SELECT DataverseName,\n         Name,\n         IsActive,\n         `Type` AS LinkType,\n         TRUE AS isLink\n  FROM Metadata.`Link`) meta\nORDER BY meta.isDataverse DESC,\n         meta.isLink DESC;" }     API Response: {     "requestID" : "cd9b7704-5e7d-465d-a6a9-82b276904948" ,     "signature" : {         "*" : "*"     },     "results" : [ { "DataverseName" : "Default" , "Name" : "Local" , "isLink" : true } , { "DataverseName" : "Default" , "isDataverse" : true , "links" : [ { "Name" : "Local" } ] }  ]     ,     "plans" :{},     "status" : "success" ,     "metrics" : {         "elapsedTime" : "152.500956ms" ,         "executionTime" : "147.085705ms" ,         "resultCount" : 2 ,         "resultSize" : 150 ,         "processedObjects" : 41     } }    
          murtadha.hubail Murtadha Hubail added a comment - - edited

          Hi Talina Shrotriya,

          I tested this query on build 7.1.3 build 3479 on macOS and it returned the IsActive flag as expected.
          As for S3 links, they do not have the IsActive flag since they do not have connected/disconnected state like Local/Remote links.
          Here is a sample output of the query with the Local link and another S3 link:

          [
            {
              "DataverseName": "Default",
              "isDataverse": true,
              "links": [
                {
                  "Name": "Local"
                },
                {
                  "Name": "Some_S3_Link"
                }
              ]
            },
            {
              "DataverseName": "Default",
              "Name": "Local",
              "isLink": true,
              "IsActive": true
            },
            {
              "DataverseName": "Default",
              "Name": "Some_S3_Link",
              "isLink": true,
              "LinkType": "S3"
            }
          ]
          

          We didn't have any changes related to that flag recently. However, please share a cbcollect info and I can take another look if needed.

          murtadha.hubail Murtadha Hubail added a comment - - edited Hi Talina Shrotriya , I tested this query on build 7.1.3 build 3479 on macOS and it returned the IsActive flag as expected. As for S3 links, they do not have the IsActive flag since they do not have connected/disconnected state like Local/Remote links. Here is a sample output of the query with the Local link and another S3 link: [ { "DataverseName": "Default", "isDataverse": true, "links": [ { "Name": "Local" }, { "Name": "Some_S3_Link" } ] }, { "DataverseName": "Default", "Name": "Local", "isLink": true, "IsActive": true }, { "DataverseName": "Default", "Name": "Some_S3_Link", "isLink": true, "LinkType": "S3" } ] We didn't have any changes related to that flag recently. However, please share a cbcollect info and I can take another look if needed.

          Thanks Talina Shrotriya for granting me access to the cluster.
          Capturing the network traffic on the new UI, here is the query I found to get the Analytics metadata:

          SELECT DataverseName,
                 DataverseName || '.' || DatasetName AS fullName,
                 DatasetName AS id,
                 TRUE AS isDataset,
                 BucketName AS bucketName,
                 `Filter` AS `filter`,
                 LinkName,
                 DatasetType,
                 (
                   SELECT idx.IndexName,
                          idx.SearchKey,
                          idx.SearchKeyType
                   FROM Metadata.`Index` AS idx
                   WHERE idx.IsPrimary = FALSE
                     AND idx.DatasetName = ds.DatasetName
                     AND idx.DataverseName = ds.DataverseName) AS indexes,
                 ExternalDetails.Properties AS externalDetails
          FROM Metadata.`Dataset` AS ds
          WHERE (BucketName IS NOT MISSING
              OR DatasetType = 'EXTERNAL')
          UNION ALL
          SELECT dv.DataverseName,
                 TRUE AS isDataverse,
                 (
                   SELECT l.Name
                   FROM Metadata.`Link` AS l
                   WHERE l.DataverseName = dv.DataverseName) AS links
          FROM Metadata.`Dataverse` AS dv
          WHERE dv.DataverseName != 'Metadata'
          UNION ALL
          SELECT DataverseName,
                 Name,
                 TRUE AS isLink
          FROM Metadata.`Link`;
          

          This query doesn't include the IsActive flag and it looks like an old query that was used for 6.6 clusters. Fixing the UI query should fix the issue. I'm resolving this ticket as no CB Server fix is needed here.

          murtadha.hubail Murtadha Hubail added a comment - Thanks Talina Shrotriya for granting me access to the cluster. Capturing the network traffic on the new UI, here is the query I found to get the Analytics metadata: SELECT DataverseName, DataverseName || '.' || DatasetName AS fullName, DatasetName AS id, TRUE AS isDataset, BucketName AS bucketName, `Filter` AS `filter`, LinkName, DatasetType, ( SELECT idx.IndexName, idx.SearchKey, idx.SearchKeyType FROM Metadata.`Index` AS idx WHERE idx.IsPrimary = FALSE AND idx.DatasetName = ds.DatasetName AND idx.DataverseName = ds.DataverseName) AS indexes, ExternalDetails.Properties AS externalDetails FROM Metadata.`Dataset` AS ds WHERE (BucketName IS NOT MISSING OR DatasetType = 'EXTERNAL') UNION ALL SELECT dv.DataverseName, TRUE AS isDataverse, ( SELECT l.Name FROM Metadata.`Link` AS l WHERE l.DataverseName = dv.DataverseName) AS links FROM Metadata.`Dataverse` AS dv WHERE dv.DataverseName != 'Metadata' UNION ALL SELECT DataverseName, Name, TRUE AS isLink FROM Metadata.`Link`; This query doesn't include the IsActive flag and it looks like an old query that was used for 6.6 clusters. Fixing the UI query should fix the issue. I'm resolving this ticket as no CB Server fix is needed here.

          Went on a call with Murtadha Hubail and saw how the query passed to the analytics API has changed on the new Capella UI. We are good to close this MB and I will follow up on the subsequent AV Thank you!!

          talina.shrotriya Talina Shrotriya added a comment - Went on a call with Murtadha Hubail and saw how the query passed to the analytics API has changed on the new Capella UI. We are good to close this MB and I will follow up on the subsequent AV Thank you!!
          wayne Wayne Siu added a comment -

          Closing as it's not a bug.

          wayne Wayne Siu added a comment - Closing as it's not a bug.

          Some explanation on why this issue occurred on the Capella UI and isn't seen on 7.1.2 build that is currently in the dev environment for Capella.

          Background:

          Up until build 7.1.3-3476, we had a bug in Analytics where the Analytics service was not reachable on Capella. See: https://issues.couchbase.com/browse/MB-54428

          Due to this, we never were able to validate the analytics query made from the new Capella UI for 7.1.2 analytics remote links.

          From Murtadha's comment: the query made from the new Capella UI would've worked on some 7.1.2 RC builds, but not the GA build.

           

          After the bug fix: https://issues.couchbase.com/browse/MB-54428 as part of the AMI validation, we observed that the analytics remote links did not work on the new Capella UI which caused us to open this MB.

          As a conclusion from this MB, we now know what the correct analytics query to be made is and will be fixing this in https://couchbasecloud.atlassian.net/browse/AV-47684

           

          Thanks,

          Talina.

          talina.shrotriya Talina Shrotriya added a comment - Some explanation on why this issue occurred on the Capella UI and isn't seen on 7.1.2 build that is currently in the dev environment for Capella. Background: Up until build 7.1.3-3476, we had a bug in Analytics where the Analytics service was not reachable on Capella. See: https://issues.couchbase.com/browse/MB-54428 Due to this, we never were able to validate the analytics query made from the new Capella UI for 7.1.2 analytics remote links. From Murtadha's comment: the query made from the new Capella UI would've worked on some 7.1.2 RC builds, but not the GA build.   After the bug fix: https://issues.couchbase.com/browse/MB-54428 as part of the AMI validation, we observed that the analytics remote links did not work on the new Capella UI which caused us to open this MB. As a conclusion from this MB, we now know what the correct analytics query to be made is and will be fixing this in https://couchbasecloud.atlassian.net/browse/AV-47684   Thanks, Talina.

          People

            talina.shrotriya Talina Shrotriya
            talina.shrotriya Talina Shrotriya
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Gerrit Reviews

                There are no open Gerrit changes

                PagerDuty