Uploaded image for project: 'Couchbase Server'
  1. Couchbase Server
  2. MB-41345

Extend external dataset support for Azure Data Lake



    • 1
    • CX Sprint 219, CX Sprint 220, CX Sprint 258, CX Sprint 259, CX Sprint 260, CX Sprint 261, CX Sprint 262, CX Sprint 263, CX Sprint 264, CX Sprint 265, CX Sprint 266


      Customer is deploying Couchbase on Azure and is using Azure Data Lake. They would like to see External Dataset capability in Azure.


      1. What kind of data (offline, archive, streaming external data, etc.) are you storing in Azure cloud?
      >>We are moving entire Data warehouse (Teradata) into ADL Gen 2.

      2. Which format?
      Azure Data Lake Storage Gen1
      Azure Data Lake Storage Gen2
      Azure Blob Storage
      >>ADL Gen2 and ADX. There are some cases of Azure Blob storage and Table storage.

      3. What type of business or technical questions they are trying to get an answer for?
      >>This will be specific to business products. Each product has their own BI reporting use cases that are answered.
      On high-level, you can consider all the DoT and IATA reporting Airlines metrics that any Airline publishes. Those are the major reporting that will be published through ADL going forward.
      On low-level, it could App specific information like App usage pattern, feature usage pattern, App or Product specific KPIs are also going to be reported.

      4. What do you plan to combine data in Couchbase with?
      >>If Couchbase Analytics engine has the mechanism to connect to ADL, then we could use analytics data beyond operational data for decision making, user queries, quick reports, etc.
      Most of the Application operational data keeps only 7-30 days’ worth information. The historical data will be moved to ADL.
      Simple example Use case: While resolving an issue for customer with his/her Flight or Bags, if the agent wants to know more about the customer experience in the past, additional queries in operational apps could be obtained from ADL storage. Now, if the App uses Couchbase it could extend its N1QL to ADL data source when required.
      Advanced Use case: If I am trying to run an ML on current data-set and would like to get more insight on historical data beyond 30 days, then running additional queries from same database engine, would allow me to keep the ML model close to microservices/serverless functions where App team can maintain.


        Issue Links

          No reviews matched the request. Check your Options in the drop-down menu of this sections header.



              Hussain.Towaileb Hussain Towaileb
              idris.motiwala Idris Motiwala (Inactive)
              0 Vote for this issue
              2 Start watching this issue



                Gerrit Reviews

                  There are no open Gerrit changes