Unlock the Power of Looker Block for Cortex Framework: A Comprehensive Deployment Guide

Photo by Carlos Muza on Unsplash

Introduction

In today’s data-driven world, businesses need a robust and scalable platform to derive actionable insights from their data. Looker, combined with the Cortex framework, offers a powerful solution for building and deploying impactful data applications. This article provides a step-by-step guide to using Looker Block to deploy Cortex Framework and shares best practices to ensure successful implementation.

Understanding Looker Block and Cortex

Looker is a cutting-edge business intelligence (BI) platform that enables users to explore, analyze, and visualize data from various sources. Looker Blocks are pre-built data models designed for common analytical scenarios within Looker. Rather than creating LookML views and exploring from scratch, you can leverage these ready-made building blocks, available on the Looker Marketplace. This feature accelerates the delivery of insights with minimal setup, making it a valuable tool for developers and end-users. Setting up a Looker Block using a BigQuery connection is straightforward and is often the default option available in the Looker Marketplace.

Cortex is a data framework that provides pre-built data models and metrics, accelerating the development of Looker applications. Together, they enable organizations to quickly create and deploy data applications that cater to their specific needs. Cortex Data Foundation is the core architectural component of the Cortex Framework reference architecture and provides packaged analytics artifacts that can be automatically deployed for use with Google Cloud BigQuery.

Steps for Looker Block Deployment

These are the step required for the installation of Looker Block Deployment :

1. Project Setup and Planning
2. Looker Block Installation
3. Data Connection and Modeling
4. Looker Explore and Dashboard Development

We will be talking about each mentioned steps above in details:

• Project Setup and Planning: Begin by setting up a new Looker project and defining the scope of your deployment. Identify the key business questions you want to address and the data sources you’ll need to integrate. If installing Looker Block from Marketplace make sure you have enabled the Marketplace in the Admin -> Platform -> Marketplace. Note: To enable the Marketplace you will need to have admin access in Looker.

Enable Marketplace in Looker

Once enabled you can access the Marketplace from right corner in Looker as mentioned below:

Marketplace Icon in Looker

• Data Connection and Modeling: Connect Looker to your data sources by creating a Database Connection and leverage the Cortex data models to structure your data. Customize the models to fit your organization’s unique requirements and create additional metrics or dimensions as needed. Setting up a database connection in Looker involves defining the connection details that allow Looker to interact with your database.

Follow these steps to configure a database connection in Looker:

Click on Admin -> Databases -> Connections -> Add Connection

After this enter the connection details by giving the connection name, Dialect (e.g. MySQL, PostgreSQL, BigQuery, etc.), Billing Project ID, Dataset, and Authentication (Service Account or OAuth). In order to setup. Persistent Derived Tables (PDTs): Enable this.

Big Query Connection

Once all fields are filled out, click Test These Settings. Looker will try to connect to the database using the information you provided. Once tested successfully save this connection.

Additional Tips:

• Service Accounts: If connecting to databases like Google BigQuery(Standard/Legacy), you may need to set up a service account and provide appropriate roles (like BigQuery Job User and BigQuery Data Editor).
• Persistent Derived Tables: You will need to create the Temp Database manually in BigQuery and Specify the Temp Database in the PDT Section.

Once the connection is set up, it’s ready to be used for LookML modeling and creating reports in Looker.

• Looker Block Installation for Cortex: Install the necessary Cortex blocks from the Looker Marketplace or set up the forked Github Repository. These blocks contain pre-built data models, metrics, and dashboards that align with common business use cases.

There are two ways of installing Looker Block depending upon the Model:

1. Installing directly from the Marketplace: If the Looker Block is available in the Marketplace installed directly from the UI. Example: we can install Cloud Cost Management: Google Cloud model from the Marketplace. This is the most simple installation using Marketplace. After clicking on a default install you can agree on the license and provide the connection name which contains the billing data and click on install as described below. This will create a Marketplace Project. Click on the Develop Panel -> Projects. You will be able to see the Cloud Cost Management Project.

2. Manually install Looker Block: The Cortex sap block in Looker is not directly available in Marketplace UI. You can still install this using one of the below options.

Option A: Marketplace Install via Git

Refer to the Looker documentation for Installing a Tool from Marketplace. Provide values for the required prompts. The only cons with this installation is you can’t customize the LookML to fit your unique business needs in case of any changes required in the pre-built Dashboards.

Option B: Manual Install via Fork of this Repository

With the Looker project based on your forked repository, you can customize the LookML to fit your unique business needs. Since Option B is the most used method for installing Looker Block to fit your unique business need will be talking about the detailed steps required to do this deployment end to end below:

• Step 1: Fork the Cortex SAP Block Github Repository by clicking on Fork as described below:

Fork the Cortex Repository

• Step 2: Once you fork the repository, create a blank LookML project with any name (e.g. cortex_sap_block) as described in the link. Once you create the cortex_sap_block you can configure the git as described in Step 3.
• Step 3: Connect the new LookML project to the forked repository

Configure Git

After clicking on Configure Git. Copy the SSH Github url of the repository as described below:

SSH Github url

Paste this copied SSH Github you will see the page as described below copy the Deploy key and paste this is Github when generating the SSH key as described in the below step.

To create the SSH Key in Github: Click on Github Profile -> Settings -> SSH and GPG Keys -> New SSH Key

• Step 4: Update the values of constants in the manifest.lkml file and commit the changes by validating the LookML. Once committed you can deploy to Production which will merge the code with the production branch. The Pre-built Dashboards will have cortex_sap_operational model since that’s the name of the file as highlighted below.

manifest.lkml

• Looker Explore and Dashboard Development: You can find the pre-built Dashboards in Folders -> LookML dashboards as described below. In case you want to fit your unique business need on top of these pre-build Dashboards you can change the LookML files or add new LookML files, commit the changes and deploy to production. On exiting the development mode you will be able to see these Dashboards.

Pre-built LookML Dashboard

Below is the snippet of some of the pre-built dashboards Finance Dashboard which are deployed:

Accounts Payable :
Find financial information such as accounts payable, accounts payable turnover, overdue payables, accounts payable aging, and cash discount utilization.

Accounts Payable Dashboard

Accounts Receivable:

Analyze total receivables, overdue receivables, days outstanding, and top companies with the highest receivables.

Accounts Receivable

Benefit of using Option B: Manual Install via Fork of this Repository:

One of the major advantages of using Option B: Manual Install via Fork of this Repository is you can add other LookML dashboards too with this by creating a separate model file in the same project, committing and pushing the changes to the local repository, and then deploy it to production. The below snippet talks about cortex_latest_inventory_operational.model which is another model file that you want to fit your unique business need on top of this pre-build Dashboards.

Additional Model File

Conclusion:

In conclusion, deploying the Looker Block for the Cortex Framework is a powerful way to enhance your data analysis capabilities. This guide has walked you through the essential steps, from project setup to the final stages of Explore and Dashboard development. While there are multiple ways to install Looker Blocks, the manual installation via a fork of the repository offers the most flexibility and is widely adopted by users. By following this comprehensive guide, you can leverage the full potential of Looker Blocks to drive insightful analytics within your organization.

References:

• https://cloud.google.com/looker/docs/blocks
• https://github.com/GoogleCloudPlatform/cortex-data-foundation
• https://cloud.google.com/looker/docs/connecting-to-your-db
• https://github.com/looker-open-source/block-cortex-sap
• https://cloud.google.com/looker/docs/marketplace#installing_a_tool_from_a_git_url

Hope you enjoyed this article and found it useful. You can reach me at LinkedIn.