Oyeah GCP!!!

Since the advent of virtualization our data has started migrating from our systems, both metaphorically and literally to what has come to be known as:the cloud. It is this very concept that has facilitated the fields of Iot, machine learning, security,big data etc, simultaneously exalting storage and networking for systems and data centers all over the globe.

But the age old question remains, how does one crack their knuckles, get on the keyboard and get started?

With this article, I take up one such cloud computing service-Google Cloud Platform(GCP), and the cherry on the cake being that it comes under the public suite services of Google.

Here we will take up the requirements from the very fundamentals and continue to build up on it!

STORAGE NEEDS

• Almost all applications need persistent durable storage
• User accounts, images, events, documents, logs, etc
• Different applications have different storage needs
• Google Cloud Platform offers seven storage options — Bigtable, Cloud Datastore, Cloud SQL, Cloud Storage, Cloud Spanner, Persistent Disk and BigQuery
• BigQuery is both a storage service and a powerful analysis tool

KEY FEATURES

· Single API across all storage classes.

· Scalable to exabytes of data.

· Designed for 99.999999999% durability!

· Very high availability across all storage classes.

· Time to first byte in milliseconds.

· Strongly consistent listing.

There are various type of Data Storage available and they are as follows, one can choose from a plethora of options, based on their preferences:

1. Spanner

2. Bigtable

3. DataStore

4. Firebase

5. Sql

6. Persistent Disk

STRUCTURED AND UNSTRUCTURED

· We can divide these seven storage services into two categories: structured and unstructured.

· If the data you want to store can be organized into a table structure with columns and rows, then it is structured data.

· Examples: user profile information, event logs, sensor measurements, sales records, or stock trade data.

· Structured data comes in many different shapes, sizes, and usage patterns, and there is a great diversity of ways to store it and interact with it.

· Cloud SQL, Cloud Datastore, Cloud Bigtable, Cloud Spanner, Persistent Disk and BigQuery all store structured data.Google Cloud Storage stores unstructured data

· Provide Cloud Storage a sequence of bytes to store, a place you want to store it, and a name to identify that particular sequence of bytes, and it stores them.

· Cloud Storage provides no insight into the internal structure.

· Data is stored in Buckets as Objects.

Now, we shall take a deeper look at each of the available data storages, their descriptions, advantages and workloads:

PERSISTENT DISK

Description:

· Fully-managed, price-performant block storage that is suitable for virtual machines and containers.

This is good for the following:

· Block storage for Google Compute Engine and Google Kubernetes Engine.

· Snapshots for data backup.

Common Workloads:

· Disks for virtual machines.

· Sharing read-only data across multiple virtual machines.

· Rapid, durable backups of running virtual machines.

CLOUD BIGTABLE

Description:

· A scalable, fully-managed NoSQL wide-column database that is suitable for both real-time access and analytics workloads.

This is good for the following:

· Low-latency read/write access

· High-throughput analytics

· Native time series support

Common Workloads:

· IoT, finance, adtech

· Personalization, recommendations

· Monitoring

· Geospatial datasets

· Graphs

CLOUD DATASTORE

Description:

· A scalable, fully-managed NoSQL document database for your web and mobile applications.

This is good for the following:

· Semi-structured application data

· Hierarchical data

· Durable key-value data

Common Workloads:

· User profiles

· Product catalogs

· Game state

CLOUD SQL

Description:

· A fully-managed MySQL and PostgreSQL database service that is built on the strength and reliability of Google’s infrastructure.

This is good for the following:

· Web frameworks

· Structured data

· OLTP workloads

Common Workloads:

· Websites, blogs, and content management systems (CMS)

· Business Intelligence (BI) applications

· ERP, CRM, and eCommerce applications

· Geospatial applications

CLOUD SPANNER:

Description

· Mission-critical, relational database service with transactional consistency, global scale and high availability.

This is good for the following:

· Mission-critical applications

· High transactions

· Scale + Consistency requirements

Common Workloads:

· Adtech

· Financial services

· Global supply chain

· Retail

CLOUD BIGQUERY

Description:

· A scalable, fully-managed Enterprise Data Warehouse (EDW) with SQL and fast response times.

This is good for the following

· OLAP workloads up to petabyte-scale

· Big Data exploration and processing

· Reporting via Business Intelligence (BI) tools

Common Workloads:

· Analytical reporting on large data

· Data Science and advanced analyses

· Big Data processing using SQL

CLOUD STORAGE

Description:

· A scalable, fully-managed, highly reliable, and cost-efficient object / blob store.

This is good for the following:

· Images, pictures, and videos

· Objects and blobs

· Unstructured data

Common Workloads:

· Storing and streaming multimedia

· Storage for custom data analytics pipelines

· Archive, backup, and disaster recovery

MOBILE STORAGE OPTIONS

· Cloud Storage for Firebase: Mobile and web access to Google Cloud Storage with serverless third party authentication and authorization.

· Firebase Realtime Database: A realtime, NoSQL JSON database for your web and mobile applications.

· Firebase Hosting: Production-grade web and mobile content hosting for developers.

· Cloud Firestore for Firebase: A NoSQL document database that simplifies storing, syncing, and querying data for mobile and web apps at global scale.

It can be a common dilemma to see which one works best on GCP(Google Cloud Platform) and in the beginning even I was in a conundrum myself and how to decide on the best fit?

And having the luxury of many options of Storage like Big table, DataStore, FireStore, FileStore, Storage, SQL, Spanner and MemoryStore can add to the confusion. So after being done a lot of research on this question, at last I concluded with the result that has been shown below.

To get a clearer idea of the perfect platform, you have to think from different perspective and ask yourself questions like:

Is your Data Structured?

Do you need a Mobile SDK?

Is your workload Analytics?

Is your data relation?

Do you need horizontal scalability?

Do you need updates or latency?

Which One To choose?

Once you’ve answered these questions for yourself, the best option will automatically present itself. All you need to do is to know in which direction your project is headed, what your data looks like and what outcomes you expect. And with this guide you can get started with everything GCP.

I hope you like this blog. Keep tuned for more such blogs.

Happy codding!