The other day, I had to do some quick data wrangling with some client’s data, decided to just dump their data and spin up a local PostgreSQL database.

In this tutorial, we will create a PostgreSQL database and access it from PgAdmin using docker-compose. In my opinion, docker compose provides the easiest way to have your Postres container and PgAdmin on the same network.

Contents of docker-compose.yml

version: "3"
services:
postgresql:
image: postgres:11.8
container_name: postgresql
ports:
- 5432:5432
volumes:
- ./postgres/init:/docker-entrypoint-initdb.d
environment:
POSTGRES_USER: root
POSTGRES_PASSWORD: root
POSTGRES_INITDB_ARGS: "--encoding=UTF-8"
hostname: postgres
restart: always
user: root
pgadmin4:
image: dpage/pgadmin4:4.27
container_name: pgadmin4
ports:
- 80:80
environment:
PGADMIN_DEFAULT_EMAIL: root@root.com

You'll learn how to set up MySQL container for your local development using docker compose. You can easily start it and shut it down when you don't need it and your data won't get lost. Very useful when you just want to create a MySQL database quick for your local development.

Image for post
Image for post
  • ./db
  • ./db/sql/
  • ./docker-compose.yml

Our MySQL data will live inside ./db directory.

You can put any sql file into ./db/sql directory, it will run automatically when during build. (It's useful when you want to restore data from some dump file)

./docker-compose.yml is our docker compose configuration file.

docker-compose.yml

version: '3'services:
# MySQL
db:
image: mysql:5.7 …

TLDR; After receiving the API request, create a new thread and run your function there. You can pass your desired data in kwargs.

We’ve all been there. What to do when you want to use python in some batch process for your calculations but do not want to wait for the response 10 minutes after triggering your API endpoint. Here’s what you can do.

Image for post
Image for post

Our Scenario

  • Google Cloud Run: Python Flask API, ready to do some tedious long running task
  • Google Cloud Scheduler: send a POST request every hour to our Flask API
  • Execution takes around 10 minutes.
  • We do not want to keep Cloud Scheduler hanging for 10 minutes, instead we send back a response immediately and we want the API to continue execution. …


The rocky road from TF-IDF to BERT

Image for post
Image for post
Photo by Markus Spiske on Unsplash

About Us

By writing on Medium, I’ll try my best to give some insight on the research we do at the AI Strategy Department of Inglewood Inc. (Tokyo/Japan).
We are a team of 5 (growing fast!) data geeks with passion for machine learning, trying to find and solve interesting problems in the world.

I hope you’ll learn something new about different machine learning approaches and about what we do.

Intro

In this particular series of articles I will walk you through our attempt to predict annual salaries from job postings we collected from the Internet. …

About

Zoltan Varadi

Data Scientist at Inglewood Inc., Tokyo

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store