Data Science

Unleash Your Data Engineering Skills: Showcasing Projects, APIs, and the Path to Success — We can’t stress enough the importance of showcasing your Data Engineering skills to secure better opportunities. One key aspect that I always emphasize is the creation of a portfolio on GitHub, demonstrating projects you’ve worked on, whether functional or not. “But Cassio, can’t I just copy an existing repository and…

The Landscape Creating web apps entirely in Python is an enticing idea. Setting up a web app requires both frontend and backend skills such as HTML, CSS, JavaScript (and the numerous frameworks) together with Python or some other server-side language on the backend. It’s a lot of work! If instead everything could…

Understand why large companies are embracing data mesh — There are major ‘data earthquakes’ underway at many large organisations around the world. It is the decentralisation of the company’s data lake towards data mesh. At one of Australia’s ‘Big Four’ banks where I’ve worked in the analytics space for half a decade, we are smack middle of a huge…

Unleash the lion. — Summarizing text is one of the many tasks ChatGPT is extremely good at. Give it a piece of content and ask for a summary. You will be surprised how quickly it responds with a well-written summary. But do not just ask for a summary. By customizing your prompt, you can…

Best Practices, Techniques, and Tools to Fully Understand Your Data — Introduction Exploratory Data Analysis (EDA) is the single most important task to conduct at the beginning of every data science project. In essence, it involves thoroughly examining and characterizing your data in order to find its underlying characteristics, possible anomalies, and hidden patterns and relationships. This understanding of your data is…

Redefine the way you work with spreadsheets. — The increasing number of plug-ins has illuminated the power of ChatGPT, or other LLMs. These plug-ins enable integration of the OpenAI API with a variety of tools, thereby facilitating the use of models that OpenAI offers, including GPT-3.5 and GPT-4. One of the tools that can be enhanced with the…

Fine-tuning models with billions of parameters is now possible on consumer hardware — Most large language models (LLM) are too big to be fine-tuned on consumer hardware. For instance, to fine-tune a 65 billion parameters model we need more than 780 Gb of GPU memory. This is equivalent to ten A100 80 Gb GPUs. In other words, you would need cloud computing to…

In my previous article, I emphasized the importance of gaining a solid understanding of statistics as a crucial skill for aspiring data scientists. Although the list of skills in that article was not presented in a specific step-by-step order, statistical knowledge was rightfully given the first priority. In response to…

Introduction Have you ever wrote an email using Gmail? If so, you have probably seen that Google has implemented an algorithm which tries to finish your sentences. No, not in the romantic way, but rather shows you, in a slightly gray font color, what it thinks your completed sentence should…