The job of an engineering team is to build effective, scalable products according to spec. The job of a product team is to understand and build solutions for users. What’s the job of a data team?
There’s no one or clear answer, which helps explain why the field is such a complete shit show right now. There are two broad mandates that data teams tend to get formed with (I’m being overly simplistic on purpose, bro):
1) Provide data to the company
2) Provide insights to the company
These might sound similar — and they’re certainly both important — but…
Airflow — it’s not just a word Data Scientists use when they fart. It’s a powerful open source tool originally created by Airbnb to design, schedule, and monitor ETL jobs. But what exactly does that mean, and why is the community so excited about it?
Every company starts out with some group of tables and databases that are operation critical. These might be an orders table, a users table, and an items table if you’re an e-commerce company: your production application uses those tables as a backend for your day-to-day operations. This is what we call OLTP, or Online Transaction…
It’s the first Tuesday of the month, and your Finance team is hounding you about those end of cycle revenue numbers. You’d help them, but you can’t focus — your dashboards are crumbling to the ground one by one as error messages fill up your browser window. Your Slack channels begin to fill up with angry messages from Product Managers about their need for up-to-the-second data about some product that the company deprioritized months ago. You try to escape but to no avail: you have a bi-weekly sync with your fate…erm manager.
The wave of AI and machine learning is happening just as the dominance of mobile is becoming set in stone. As mobile devices become more ubiquitous and powerful, a lot of the machine learning tasks we think of as requiring months of high-powered compute time will be able to happen right on your phone. This post will outline why edge devices are increasingly important, and how machine learning works with them.
Of all of the major technology shifts over the past decade, the migration to mobile and edge devices is one of the most prominent:
(Disclaimer: I’m not employed by Algorithmia or affiliated financially with the company in any way. I’m just someone with a Data Science background who finds the company compelling.)
There’s no doubt that we’re entering the age of AI, with Machine Learning touching almost everything we’re involved in on a day-to-day basis. Spurred on by step innovations in data storage and computing power, Neural Nets are back from the 70’s with a bang. Medicine, security, customer service, fraud detection, you name it — there are well funded companies applying Machine Learning to improve and augment it. …
(Disclaimer: I’m not employed by Zodiac or affiliated financially with the company in any way. I’m just someone with a Data Science background who finds the company compelling.)
One of the most interesting shifts that technology has spurred over the past decade is the consumer-ization of enterprise. While the default software providers of a previous age (think: Microsoft, Oracle) were able to sell commoditized, poor-UX software through lock-in economics, things are changing quickly. The new age of software providers are customer focused. The Slacks of today actually care about their customer experience, and design their products accordingly.
While the increased…
Technology is an agent for change, but that change can manifest itself in a number of different ways. New technology can change the way we act or feel, but it can also shift our way of thinking and looking at things. For example, the Graphical User Interface (GUI), finished in 1973 at Xerox’s research labs in Palo Alto, enabled computer users to see what they were working on for the first time and think about computer assets in an abstracted way. Advances in mobile processing power and usage patterns are forcing us to think of the mobile phone as the…
Note: all of the following applies to creating and training AI and Machine Learning models. Deploying those models at scale is a much different and more difficult challenge, and will be the subject of a follow-up post.
I like to say (often to myself) that things are getting both simpler and more complicated in the world of writing software.
On the one hand, the expectations for an MVP today exceed anything we’ve ever seen — software is expected to run and not crash, work across devices, etc.
As a Data Science grad from NYU and a VC analyst, I’m lucky to get to see and evaluate a lot of startups that are involved with AI (note: this term has some baggage. For the purposes of this post, I use AI and ML interchangeably to make things simpler). Heck, I’ve seen AI applied to some of the most obscure topics you can imagine, ranging from industrial energy usage all the way to finding the right GIFs. …
To say that there’s disagreement about whether VCs are aligned with their founders would be a serious understatement. There seem to be two competing narratives about those creatures known as Venture Capitalists operate: