Creating a system that works in a changing world is challenging - DSBoost #8
Discover more from DSBoost
Creating a system that works in a changing world is challenging - DSBoost #8
Welcome to the 8th issue of DSBoost, the weekly newsletter where you can discover interesting people in the ML/AI world, get the main takeaways of a relevant podcast, and stay up to date with the latest news in the field!
💬 Interview of the week
This week we interviewed Rune. Enjoy:
What did you study/are you studying (if your background is different from DS, how did you end up in the field)?
I studied Computer Science back before Data Science was a big thing (no real education) and I finished a Ph.D. in 2009. Since then I have worked with Big Data backend systems providing data to analysts (both data analysts and data scientists).
What are your favorite resource sites and books (ML/AI)?
I actually have old-fashioned RSS feeds from many sources - but Reddit is a great place to get trends and general views.
What got you into your current role (portfolio, certification, etc.)?
Big data is "easy" to solve on small scale, that is, if your company group only has a few data sources and few simple requirements in the preparation of data. The challenge growths exponential with the number of sources and end-user requirements. It all started with someone building a small-scale proof-of-concept in a few weeks and sold the idea to the senior management, that it would be easy to build a company-wide big-data platform. This turned out not to be the case. This attracted me to the world. The challenge and the difficulty.
What do you enjoy the most in your work?
I actually love helping others. There is so much potential in every organization, and I just love helping that potential get the skills or insights they need. This is why teaching Python is a passion of mine.
What tools do you use the most / favorite tools?
PyCharm - I feel it helps me the most. It makes a lot of things easy for me.
Do you use ChatGPT or other Al tools during your work? If so, how do they help you? Do they change your approach to problems?
I do use them. I use it more in text writing - getting inspired, lookup how you can explain things. I use it mostly as a writing assistant. As a programmer, I think it will get better. In some areas, it seems to have great potential and is getting close. As far as I see, AI will be a better and better assistant in helping developers create code.
What is your favorite topic within the field?
I love the development of AI and follow the long discussion of how/if/when AI can replace the majority of developers out there. That discussion started for me more than 10 years ago. There is nothing new about it.
Which one of the recent AI/ML models will have the most significant impact on the industry in your opinion?
I think it is funny. Since I started professionally in 2009 it has only become easier to develop code, with great IDEs and awesome frameworks. For instance, with FastAPI I can set up a REST-API fully functionally and ready for real-world production in a matter of few minutes. Back in 2009, it would take months to create something similar, and the risk it would not be stable would be high. This implies that it has become cheaper to develop software, but still, the demand his over the years increased.
Hence, the impact is difficult to predict. I think AI will make it even cheaper to create awesome applications faster and cheaper, but this might even create a bigger demand for people that use their skills to craft them fast.
From idea to production will be faster and cheaper.
What are you currently learning or improving (topics you are interested in nowadays)?
I follow the modern trends with OpenIA, GPT-4, etc. It is important to see the potential in it. I am very impressed by how far it came.
What is the biggest mistake you've made? (DS related)
Data is a complex thing. Most think data is just data. But data needs to be transformed, continuously monitoring quality, and integrity, I could go on. When you build systems, you want to make one-size-fits-all data processing.
This is impossible when the number of data sources grows.
Believing and trying to build that is probably one of the biggest mistakes.
What is your most significant achievement? (preferably DS related)
The world changes constantly. Creating a system that works in a changing world is one of the most challenging things. I think, my biggest achievement is understanding that simple statement and trying to follow it. We think we just need to build this one thing - but when we finish it, the world has changed.
Can you share a fun fact about yourself?
I train Brazilian Jiu-Jitsu to find the inner piece.
What do you find the hardest about creating content on YouTube?
Not to fall prey to a click-bait attitude in creating content.
Example. This one guy posts a video about (fictional title to not point fingers) "Avoid looking like a complete NOOB as a Python developer" - then he shows two ways NOT to write Python code, because it will reveal how NOOB you are. Then he shows you how to write code. This sounds like good advice, right? To be honest, I think not, as a developer for many years I have seen no developer care about what he showed. We are paid to make code that works and is easy to maintain, not to make fancy one-liners. What a video like that does, makes people insecure about their skills. I feel embarrassed that people make such videos, that do not encourage people to learn what matters but focus on making people more insecure. But I see the point. A video like that gets 100k-1m views. Hence, it is very tempting to promote such things. But I want to be true to helping people, not to make them insecure.
What would you do differently if you were to start now?
I started in corona lockdown. I started on YouTube, Facebook, and Twitter. I thought there would be synergy between it. I thought I could get subscribers and viewers from one platform to another. That is not the case. Focus only on ONE. Make it great. I still struggle a lot from not following that advice.
Please give one piece of advice for someone who wants to start sharing ML/DS/Python content on YouTube.
Just start. I felt really embarrassed about it. But just start and improve one thing at a time. Focus only on YouTube and grow that. We tend to compare ourselves to the best, those that post the first perfect video and get 100k subscribers, it can be discouraging. But focus on your own journey. Only do it if you really enjoy it - feel proud of every single person you help. You are helping this world become a better place.
🎙️ Podcast of the week
SDS 661: Designing Machine Learning Systems with Chip Huyen
Key takeaways:
The key to using machine learning models effectively is to use them for augmenting creativity in specific tasks, rather than relying on them for definite answers. It's important to focus on solving problems rather than being technology-oriented.
When aligning a machine learning project with business intent, context, and metrics, it's crucial to first understand the problem and its root causes. Often, solutions don't necessarily require machine learning.
Training data is also critical, and important considerations include sampling, labeling, class imbalance, and data augmentation.
The use of brute force methods, such as scaling up models with more computing and data, has been a highly effective approach, and many recent breakthroughs in AI have come from this approach.
Leveraging fresh data in real-time machine learning can significantly improve accuracy and reduce the time between the event and the response. However, this comes at a cost and requires efficient optimization of resources.
Thanks for reading DSBoost! Subscribe for free to receive new posts and support my work.
🧵 Featured threads
🤖 What happened this week?
GPT-4 was finally released! An improved version of GPT-3 with multimodal capabilities! And you can already try it if you have ChatGPT+.
Microsoft announced Microsoft 365 Copilot, which will enhance productivity in daily apps like Word, Excel, PowerPoint, Outlook, and Teams. Also, Microsoft released Business Chat which can generate status updates based on natural language prompts and data from various sources.
Soon you will be able to create your app easily using AI with FlutterFlow AI Gen.
👥 Under the radar
In this issue, we feature Muhammad Anas, a 14 years old Machine Learning enthusiast with a brilliant future ahead! Here are some words from him:
Hello there! My name is Muhammad Anas and I'm a 14-year-old ML maniac.
Ever since I first discovered machine learning, I've been fascinated by its endless possibilities and the power it has to change the world. I've spent countless hours learning about different ML algorithms and experimenting with various datasets, always looking for new ways to push the boundaries of what's possible. But what really excites me is the prospect of applying ML to algorithmic trading. I find the idea of using data and technology to make smarter investments incredibly intriguing, and I believe there's huge potential for innovation in this field.
I am also working on creating a community for ML beginners who share my passion. I want to provide a supportive space where we can learn from each other, collaborate on projects, and inspire one another to push the limits of what's possible. Join me on this exciting journey of discovery and innovation!