Why It Feels Impossible to Get a Data Science Job
Apparently, "data is the new oil," with huge demand growing yearly, so why does it feel so hard to get a job as a data scientist right now?
Well, in this article, I explain why this is the case and provide advice on what you can do right now to increase your chances of getting hired!
The Economy
It's no secret that times are tough at the moment. There are terrible things going in the world, financial stability has reduced, and people are overall struggling. Overall, the economy is not in a good position as it was in the past, which naturally causes businesses to suffer.
In the UK, interest rates were 0.25% at the start of 2022 and are now 5.25%. That is a steep increase, especially considering interest rates were practically nothing for the past decade. They have been below 1% since the 2008/2009 financial crisis.
An increase in interest rates means the cost of borrowing goes up for tech startups, making it harder to stay cash flow positive and keep their financial stability. In extreme cases, this can even lead to bankruptcy.
With this increase in risk, Venture Capitalists are less likely to invest their money into companies, particularly tech start-ups, which are notoriously riskier than other businesses.
This decrease in funding means that companies have less money, so they have to be more cautious with their hiring and are less likely to take on more employees than they need.
Below is an incredible deep-dive article by Gergely Orosz on how interest rates affect tech startups.
The end of 0% interest rates: what it means for tech startups and the industry
Unemployed Talent
Layoffs have been everywhere in tech recently, mainly due to the economic situation I discussed above.
According to this BBC article, 23,670 workers were laid off across 85 tech companies in January 2024 alone, including big tech companies like Microsoft and Google.
There have also been massive layoffs at other tech companies, like Spotify, which cut 17% of its workforce in December 2023, and Twitter/X, which laid off around 80% of its employees since Elon Musk took over in April 2022.
Data scientists have been laid off less in general than software engineers, but it's still a considerable amount.
These companies have some of the world's most talented data scientists, engineers and general tech professionals. Securing a position at one of these firms is very tough, and you are likely in the upper echelon of practitioners in your field.
The problem now is that all these fantastic people are looking for the same jobs as regular data scientists trying to break into the industry. Couple this with the fact that the number of open tech jobs is over half less now than in 2022, and the environment is tough to compete in.
Supply vs Demand
Since Harvard Business Review quoted Data Science as the "sexiest job of the 21st century" in 2012, the number of data scientists and people wanting to be data scientists has drastically increased. Most large and medium companies nowadays have data scientists or some related data professionals.
Loads of people are taking online courses doing master's degrees in machine learning, and there are now even three-year bachelor's degrees dedicated to data science. This is amazing considering how new the field is compared to other subjects like physics or engineering.
Obviously, the supply of data science positions, particularly junior and grad roles, is rising year on year. However, as we saw in the previous sections, the opposite is true for the demand, or at least the demand is not meeting the supply rate.
There are essentially not enough roles available for everyone who wants to be a data scientist, and companies don't want to risk hiring someone with no experience due to their tighter financials due to the economic situation. This makes it incredibly to tough to break in for entry level positions.
The Value of Data Science
As my good friend Matt Chapman quoted in his recent post,
"ML models inside Jupyter notebooks have a business value of $0"
Gone are the days when data scientists would build PoCs in Jupyter Notebooks. Senior management practically gets no benefit from this, and simply doing model.fit() and model.predict()just doesn't really cut it anymore.
There seems to be an increase in chat about the idea of a "Full-Stack Data Scientist" or "ML Engineer", basically someone who can implement a data science solution end to end into production, so it can actually generate value.
Check out this article from Shaw Talebi if you want to learn more about Full-Stack Data Science.
From my perspective, this trend is a natural evolution. Building models has become remarkably straightforward, with libraries like XGBoost, PyTorch and good old Sci-Kit Learn doing most of the heavy lifting.
Not to mention, we no longer even need to spend hours training models; we can simply access massive state-of-the-art models from large corporations in seconds by pinging a model's API using tools like Hugging-face or LangChain.
See this tutorial on how to call SOTA models using LangChain; it's surprisingly reasonably simple.
Unfortunately, most data science courses, degrees and bootcamps don't teach these things and still focus a lot on theory. This is not a bad thing; I think any top data scientist should have the baseline theoretical knowledge to excel.
However, companies want you to generate impact. They can no longer afford R&D like they used to due to tighter budgets, so you need to know how to implement your solution, and many entry-level candidates probably need to gain this knowledge.
To overcome this, try to up-skill yourself in model deployment and software engineering. These tools are invaluable and will really serve you well in the long run. I plan on writing an entire article on the knowledge you need and how I plan to learn these things.
What Can You Do?
I appreciate that it is very hard to land a role at the moment, particularly if you have no previous experience and are fresh out of university or college.
There is no silver bullet that I can share that will immediately get you a data science job unfortunately.
However, if you're approaching every job application the same with little success, then maybe it's time to change it up a bit, and one of the best ways is to try to make yourself stand out.
I have a separate post explaining exactly how you can stand out in your data science application giving you an edge over other applicants. The entire article is linked below, but let me give you a quick summary in this post.
To stand out, you must do things other people are not doing and leverage your unfair advantages. Knowing exactly what you should do can be difficult though. Some straightforward ideas to try are:
- Write technical articles and share them. This is unbelievably easy; I am surprised that more people don't do it. All you have to do is learn about topic X and write a post about topic X. For example, learn about functions in Python and write a blog post about functions in Python. Your goal is not to go viral but to expand your reach, as every person who reads it learns who you are, and this can open more opportunities than you think.
- Do Kaggle competitions and try to do well in them. If you want to work in a specific data science area, compete based on that domain. This will teach you more than you think, not to mention show people your work.
- Create a detailed website or portfolio to showcase your work and enthusiasm for the field. This will help paint a picture of who you are and show that you are serious about data science.
- Presenting or publishing at conferences is more challenging, but it will unlock substantial networking opportunities if you can do it. This is mainly applicable if you are doing something like a data science or Machine Learning master's degree.
From experience, many data scientists, particularly at the entry level, won't have these, so they will give you that edge during the interviews. It's the classic Pareto principle, doing 20% will give you an 80% advantage.
Other advice that I can give you and I have seen be successful for other people is:
- Leverage your network as much as possible to get that first foot through the door. Reach out to people in the industry and ask them to refer you, even if you are not "best friends."
- If you are transitioning into data science, apply for companies in an industry similar to your previous profession. For example, if you worked in marketing, apply for a position related to data science. I have seen loads of people land roles as data scientists as they have significant domain knowledge.
- Try for similar roles like data analyst or engineer to build your skills and then move over to data science in a year or so. Again, I have worked with many people who were analysts and then moved over to scientists, and they said the transition was relatively straightforward, especially if they moved internally.
You obviously should do what you think is right for you, but these tactics may be worth trying if you are really struggling. The point is to try to do different things to other people.
This is an example of the "side door." Instead of going straight to the front door (regular applications) or the back door (reserved for celebrities), carve your own entry using the side door. A great article on this topic is linked below if you want to read more about it.
There is no "correct" way to get a job, particularly when trying to break into an industry. So, use all the advantages you have at your disposal if you can.
Summary & Further Thoughts
As we can see, multiple factors are affecting the data science job market right now, and they are all interlinked in some way. Due to the economic conditions, there are more layoffs and fewer jobs. This leads to more competition and a greater supply than demand, making it tougher to get a job.
You can increase your chances by making yourself stand out by writing articles, creating a portfolio, and entering Kaggle competitions. You can also utilise your network to get your foot into the door. I appreciate that it's tough now, but there are ways to break in.
Another Thing!
I have a free newsletter, Dishing the Data, where I share weekly tips and advice as a practising data scientist. Plus, when you subscribe, you will get my FREE data science resume!
Connect With Me!
- LinkedIn, X (Twitter), or Instagram.
- My YouTube Channel to learn technical data science and machine learning concepts!
