There’s a side to data science that often goes unnoticed but is equally important.
Let me unveil my observations, drawn from my own 3+ years of data science & AI engineering career.
Intro
The ability to code, analyze complex algorithms, and work with large datasets is undeniably crucial.
No doubts. I’ve written already plenty of articles about technical skills & recorded 20+ podcast episodes about it.
However, there is another side to data science that often goes unnoticed but is equally vital: non-technical skills.
As a data scientist & data engineer, I have witnessed the difference between OKayish data scientists and GREAT data scientists.
Believe me, the difference is not where you think it is.
Let’s shed some light on the importance of honing these often-overlooked skills!
1. Hacker’s Spirit
Of all the non-technical skills this one is the closest to being technical.
The latest research shows that 42% of Data Scientists have at least a Master’s Degree. Almost 40% have a Ph.D.
So you can be fairly confident that any candidate who makes it through the screening phase and into the technical interview has a baseline competency in one of the most common programming languages e.g. Python or R.
BUT!
More important than knowing how to code in one of those fancy programming languages is a Hacker’s Spirit — or in other words, how you can react to working with:
- New unfamiliar coding languages
- Untypical data formats
- Custom tools within the company
- Challenging programming tasks
- Unknown libraries, etc.
This is known as a “Hacker’s Spirit”.
Every exceptional data scientist and engineer I know possesses an inherent hacker mindset.
They break things down, hack them & try to find shortcuts wherever they can.
2. Thinking Stats
For me, statistical thinking is a way of understanding a complex world (usually in the form of various data formats) by describing it in relatively simple terms that nonetheless capture essential aspects of its structure + provide some idea of how uncertain we are about our understanding.
This is the way people can fight their biases, challenge assumptions, question the results, etc.
Example:
In recent years most Americans have reported that they think that the number of violent crimes increased compared to the previous years (Pew Research Center).
However, a statistical analysis of the actual crime data shows that in fact, violent crime has steadily decreased since the 1990s.
Intuition fails us because we rely upon best guesses.
Statistical thinking provides us with the tools to more accurately understand the world and overcome the fallibility of human intuition.
Here are some tips on how to be better at thinking stats:
- Build a strong statistical foundation
- Learn from real-world examples
- Understand biases + ask the right questions
- Seek continuous learning
- Cultivate critical thinking
3. Data Intuition
Merely possessing a data science degree or certification does not automatically equate to possessing strong data intuition.
Data intuition is the ability to understand and interpret data patterns, trends, and insights based on your’s experience, knowledge, and instincts. Data intuition empowers us (data professionals of any kind) to make informed decisions and draw meaningful conclusions from data, even in situations where the information may be incomplete or ambiguous.
Data scientists with developed data intuition excel not only in identifying patterns within structured and unstructured datasets but also in discerning business or customer needs and project pain points, etc.
It is developed over time. Primarily through experience working with various datasets and in various domains, but not only.
It takes time to master it.
From my personal perspective, mastering data intuition can be done through:
- A tedious process of analyzing tons of data
- Constantly improving mathematical & statistical knowledge
- Asking the right questions
- Problem/challenge decomposition to the lowest level possible
Data intuition might be compared to the previous non-technical skill (thinking stats), but it focuses more on domain knowledge rather than a statistical way of thinking.
4. Team Work
Great things in business are never done by one person. They’re done by a team of people.
- Steve Jobs
Contrary to the perception of data scientists working in isolation, they are intricately woven into the fabric of the organization, and actively engaged at various levels in any kind of business throughout.
On one hand, they will have to collaborate with the teams to understand requirements and gather feedback to reach beneficial solutions.
On the other hand, they will have to work with fellow data scientists, data architects, and data engineers in order to handle the data. This collective effort ensures the seamless execution of all the data tasks, as they leverage each other’s expertise, pooling their knowledge to unlock the full potential of the data at their disposal.
That’s a wrap!
Thanks for reading! I hope you enjoyed it!