Data Science code quality and craftsmanship culture
Code is everywhere ; in IT and operational technology, and nowadays in all digital interactions ; code powers all routines of our lives, from the tiniest to the biggest.
Trusting our lives to code brings forward new challenges, in particular reliability and transparency.
Producing code that is robust and easy to maintain is first and foremost a question of culture, the culture of quality. Helping this culture emerge in a company takes time, patience and a lot of care, because it involves not only individuals, but also teams and, eventually, the whole company.
You often pay the price of poor quality, on numbers of levels:
- On a human level: working in an environment where implementation quality is ignored is quickly demotivating. Eventually, good Data Scientists will leave rather than see their skills decrease.
- On a financial level: a poor-quality model soon transforms into a non-scalable source of bugs and regressions, becoming more expensive to maintain and harder to change.
- On a strategic level: achieving a high level of quality is not an end in itself. However, in a digital world, precision matters, delivering faster is paramount in order not to be overtaken by competitors.
Let’s examine further what code quality culture is and how to increase it at all levels, individual, team, and company.
What do we mean by code quality?
When we talk about code quality, we mean technical excellence for the model implementation, professionalism in the code development and full end to end responsibility for everyone involved.
Technical excellence does not only mean an adequate model implementation. It also means that the product has been properly crafted technically, which eases maintainability and guarantees efficient execution.
When using the term professionalism, we do not only refer to the skills of individuals, but also to their ability to balance their efforts with the value created (rather than seek ‘absolute’ perfection).
And finally, end to end responsibility refers to taking a strong personal commitment to contribute to the company’s goals, i.e., being driven by a common purpose. Personal commitment grows when people work and make decisions together and feel successful at what they do, a strong case for promoting self-organisation in the workplace.
Combining these three qualities (technical excellence, professionalism, commitment) usually result in higher code quality, therefore better products and stronger customer satisfaction.
How can we grow a culture of code quality?
The other question is: How can we nurture quality values so that they become more frequent rather than the exception?
We all know that a culture change cannot be enforced. However, culture can change through contact with other cultures.
Let us dive into software craftsmanship, a more and more common practice in the field of software development, widely recognised for its focus on the quality of its realisations. Software craftsmanship is an approach to software development that highlights practices that achieve software quality, arguing that it is not enough for software to work, it must also be well designed.
In 2009, Robert C. Martin offered to “raise the bar” in terms of software development considering that the Agile Manifesto did not insist enough on quality in realisations. He made the following manifesto:
Not only working software,
But also well-crafted software
Not only responding to change,
But also steadily adding value
Not only individuals and interactions,
But also a community of professionals
Not only customer collaboration,
But also a productive partnership.
The “Craft” culture is a set of values empowering people to always strive to do better. The values are accepted individually and shared collectively. Sharing these values with colleagues could be a good first step to spread a new culture of quality and producing well-crafted code.
But how can you concretely help a culture of quality emerge in your company?
Cultural change is based on daily practices and also supported by an environment that promotes these practices. Since Data Science is changing daily, meeting with others and continuous learning are key to anchor meaningful changes.
At an individual level, you can:
- Read books and blogs,
- Train yourself by practising new tools and techniques,
- Meet other people at conferences & meetups,
- Find and interact with mentors.
To share and practice these values with your team, you can:
- Organize collective or pair code review to guarantee collective ownership of the code,
- Learn and share development practices through pair modelling,
- Establish and use standards together (for ex. Cookiecutter template for project management),
- Give and receive positive & constructive feedback (remember that feedback is a gift to the other).
At the company level, the challenge is to foster a healthy and collaborative environment where quality is valued on equal footing with achieving business objectives. You can start with a few concrete actions:
- Support continuous improvement by promoting self-organisation in teams,
- Create meeting places between teams and fostering the emergence of communities of practice,
- Dedicate a budget to learning,
- Identify and support influential leaders that embody and disseminate the values of quality.
Emerging the culture of quality in a company is a long-term initiative, it’s a journey that will require commitment, cooperation and mastery from individual to corporate level.
Saagie Data Fabric creates and deploys data pipelines and analytics models into production while respecting your IT/Development situation. Feel free to quickly check our user guide on jobs/pipelines versioning available.