liveBook · Manning

Modularity

As explained in Software Engineering for Data Scientists

Modularity is the organization of software into separable components with clear relationships and relatively independent responsibilities.

Modularity organizes software into smaller, self-contained units that perform particular tasks and can be developed, maintained, shared, and understood independently. In data science, it replaces a long notebook or Python script with an organized sequence of pipeline components. section 3.3

A modular machine learning workflow can separate responsibilities into files such as:

Separate functions and files improve readability and portability. A cleaning module can define functions for reading data, capping outliers, and coordinating those operations through a higher-level function. The guarded __main__ block allows other scripts to import these functions without executing the module's main code. section 3.3.1

Cleaning customer churn data

A script can define clean_data and import it elsewhere with `from code_modularization_example import clean_data`. The calling script reads the customer churn dataset and applies `customer_data = clean_data(customer_data)` without rerunning the module's complete execution workflow. section 3.3.1

Figure 3.1 This diagram demonstrates how we might organize a sequence of modular files, each performing a separate task as part of an overall goal - in this case, implementing a machine learning model.
ch3 modularity

Use the workflow diagram to trace how separate files perform sequential tasks within an overall machine learning goal. section 3.3

How Software Engineering for Data Scientists covers it

How deep it goes
Explains modularization through file organization, reusable functions, imports, and a guarded main block.
Examples
Uses ordered Python pipeline files, customer churn cleaning functions, pandas data loading, and imported reusable code.
What you can do after
Build modular Python workflows that improve readability, portability, maintainability, and production integration.
How it connects
Connects functions and modules with data processing, feature engineering, model training, evaluation metrics, and production pipelines.

Also explained in

Compare how 16 books explain modularity →

Related concepts

This explainer was generated by Manning's knowledge graph from the text of Software Engineering for Data Scientists. The section links point to the parts of the book it draws on.