Database, Data Warehouse, Data Lakes.
A large portion of the time producing a paper is used asking your colleagues about the data and loading and merging tables.
By the time the frame is ready, you almost forgot why you wanted that dataset.
Build transformation blocks, starting from those ones that hide any data source and any data merge. The rest is will follow.
Exquisite code (paraphrasing cadavre exquis, literally exquisite cadaver), is a propose method by
which a program is collectively assembled.
Each collaborator adds to a line of code in sequence,
either by following a rule (e.g. "the line should be a function that takes the previous line input
and produces a new output") or by being allowed to see only the last line of code that the previous person contributed.
Think about functional programming. Build cohesive intellectual blocks. Named them transformations.
Create a framework to allow these IP blocks to be used in functional programming pipelines.
Add IP blocks as needed. Retain transformation workflows. See how new joiners are capable to learn from your retained IP.
See how new joiners contribute with new pipelines and IP blocks. Allow people to play Lego. Retain the blocks and the best models.
Have you ever try removing all the ifs in your code?