reproducibility
Table of Contents
- 1. reproducibility
- 1.1. ReScience/ten-years: Ten Years Reproducibility Challenge
 - 1.2. Replication crisis - Wikipedia
 - 1.3. Is the reproducibility crisis reproducible? - by Ben Recht
 - 1.4. Reproducible Data Science and why it matters | by Carl W. Handlin | Rappi Tech
 - 1.5. Barriers to reproducibility — The Turing Way
 - 1.6. Reproducible Builds — a set of software development practices that create an independently-verifiable path from source to binary code
 - 1.7. GoogleCloudPlatform/professional-services-data-validator: Utility to compare data between homogeneous or heterogeneous environments to ensure source and target tables match
 - 1.8. Outreach and Publications - Research Object Crate (RO-Crate)
 - 1.9. Tazro Ohta Verifying workflow reproducibility using RO-Crate - YouTube
 - 1.10. Diffs
 - 1.11. Maneage – Managing data lineage
 
 
1. reproducibility
1.2. Replication crisis - Wikipedia
https://en.wikipedia.org/wiki/Replication_crisis
Cita trabajos de Luhman acerca de porqué pasa, en general por que la gente tiene distintos códigos sobre lo que es importante y lo que no
Social systems theory, as expounded in the work of German sociologist Niklas Luhmann, offers a similar diagnosis.
This theory holds that each system, such as economy, science, religion or media, communicates using its own code: true and false for science, profit and loss for the economy, news and no-news for the media, and so on.
According to some sociologists, science’s mediatization, its commodification and its politicization, as a result of the structural coupling among systems, have led to a confusion of the original system codes.
If science’s code of true and false is substituted with those of the other systems, such as profit and loss or news and no-news, science enters into an internal crisis.
1.5. Barriers to reproducibility — The Turing Way
- Limited incentives to give evidence against yourself (if I share my data people may find mistakes)
 - Publication bias towards novel findings
 - Held to higher standards than others
 - Not considered for promotion
 - Big data and complex computational infrastructure
 - Being reproducible does not mean the answer is right
 - Takes time
 
1.6. Reproducible Builds — a set of software development practices that create an independently-verifiable path from source to binary code
Sponsorized by google
1.9. Tazro Ohta Verifying workflow reproducibility using RO-Crate - YouTube
1.9.1. sapporo-wes/tonkaz: CLI tool to verify workflow reproducibility
Tonkaz is a CLI tool to verify workflow reproducibility. It compares the RO-Crate of workflow execution results and calculates the reproducibility level of each output file.
- Level3 ⭐⭐⭐ : Files are identical with the same checksum
 - Level2 ⭐⭐   : Files are different, but their features (file size, map rate, etc.) are similar (within threshold: 0.05)
 - Level1 ⭐     : Files are different, and their features are different (beyond threshold)
 - Level0        : File not found
 
Tonkaz supports ONLY RO-Crate generated by Sapporo-service and Yevis-cli.