Building Spark Lineage For Data Lakes

Summary

Spark lineage has been a blindspot for the data engineering industry so we set off to engineer a solution. Here's how we did it.