Datakin

When we started Datakin, we had an ambitious goal: to bring data lineage to the modern data stack. This goal has not changed; our new friends at Astronomer share it with us!

“Our job duration tab enables a more detailed inspection of how a given job fits into the overall pipeline. You can evaluate the execution times of all the upstream jobs in the most recent run cycle.” datak.in/3n8VocI

Without a real time #datalineage graph “not only is it hard to identify duration issues, it’s costly and time-consuming to diagnose their cause.” datak.in/30kpNfj

Collecting #datalineage from @ApacheSpark is easy, and can help you keep your pipeline running smoothly. Read more on the @openlineage blog: datak.in/3wHiV7H

Sometimes a long-running data pipeline job is more than just an inconvenience, it is the sign of a more complex problem. In this post, Peter Hicks shows how to identify data pipeline bottlenecks with #datalineage and Datakin: datak.in/3D6PVbU

The team at Northwestern Mutual has begun to capture and trace the lineage of key datasets, creating a real-time map of their pipelines. Over on the OpenLineage blog, Kevin Mellott explains how real-time #datalineage helps them stay ahead. datak.in/3Cip0JY

"If you use Datakin to observe @getdbt models as they run, you can always know exactly where your datasets came from and how they were created." datak.in/3vgLZlQ

Using @OpenLineage, the team at Northwestern Mutual has begun to capture and trace the #datalineage of key datasets, creating a real-time map of their pipelines. Register now for the session on Oct 21 at datak.in/3lJZBTz and learn how it helps them stay ahead of the game.

In this tutorial, we show how to use Datakin to observe @getdbt jobs. Learn how to trace #datalineage, observe changes across the pipeline, and troubleshoot performance bottlenecks: datak.in/3BL0XTB

"Increasingly, we will find data science at the edges, decentralized and largely ungoverned. We think that data lineage is the key. It can establish a 'chain of custody' for key datasets, contextualize fragmented work, and build organzational trust." datak.in/3Bfigfg

#DataLineage tip: “Be sure to use the `{{ ref() }}` and `{{ source() }}` jinja functions when referring to data sources if you want to accurately capture @getdbt data lineage with @MarquezProject.” datak.in/3FkfbNe