Lineage is metadata, which helps to find root cause if odd results appear.
Lineage are things like:

• where the data came from?
• what processes it has been through?
• and what condition is in?
• what kinds of uses a data is suited for?
• what is the condition of the data?

issue	Solution
Latency, throughput	Dataflow to Bigtable
Reusing Spark pipelines	Cloud Dataproc
need for visual pipeline building	Cloud Data Fusion