Skip to content

Write DAG to compute base Commons Impact Metrics Iceberg tables

Milimetric requested to merge commons-impact-metrics-jobs into main

Generates the 5 main Iceberg datasets of the Commons Impact Metrics data product monthly.

* wmf_contributors.commons_category_metrics_snapshot
* wmf_contributors.commons_media_file_metrics_snapshot
* wmf_contributors.commons_pageviews_by_category
* wmf_contributors.commons_pageviews_by_media_file
* wmf_contributors.commons_edits

Uses the following data sources:

* wmf_raw.mediawiki_page
* wmf_raw.mediawiki_image
* wmf_raw.mediawiki_imagelinks
* wmf_raw.mediawiki_categorylinks
* wmf_raw.mediawiki_revision
* wmf_raw.mediawiki_private_actor
* wmf.pageview_hourly
* canonical_data.wikis

Only categories and media files that belong to category trees present in the Commons category allow-list will be reported on.

Bug: T358699

Edited by Xcollazo

Merge request reports