How To: Debugging Discrepancies in Alli Data Library

When to Use This

The Alli Data Library consolidates data from several sources. If you notice that the data in the Data Library does not match the corresponding metrics in a platform UI, follow the steps below to quickly identify and resolve the discrepancy.

Troubleshooting

1. Verify and Focus the SQL

Double check that the same filters used in the platform UI are also applied in your Data Library SQL query.
Simplify your SQL to the smallest query that still produces the discrepancy. For example:
- Focus on a specific date range or the set of dates where the data is incorrect.
- Narrow down to one or a few specific accounts, campaigns, or ad groups where the issue appears.
- Focus on the specific metrics or breakdowns that are incorrect.
This process often reveals the source of the discrepancy. For example, if the issue is limited to campaigns under the same account, it might be that this account was accidentally excluded from the underlying datasources.
- In such cases, file an ASD ticket to have that account (or any new account) added to the Data Library: All Service Desk → Alli Data Request.

2. Verify the Latest Data has Loaded

The Data Library is a set of views that consolidate different datasources. If the data in the underlying sources is incorrect, it will also be incorrect in the Data Library.
For example, if you see a discrepancy in Verizon data in the {client}_core.display_campaign view, start your investigation at the {client}.verizon_insights_ad datasource (see green line below).

If the raw datasource matches the platform UI but does NOT match display_campaign:
- The issue is in the data processing that creates display_campaign.
- Resolution: Notify the Data Engineering team to correct the bug in the Data Library.

If the raw datasource does NOT match the platform UI but matches display_campaign:
- The Data Library is likely correct.
- Resolution 1: Backfill the raw datasource to reload the latest data, then check against the platform UI.
- Resolution 2 (less common): Update the platform connector within Alli Data, backfill the raw datasource, and check the latest data against the platform UI.

For more information on data lineage, see the Alli Data Library space. You can also reach out in #alli-data-library.

3. Verify the Data Library App Executed Successfully After the Latest Data Load

The Data Library is mostly view-based, so it reflects the underlying tables at all times. However, some datasets (such as product feed data models) require a Data Library refresh before the latest data is reflected.
The Data Library refreshes at least once per day, typically in the early morning after most data loads.
Check the status of the latest Data Library app execution in the client’s Alli Marketplace space. The app follows this naming convention:
- ADL 2.0 - Refresh - {client name} - {target database} - {platform} - {schedule}
- Example for “Acme” client:
  - Redshift: ADL 2.0 - Refresh - Acme - Redshift - All Platforms - Daily
  - BigQuery: ADL 2.0 - Refresh - Acme - BigQuery - All Platforms - Daily

If the data in question depends on a new execution of the Data Library:
- Check that the Data Library app ran today.
- If the latest execution failed, the Data Engineering team has already been alerted.
- If the data requires a Data Library refresh, you may trigger an execution yourself. This is sometimes necessary if raw data loads into the underlying sources too late for the scheduled execution to include them.
- If you are unsure whether to refresh or if the DE team has already been notified, check the app status and consult with the Data Engineering team if needed.

Example:

Search for ADL 2.0 App and select link

Confirm date and status (green check mark) for last scheduled run.

4. Reach Out to the Data Engineering Team

If you have:

Vetted the SQL
Confirmed that data in the raw datasource tables matches the platform UI
Verified that the latest Data Library app execution post-load was successful
And Data Library still shows a discrepancy

Then the Data Engineering team can investigate further.

Reach out to the Data Engineering team via:
- The #alli-data-library Slack channel
- Ping @support-alli-data-library and submit an ASD ticket with:
  - The SQL query used to retrieve incorrect results from Data Library (as editable text)
  - The correct target metrics from the platform (as a screenshot or link to a dashboard, tracker, or platform page)
  - Log the ASD ticket as Report a Bug and select Feature: Data Library

If you do not provide the query or validation of what the data should be, the team will request it, which may delay resolution.