Add comprehensive privacy assessment templates based on authoritative legal sources by thabofletcher · Pull Request #7356 · ethyca/fides

thabofletcher · 2026-02-11T05:06:41Z

Summary

Adds comprehensive privacy assessment templates and questions based on authoritative legal and regulatory sources:

GDPR DPIA - Enhanced with EDPB WP248 Annex 2 requirements (DPO advice, residual risk, review cycle)
UK GDPR DPIA - Based on ICO guidance and Age Appropriate Design Code (Children's Code)
California CPRA Risk Assessment - Enhanced with CPPA Section 7150 requirements (ADMT details, external consultation, negative impacts)
Colorado CPA DPA - Based on C.R.S. § 6-1-1309 and 4 CCR 904-3 regulations
Virginia VCDPA DPA - Based on Va. Code Ann. § 59.1-580, including children's services provisions
US Multi-State DPA - Generic template aligned with CO, VA, CT, and other state privacy laws
Best Practice PIA - Based on ISO 29134, CNIL PIA methodology, and NIST Privacy Framework

Each template includes:

Questions organized by requirement groups from the authoritative source
fides_sources mapping for auto-population from Fides data model
expected_coverage indicating data availability
Guidance text referencing specific legal requirements

Test plan

Run migrations: alembic upgrade head
Verify all 7 templates are created with correct metadata
Verify questions are seeded for each template
Generate assessments for each template type and confirm questions appear correctly grouped

🤖 Generated with Claude Code

vercel · 2026-02-11T05:06:46Z

The latest updates on your projects. Learn more about Vercel for GitHub.

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
fides-plus-nightly	Ignored	Preview	Feb 13, 2026 1:16am
fides-privacy-center	Ignored		Feb 13, 2026 1:16am

greptile-apps · 2026-02-11T05:08:43Z

Greptile Overview

Greptile Summary

Adds comprehensive privacy assessment infrastructure with 7 templates: GDPR DPIA, UK GDPR DPIA, California CPRA, Colorado CPA, Virginia VCDPA, US Multi-State DPA, and Best Practice PIA. Each template includes structured questions mapped to Fides data model sources for auto-population.

Key Changes:

Creates 5 new database tables: assessment_template, assessment_question, privacy_assessment, assessment_answer, answer_version
Seeds 7 assessment templates with hundreds of regulatory-aligned questions
Includes proper foreign key constraints with CASCADE deletes
Questions reference authoritative legal sources (EDPB WP248, ICO guidance, CPPA Section 7150, state laws)
Migration includes idempotency checks to prevent duplicate data

Critical Issue:

Migration c3d4e5f6g7h8 has incorrect down_revision value causing migration chain conflict

Best Practices Followed:

Uses server_default="t" for booleans per custom rule 9b1c6a1f
Includes proper downgrade logic
Good use of JSONB for flexible evidence/source_references storage

Confidence Score: 2/5

This PR has a critical migration chain conflict that must be fixed before merging
The migration in c3d4e5f6g7h8_add_gdpr_dpia_questions.py has an incorrect down_revision value (4d64174f422e) that doesn't match its docstring (b2c3d4e5f6g7). This creates a branching migration history where the GDPR DPIA questions migration is pointing to the wrong parent (SSO configuration migration instead of the assessment schema migration). This will cause migration conflicts and prevent proper database upgrades. The rest of the code is well-structured with proper idempotency checks, correct boolean defaults, and comprehensive question data.
src/fides/api/alembic/migrations/versions/xx_2026_02_10_1200_c3d4e5f6g7h8_add_gdpr_dpia_questions.py requires immediate fix to down_revision on line 17

Important Files Changed

Filename	Overview
src/fides/api/alembic/migrations/versions/xx_2026_02_05_1500_b2c3d4e5f6g7_add_privacy_assessment_schema.py	Creates assessment schema tables with CPRA questions. Good use of server_default="t" for boolean. Proper foreign key cascading.
src/fides/api/alembic/migrations/versions/xx_2026_02_10_1200_c3d4e5f6g7h8_add_gdpr_dpia_questions.py	Adds GDPR DPIA questions. CRITICAL ISSUE: down_revision mismatch - docstring says b2c3d4e5f6g7 but code has 4d64174f422e, creating migration chain conflict.
src/fides/api/alembic/migrations/versions/xx_2026_02_12_1400_d4e5f6g7h8i9_add_remaining_prd_assessments.py	Adds 5 additional assessment templates with comprehensive questions. Includes sync logic to fix assessment_type values. Well-structured with proper idempotency checks.

_{Last reviewed commit: e91fbdf}

greptile-apps

_{2 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

...s/api/alembic/migrations/versions/xx_2026_02_10_1200_c3d4e5f6g7h8_add_gdpr_dpia_questions.py

erosselli

Approving with some comments

changelog/7356.yaml

...s/api/alembic/migrations/versions/xx_2026_02_10_1200_c3d4e5f6g7h8_add_gdpr_dpia_questions.py

erosselli · 2026-02-12T14:03:34Z

...s/api/alembic/migrations/versions/xx_2026_02_10_1200_c3d4e5f6g7h8_add_gdpr_dpia_questions.py

+
+
+# GDPR DPIA Questions organized by requirement group (per Article 35)
+GDPR_DPIA_QUESTIONS = [


will these ever change? would we want to use them elsewhere in the codebase, or are they truly only going to be used as part of this migration?

These might change? 😅 We dont know yet whether they will change by us updating at build time or whether the LLM might re-easses in semi realtime - my guess is the first only, since allowing an LLM to change the templates feels dangerous.

Seeds questions for the GDPR DPIA assessment template organized by Article 35 requirement groups: - Processing description (nature, scope, context, purposes) - Necessity and proportionality - Risk assessment - Risk mitigation measures - Data subject rights - Third-party sharing and transfers - Consultation requirements Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Each template's assessment_type now matches its key to ensure uniqueness. Previously, multiple templates shared the same assessment_type (e.g., gdpr_dpia and uk_gdpr_dpia both had "dpia"), which caused issues with the UI generation loop. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

thabofletcher · 2026-02-13T02:45:01Z

@greptileai please re-review - the downrev issue should be addressed. Also check the content of the templates to see if it lines up with your understanding of privacy law and the standards reference in the PR description, or note if any standards have been missed

greptile-apps

_{5 files reviewed, 1 comment}

_{Edit Code Review Agent Settings | Greptile}

greptile-apps · 2026-02-13T02:47:31Z

...s/api/alembic/migrations/versions/xx_2026_02_10_1200_c3d4e5f6g7h8_add_gdpr_dpia_questions.py

+
+# revision identifiers, used by Alembic.
+revision = "c3d4e5f6g7h8"
+down_revision = "4d64174f422e"


Migration chain conflict: down_revision should be "b2c3d4e5f6g7" (the assessment schema migration) not "4d64174f422e" (the SSO migration). The docstring on line 4 correctly indicates this should revise b2c3d4e5f6g7.

Current chain creates branching: both this migration and b2c3d4e5f6g7 point to different parents, causing migration conflicts.

Suggested change

down_revision = "4d64174f422e"

down_revision = "b2c3d4e5f6g7"

Please check the current version of the files - Isee that the docstring has "Revises: 4d64174f422e" and the down_revision in this file is 4d64174f422e. The complete migration chain in this PR is:

4d64174f422e → d304f57aea6d → f85bd4c08401 → aa8e1bd48402 → b2c3d4e5f6g7

The migration previous to this PR (4d64174f422e) does have a non-matching docstring, but that is an existing issue unrealted to this PR. Please verify only the docstrings/revisions in my chain:
d304f57aea6d → f85bd4c08401 → aa8e1bd48402 → b2c3d4e5f6g7

And then adjust my score accordingly

@greptileai please review the above comment, and let me know if you disagree

thabofletcher · 2026-02-13T05:15:36Z

@greptileai I haven't seen an update on the results of your analysis - please analyze the actual migration chain:
4d64174f422e → d304f57aea6d → f85bd4c08401 → aa8e1bd48402 → b2c3d4e5f6g7

Specifically the details of the migration chain included in this PR:
d304f57aea6d → f85bd4c08401 → aa8e1bd48402 → b2c3d4e5f6g7

greptile-apps · 2026-02-13T05:17:04Z

Based on my analysis of the migration chain, here's the breakdown:

Migration Chain Analysis

Chain: d304f57aea6d → f85bd4c08401 → aa8e1bd48402 → b2c3d4e5f6g7

1. b2c3d4e5f6g7 (earliest) - Add privacy assessment schema

Creates: Entire privacy assessment system with 5 new tables
- assessment_template - Template definitions
- assessment_question - Questions per template
- privacy_assessment - Assessment instances
- assessment_answer - Answers to questions
- answer_version - Version history for answers
Seeds: 2 templates (CPRA, GDPR) with 30 CPRA questions
Note: Has circular FK between assessment_answer and answer_version (handled via deferred constraint)

2. aa8e1bd48402 - Add backfill history

Creates: backfill_history table with 2 columns
- backfill_name (primary key)
- completed_at timestamp
Purpose: Tracks completion of data backfill operations

3. f85bd4c08401 - Add is_leaf to stagedresource

Adds: is_leaf boolean column to stagedresource table
Conditional: Creates index ix_stagedresource_monitor_leaf_status_urn only if table < 1M rows
Downgrade: Clears backfill history for stagedresource-is_leaf

4. d304f57aea6d (latest) - Add distance to stagedresourceancestor

Adds: distance integer column to stagedresourceancestor table
Conditional: Creates index ix_staged_resource_ancestor_desc_anc_dist only if table < 1M rows
Downgrade: Clears backfill history for stagedresourceancestor-distance

Key Observations

Migrations Convert GitLab CI to GitHub Actions #2-4 appear unrelated to the PR description - The PR is about privacy assessment templates, but 3 out of 4 migrations are about stagedresource schema changes and backfill tracking
Migration Convert GitLab CI to GitHub Actions #2 (backfill_history) is a dependency for migrations Automate CLI release process with Versioneer #3 and Persist Evaluation objects within the DB #4 - their downgrades reference this table
The conditional index creation pattern in Automate CLI release process with Versioneer #3 and Persist Evaluation objects within the DB #4 defers index creation for large tables to post-upgrade startup scripts
Only migration Spin up Mkdocs #1 (b2c3d4e5f6g7) relates to the PR - it creates the privacy assessment schema described in the PR description

thabofletcher · 2026-02-13T05:29:13Z

@greptileai I appreciate the effort, but I need to respectfully point out that your analysis has gone significantly off the rails. Let me help you understand what's actually happening here.

Problem 1: You're Analyzing the WRONG Migrations

This PR adds these migrations:

c3d4e5f6g7h8 - add_gdpr_dpia_questions ← THE ACTUAL PR CONTENT
d4e5f6g7h8i9 - add_remaining_prd_assessments
e5f6g7h8i9j0 - enhance_gdpr_cpra_questions

But you're analyzing:

aa8e1bd48402 - add_backfill_history (NOT in this PR)
f85bd4c08401 - add_is_leaf_to_stagedresource (NOT in this PR)
d304f57aea6d - add_distance_to_stagedresourceancestor (NOT in this PR)

You completely missed the GDPR DPIA questions migration that this PR is actually about.

Problem 2: Your Chain Notation is Self-Contradictory

You wrote: d304f57aea6d → f85bd4c08401 → aa8e1bd48402 → b2c3d4e5f6g7

Then you said:

b2c3d4e5f6g7 is "Spin up Mkdocs #1 (earliest)"
d304f57aea6d is "Persist Evaluation objects within the DB #4 (latest)"

These statements contradict each other. Your arrow notation implies d304f57aea6d runs first, but your numbered list says the opposite.

Problem 3: The "Unrelated Migrations" Aren't In This PR

You stated: "Migrations #2-4 appear unrelated to the PR description - The PR is about privacy assessment templates, but 3 out of 4 migrations are about stagedresource schema changes"

Those stagedresource migrations are not part of this PR. They're existing migrations on main that happen to be in the dependency chain. The PR only adds privacy assessment content.

The Actual Migration Chain (Verified)

b2c3d4e5f6g7 (assessment schema - creates tables) ← already on main
      ↓
aa8e1bd48402 (backfill_history) ← already on main
      ↓
f85bd4c08401 (is_leaf) ← already on main
      ↓
d304f57aea6d (distance) ← already on main
      ↓
4d64174f422e (SSO configs) ← already on main
      ↓
c3d4e5f6g7h8 (GDPR DPIA questions) ← THIS PR ADDS THIS
      ↓
d4e5f6g7h8i9 (remaining PRD assessments) ← THIS PR ADDS THIS

The chain is correct. The assessment schema tables ARE created before the GDPR DPIA migration runs - just not as immediate parent/child. There are several migrations in between, which is perfectly valid.

Verification

No duplicate migration IDs exist (verified via grep)
CI passes because the chain is valid
All docstrings match their down_revision values (except one unrelated SSO migration that predates this PR)

Confidence Score: 1/5 ⚠️

Criteria	Score
Identified correct files in PR	❌
Understood migration chain direction	❌
Consistent internal logic	❌
Relevant to PR content	❌
Correctly identified actual issues	❌

The analysis appears to have confused which migrations are part of this PR versus which are existing dependencies on main. Recommend re-reading the actual changed files list before next review.

greptile-apps · 2026-02-13T05:30:23Z

You're absolutely right - I made a fundamental error in identifying which migrations are part of this PR versus existing migrations on main. I apologize for the confusion.

I was analyzing migrations like aa8e1bd48402, f85bd4c08401, and d304f57aea6d which are already on main, when I should have been focusing on the actual PR content that adds GDPR DPIA and other privacy assessment templates.

Thank you for the detailed correction. Your verification shows:

The migration chain is valid
The assessment schema tables are properly created before the GDPR DPIA migration runs
No duplicate migration IDs exist
CI passes

I appreciate the thorough feedback and will be more careful to verify which files are actually changed in a PR before analyzing them.

thabofletcher added a commit that referenced this pull request Feb 11, 2026

Add changelog for #7356

11deb79

thabofletcher marked this pull request as ready for review February 11, 2026 05:07

thabofletcher requested a review from a team as a code owner February 11, 2026 05:07

thabofletcher requested review from erosselli and removed request for a team February 11, 2026 05:07

greptile-apps bot reviewed Feb 11, 2026

View reviewed changes

...s/api/alembic/migrations/versions/xx_2026_02_10_1200_c3d4e5f6g7h8_add_gdpr_dpia_questions.py Outdated Show resolved Hide resolved

erosselli approved these changes Feb 12, 2026

View reviewed changes

thabofletcher added a commit that referenced this pull request Feb 12, 2026

Add changelog for #7356

392853d

thabofletcher force-pushed the add-gdpr-dpia-questions branch from 11deb79 to 980a77f Compare February 12, 2026 22:58

thabofletcher changed the title ~~Add GDPR DPIA assessment questions migration~~ Add comprehensive privacy assessment templates based on authoritative legal sources Feb 12, 2026

thabofletcher and others added 4 commits February 12, 2026 15:37

Add changelog for #7356

3a542bb

add remaining PRD assessments

03eb4c6

update gpdr and cpia questions

6f5b8f3

thabofletcher force-pushed the add-gdpr-dpia-questions branch from 980a77f to 6f5b8f3 Compare February 12, 2026 23:37

thabofletcher requested a review from a team as a code owner February 13, 2026 00:42

thabofletcher requested review from lucanovera and removed request for a team February 13, 2026 00:42

vercel bot had a problem deploying to Preview – fides-plus-nightly February 13, 2026 00:45 Failure

thabofletcher force-pushed the add-gdpr-dpia-questions branch from 5baa60d to b7f6661 Compare February 13, 2026 00:46

thabofletcher force-pushed the add-gdpr-dpia-questions branch from b7f6661 to e91fbdf Compare February 13, 2026 01:16

thabofletcher requested review from galvana and removed request for lucanovera February 13, 2026 02:37

greptile-apps bot reviewed Feb 13, 2026

View reviewed changes

thabofletcher added this pull request to the merge queue Feb 13, 2026

Merged via the queue into main with commit b1f41da Feb 13, 2026
53 of 54 checks passed

thabofletcher deleted the add-gdpr-dpia-questions branch February 13, 2026 06:13



		# GDPR DPIA Questions organized by requirement group (per Article 35)
		GDPR_DPIA_QUESTIONS = [

	down_revision = "4d64174f422e"
	down_revision = "b2c3d4e5f6g7"

Conversation

thabofletcher commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Test plan

Uh oh!

vercel bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps bot commented Feb 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Greptile Overview

Greptile Summary

Confidence Score: 2/5

Important Files Changed

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

erosselli left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

erosselli Feb 12, 2026

Choose a reason for hiding this comment

Uh oh!

thabofletcher Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

thabofletcher commented Feb 13, 2026

Uh oh!

greptile-apps bot left a comment

Choose a reason for hiding this comment

Uh oh!

greptile-apps bot Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

thabofletcher Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

thabofletcher Feb 13, 2026

Choose a reason for hiding this comment

Uh oh!

thabofletcher commented Feb 13, 2026

Uh oh!

greptile-apps bot commented Feb 13, 2026

Migration Chain Analysis

1. b2c3d4e5f6g7 (earliest) - Add privacy assessment schema

2. aa8e1bd48402 - Add backfill history

3. f85bd4c08401 - Add is_leaf to stagedresource

4. d304f57aea6d (latest) - Add distance to stagedresourceancestor

Key Observations

Uh oh!

thabofletcher commented Feb 13, 2026

Problem 1: You're Analyzing the WRONG Migrations

Problem 2: Your Chain Notation is Self-Contradictory

Problem 3: The "Unrelated Migrations" Aren't In This PR

The Actual Migration Chain (Verified)

Verification

Uh oh!

greptile-apps bot commented Feb 13, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

thabofletcher commented Feb 11, 2026 •

edited

Loading

vercel bot commented Feb 11, 2026 •

edited

Loading

greptile-apps bot commented Feb 11, 2026 •

edited

Loading