Add log redaction tutorial sample code #9066
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
465b9f6 to
8e286e6
Compare
19fa609 to
9067af7
Compare
arbrown
approved these changes
Jan 30, 2023
bourgeoisor
reviewed
Jan 30, 2023
dandhlee
requested changes
Jan 31, 2023
Create logging/redaction folder to store assets for log redaction tutorial. Assets will include: * documentation (README.md) * boilerplate code to start the tutorial * final version of the code
fix package imports; add requirements.txt file to allow import google-cloud-logging that is not part of Beam SDK; fix logging_v2 client initialization; refactor some class, function and variable names;
Add a reference example for custom image Dockerfile
Add "final" version of the code for tutorial. Format boilerplate code to keep same quote style and to add placeholder comments for code addition.
Execute redaction function before sending logs in the pipeline. Refactoring argument names.
6e0d3b9 to
fb8bfc3
Compare
dandhlee
approved these changes
Feb 1, 2023
telpirion
pushed a commit
that referenced
this pull request
Mar 13, 2023
* doc(logging): add log redaction boilerplate code Create logging/redaction folder to store assets for log redaction tutorial. Assets will include: * documentation (README.md) * boilerplate code to start the tutorial * final version of the code * chore(fix): refactor dataflow fix package imports; add requirements.txt file to allow import google-cloud-logging that is not part of Beam SDK; fix logging_v2 client initialization; refactor some class, function and variable names; * chore(doc): add custom image Dockerfile Add a reference example for custom image Dockerfile * chore(fix): refactor code sample for tutorial Add "final" version of the code for tutorial. Format boilerplate code to keep same quote style and to add placeholder comments for code addition. * chore(fix): adding redaction step to pipeline Execute redaction function before sending logs in the pipeline. Refactoring argument names. * chore: fix compilation error * chore: comment text correction * chore: update codeowners * chore: change indent size from 2 to 4 * chore: fix lint errors * chore: minor text fix * chore: updating DEE observability ownership * chore: address style comments from dandhlee@
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Code to support an interactive tutorial that demonstrates log redaction using DLP at ingestion time.
Note: Before submitting a pull request, please open an issue for discussion if you are not associated with Google.
Checklist
nox -s py-3.9(see Test Environment Setup)nox -s lint(see Test Environment Setup)