Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ruby: add XML tables to dbscheme #10863

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Conversation

aibaars
Copy link
Contributor

@aibaars aibaars commented Oct 17, 2022

No description provided.

@aibaars aibaars requested a review from a team as a code owner October 17, 2022 17:31
@github-actions github-actions bot added the Ruby label Oct 17, 2022
Copy link

@github-advanced-security github-advanced-security bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CodeQL found more than 10 potential problems in the proposed changes. Check the Files changed tab for more details.

XmlParent getParent() { xmlChars(this, _, result, _, _, _) }

/** Holds if this character sequence is CDATA. */
predicate isCDATA() { xmlChars(this, _, _, _, 1, _) }

Check warning

Code scanning / CodeQL

Acronyms should be PascalCase/camelCase.

Acronyms in isCDATA should be PascalCase/camelCase.
Comment on lines 1 to 2
description: <INSERT DESCRIPTION HERE>
compatibility: full|backwards|partial|breaking
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You forgot to change this.

Copy link
Contributor

@calumgrant calumgrant Oct 18, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did the PR checks pass?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah indeed. I changed it, but I regenerated the files because I had forgotten about the numlines table.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why did the PR checks pass?

CI doesn't check the description or compatibility fields. It only checks the dbscheme and whether the upgrade queries can be compiled (there are none in this case). The "upgrade" is simply adding empty tables which can be achieved by doing nothing at all ;-)

Comment on lines +1 to +2
/**
* Provides classes and predicates for working with XML files and their content.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like another candidate for a shared library pack, but that can be done separately.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I actually copied the XML.qll files that the other languages share via identical-files.json . Unfortunately, the shared file contains an invalid (for us) import semmle.files.FileSystem statement. In addition the shared file has very many deprecated, renamed classes. So I cleaned it up.

Ideally we make this a proper shared library pack based on @erik-krogh's work on a shared Locations/Files library.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ideally we make this a proper shared library pack based on @erik-krogh's work on a shared Locations/Files library.

I think it will take a while before I got the CLI features I need to make that library. So don't wait on that.

--language ruby --source-root "${{ github.workspace }}/repo" \
"${{ runner.temp }}/database"
codeql database trace-command --threads 4 --index-traceless-dbs "${{ runner.temp }}/database"
codeql database index-files -l xml --include "**/*.xml.erb" "${{ runner.temp }}/database"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am a bit surprised that our xml parser handles xml.erb files in the general case. Do you think there could be spurious syntax errors, or will it be guaranteed to work?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I don't think it would work in the general case, but it might be alright for all the .xml.erb files in the five projects above.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There will be syntax errors especially on <% token, but it mostly works. Note that this workflow is just for generating stats. Real XML files are pretty rare in Ruby projects, so this was the closest thing I could find.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good point. I don't think it would work in the general case, but it might be alright for all the .xml.erb files in the five projects above.

There are plenty of XML syntax errors in the xml.erb files in the 5 projects we currently use. However, these errors are non-fatal. The purpose is just to gather some stats about XML-like data. For that purpose it works fine.

Copy link
Contributor

@nickrolfe nickrolfe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

Copy link
Contributor

@calumgrant calumgrant left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Feel free to hold this back from the GA if you think it's too risky.

Should we also update the autobuilder to extract XML files by default (and common extensions), or will this always be a manual build step?

@aibaars
Copy link
Contributor Author

aibaars commented Oct 18, 2022

Feel free to hold this back from the GA if you think it's too risky.

Should we also update the autobuilder to extract XML files by default (and common extensions), or will this always be a manual build step?

We could, but I don't think it is very useful. Let's wait for a use-case first. It took me a while to find some repo with a non-trivial amount of xml-like data to gather stats, so I suspect that XML data is not very popular in Ruby code bases..

@aibaars aibaars removed the Ruby GA label Oct 18, 2022
@aibaars aibaars marked this pull request as draft October 18, 2022 10:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants