Updating our Fork's dev branch with the latest. #2
Merged
sgoggins merged 17 commits intoComputationalMystics:devfrom Sep 21, 2018
Merged
Updating our Fork's dev branch with the latest. #2sgoggins merged 17 commits intoComputationalMystics:devfrom
sgoggins merged 17 commits intoComputationalMystics:devfrom
Conversation
When someone does git rebase on the remote repo, this can cause issues including an error stating you have unmerged local changes. Since we're just pulling the repos and not doing any actual development in what Facade analyzes, it's extremely unlikely that there would be actual new unmerged changes. The solution, which would be drastic in other circumstances but fine here, is just to hard reset the local repo to the new HEAD and delete any extraneous files using git clean. Signed-off-by: Brian Warner <brian@bdwarner.com>
When a user uses the Configure page to update settings, we sanitize the input. However, it was a bit too sanitary, and long strings were being overly truncated. This is fixed now. Signed-off-by: Brian Warner <brian@bdwarner.com>
"Patches landed by", not "Patched landed by". Durrrr. Signed-off-by: Brian Warner <brian@bdwarner.com>
Every once in a while I do something mind-blisteringly dumb. In this case, I was reading a db config file using a relative path, rather than determining the absolute path based upon the script location. When running facade-worker.py manually this was fine. It interpreted the relative path correctly. However, for cron jobs, the working directory is the user's home directory. By literally nothing more than unfortunate coincidence this wasn't failing on my machine because I had an old db.cfg sitting in my home directory, and Facade was picking up that. Now things should work just fine both for manual runs and for cronjobs. Signed-off-by: Brian Warner <brian@bdwarner.com>
Release policy updated to be a little less draconian in requiring major number increases. Adding new tables to the Facade database should never harm backwards compatibility, so these types of changes can be minor releases. Signed-off-by: Brian Warner <brian@bdwarner.com>
When a user changes a Facade setting, the old settings are kept in the settings table for auditability. When Facade needs a setting, it grabs the one with the most recent timestamp. However, when importing on a very fast server, it's possible the settings could be added so quickly that the timestamps are indistinguishable. To deal with this, we're now exporting settings in order of their ID, which ensures they're read in order as well. When importing, there's now a very short delay between lines, ensuring that there's a meaningful difference between the timestamps. Signed-off-by: Brian Warner <brian@bdwarner.com>
Unbeknownst to me, when you have a NULL value in a MySQL constraint, you're allowed to have multiple matches. What this means is that if you have a UNIQUE constraint on a table where three columns are mandatory and the fourth is not, it'll keep merrily adding duplicate rows where the mandatory ones are the same and the fourth is empty. This isn't the behavior we wanted, as it allows duplicate tags. The solution was the remove the old constraint and add a new one that excludes the column that allows NULL values. Yes, it's arguably less strict, but it has the major advantage of actually working, so it has that going for it. Signed-off-by: Brian Warner <brian@bdwarner.com>
This is the initial commit of the Facade CLI. It's a 100% Python3 user interface which can be used instead of the PHP-based web interface. It is mostly at feature parity with the web UI. The main things missing are: * Project-specific stats (you'll have to export an Excel file) * Bulk repo imports from GitHub, cgit, and Gerrit * Filtered exports Otherwise, the other major features are there. In addition, it's possible to import cli/facade.py using other Python programs, and it'll provide some useful basic management functions. These are documented at the head of the file. Future tasks include: * Adding the ability to do reports * Adding the ability to do bulk repo imports * Adding export filters * Adding a setup option for CLI-only that doesn't require Apache This will be a work in progress, but the goal is to maintain it in parallel to the web interface. Signed-off-by: Brian Warner <brian@bdwarner.com>
Passed the database cursor an int. It wanted a tuple. Who knew? The debugger, apparently. Fixed now. Signed-off-by: Brian Warner <brian@bdwarner.com>
Ubuntu 16.04 LTS does not ship a Python3 package we need for the cli tool (python3-texttable) so now the minimum requirement is 16.10. Signed-off-by: Brian Warner <brian@bdwarner.com>
When you delete large volumes of data from mysql (for example, you removed a repo or project and analysis data is removed from analysis_data and the caches) it doesn't reclaim the disk space. For most use cases this is probably not a big deal, but for long-lived installations it could cause performance and bloating issues. Now when a project or repo is deleted, we optimize the tables. This will increase the run time during deletions, but should improve performance long term. Signed-off-by: Brian Warner <brian@bdwarner.com>
When deleting a repo, the new weekly caches weren't being cleared. They're being cleared now. When deleting a project, the unknown_cache table wasn't being cleared because the project was being pruned before it could be recached. As such, facade-worker.py had no way of knowing that it should recache the deleted project. Now, we just clear the cache during delete. Signed-off-by: Brian Warner <brian@bdwarner.com>
When exporting, it turns out that it's pretty easy to get too many lines in the Excel file, which maxes out at a little over a million. This patch adds some useful features, such as: * Filtering results by date * Filtering results by affiliation * Filtering results by project * And when all else fails, telling the user the Excel file won't actually work Signed-off-by: Brian Warner <brian@bdwarner.com>
The cli requires texttable, and now the setup scripts will install it. Signed-off-by: Brian Warner <brian@bdwarner.com>
Now that we have a CLI, the web UI is optional. Well, optional in the sense that you don't strictly need it, but truth be told it is still a lot nicer to use. But, maybe you don't want to run a web server on your machine, or maybe you want to do the analysis on one system and display it using another. This should all be possible now. Signed-off-by: Brian Warner <brian@bdwarner.com>
When originally written, a special tag without an explicit end date was given a NULL end_date in the database. This meant that we had to check for an explicit end_date and if it wasn't expected, had to do some contortions to change the SQL queries around. It is much more straightforward to choose an end date that is just impossibly far into the future. This makes math much easier when matching tags to data points. Signed-off-by: Brian Warner <brian@bdwarner.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.