pmdtester
- home
- code
- bugs
- build-status
-
- gem-version
-
DESCRIPTION:
A regression testing tool ensure that new problems and unexpected behaviors will not be introduced to PMD project after fixing an issue and new rules work as expected.
FEATURES/PROBLEMS:
The diff report can be generated according to the base and patch branch of PMD on a list of standard projects (e.g. Spring Framework, Checkstyle, OpenJDK, etc.).
Rule violations and code duplications are compared to report, which are new, removed or changed.
While executing PMD, JDK Flight Recorder (jfr) is enabled and a recording is created. This allows to investigate performance and memory issues afterwards.
SYNOPSIS:
Options:
-r, --local-git-repo path to the local PMD repository
-b, --base-branch name of the base branch in local PMD repository
-p, --patch-branch name of the patch branch in local PMD repository
-bc, --base-config path to the base PMD configuration file
-pc, --patch-config path to the patch PMD configuration file
-c, --config path to the base and patch PMD configuration file
-l, --list-of-project path to the file which contains the list of standard projects
-m, --mode the mode of the tool: 'local', 'online' or 'single'
single: Set this option to 'single' if your patch branch contains changes
for any option that can't work on main/base branch
online: Set this option to 'online' if you want to download
the PMD report of main/base branch rather than generating it locally
local: Default option is 'local', PMD reports for the base and patch branches are generated locally.
-t, --threads Sets the number of threads used by PMD. Set threads to 0 to disable multi-threading processing.
-f, --html-flag whether to not generate the html diff report in single mode
-a, --auto-gen-config whether to generate configurations automatically based on branch differences,this option only works in online and local mode
--filter-with-patch-config whether to use patch config to filter baseline result as if --auto-gen-config has been used. This option only works in online mode.
--keep-reports whether to keep old reports and skip running PMD again if possible
-d, --debug whether change log level to DEBUG to see more information
--error-recovery enable error recovery mode when executing PMD. Might help to analyze errors.
--baseline-download-url download url prefix from where to download the baseline in online mode
--no-cpd do not execute CPD (Copy Paste Detector) and compare duplications; only execute PMD
--no-pmd do not execute PMD and compare rule violations; only execute CPD (Copy Paste Detector)
-v, --version
-h, --help
Quick start
Run local mode
pmdtester -b main -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -a
Run single mode
pmdtester -p YOUR_DEVELOPMENT_BRANCH -pc CONFIG_ONLY_CONTAINS_NEW_PMD_JAVA_RULE -m single
Run online mode
pmdtester -b main -p YOUR_DEVELOPMENT_BRANCH -r PATH_TO_LOCAL_PMD_REPO -m online -a
Output
The tool creates the following folders:
target
├── repositories <- the analyzed projects are cloned here
│ ├── PROJECT_NAME_1
│ ├── ......
│ └── PROJECT_NAME_n
├── reports
│ ├── BASE_BRANCH_NAME <- the base baseline is placed here
│ ├── PATCH_BRANCH_NAME <- the patch baseline is placed here
│ └── diff
│ ├── index.html <- the summary report of diff reports
| ├── summary.txt <- short summary in text form to be used as a PR comment
| ├── conclusion.txt <- GitHub action check conclusion (e.g. 'neutral' or 'success')
│ ├── base_config.xml <- pmd config from the base branch
│ ├── patch_config.xml <- pmd config from the patch branch
│ ├── css <- css resources are placed here
│ ├── js <- js resources
│ ├── PROJECT_NAME_1
│ │ ├── diff_pmd_data.js <- contains the violations as js/json
| | ├── diff_cpd_data.js <- contains the duplications as js/json
│ │ ├── index.html <- the diff report of PROJECT_1
| | ├── base_pmd_report.html <- full pmd report of the baseline
| | ├── base_cpd_report.html <- full cpd report of the baseline
| | ├── patch_pmd_report.html <- full pmd report of the patch branch
| | └── patch_cpd_report.html <- full cpd report of the baseline
│ ├── .......
│ └── PROJECT_NAME_n
│ ├── diff_pmd_data.js <- contains the violations as js/json
│ ├── diff_cpd_data.js <- contains the duplications as js/json
│ ├── index.html <- the diff report of PROJECT_N
│ └── ...
├── pmd-bin-<version>-<branch_name>-<sha1> <- cached pmd builds that are reused
└── pmd-bin-....
The baseline format
branch_name
├── branch_info.json
├── config.xml
├── project-list.xml
├── STANDARD_PROJECT_NAME_1
│ ├── pmd_report_info.json
│ ├── pmd_report.xml
│ ├── cpd_report_info.json
│ └── cpd_report.xml
├── ......................
└── STANDARD_PROJECT_NAME_n
├── pmd_report_info.info
├── pmd_report.xml
├── cpd_report_info.info
└── cpd_report.xml
REQUIREMENTS:
-
Ruby 4 or higher
Runtime dependency
nokogiri ~> 1.19
slop ~> 4.10
rufus-scheduler ~> 3.9
logger-colors ~> 1.1
liquid ~> 5.11
base64 ~> 0.3
bigdecimal ~> 4.0
logger ~> 1.7
Development dependency
hoe ~> 4.6
hoe-bundler ~> 1.5
hoe-git ~> 1.6
minitest ~> 6.0
mocha ~> 3.0
rubocop ~> 1.84
test-unit ~> 3.7
rdoc ~> 7.2
rake ~> 13.3
debug ~> 1.11
INSTALL:
gem install pmdtester --pre
DEVELOPERS:
git clone https://github.com/pmd/pmd-regression-tester.git
cd pmd-regression-tester
gem install bundler
bundle config set path "vendor/cache"
bundle install # once
bundle exec rake verify # run this command before commit your changes
bundle exec pmdtester ... # run this to directly execute pmdtester from source
Run all unit tests:
bundle exec rake clean test
Run all integration tests:
bundle exec rake clean integration-test
Run a single test class, e.g.:
bundle exec ruby -I test test/test_project_diff_report.rb
bundle exec ruby -I test test/integration_test_runner.rb
Run a single test, e.g.:
bundle exec ruby -I test test/test_project_diff_report.rb -n test_diff_report_builder
Debugging
This assumes using vscode with the extension Ruby LSP.
-
Execute a single test with debugger:
bundle exec rdbg --open --command -- ruby -Itest test/test_word_differ.rb -n test_simple_difference -
The debugger will wait, until a debugging session is attached. You can use the “Attach debugger” run configuration in vscode.
-
In case, Ruby LSP doesn’t find an locally running debugger (“No debuggee processes found”), you might need to explicitly set the debugger socket
# find local debugger sockets bundle exec rdbg --util=list-socks -
Then change “Attach debugger to socket” config to specify a “debugSocketPath” explicitly and run this:
{ "type": "ruby_lsp", "name": "Attach debugger to socket", "request": "attach", "debugSocketPath": "/run/user/1000/rdbg-382154" }
Note: The socket changes for every debugging session.
Releasing
-
Update
History.md(version and date) -
Update
lib/pmdtester.rb(version) -
Run “bundle exec rake verify” and add new
pmdtester.gemspec(new version) -
Commit (“Prepare release x.y.z”).
-
Tag this commit (“git tag vx.y.z”).
-
Update History.md and lib/pmdtester.rb for the next development version, run again “bundle exec rake verify”
-
Commit (“Prepare next development version x.y.z-SNAPSHOT”).
-
Push to main.
-
Push the tag. Github Actions will build and publish the new gem
-
A github release is automatically created, verify it on github.com/pmd/pmd-regression-tester/releases
-
To make pmd’s main CI use the new version (in pmd/pmd), go to the directory “.ci/files” and run ‘bundle lock –update`. Commit these changes.
-
Rename milestone to version, close it and create a new “Next” milestone: github.com/pmd/pmd-regression-tester/milestones