We use SonarCloud integration with Bitbucket and have a build step to stop PRs from getting merged if the coverage on “new code” falls below a certain threshold.
However, we are facing weird behaviours on how sonar finds a diff against the destination branch. SC is finding really old files as having changed (Though it clearly hasn’t and does not show as a difference in bitbucket). We have no workaround for this at the moment.
We are not doing a shallow clone of the repository either as suggested in a couple of other threads.
Can you provide your job log - redacted as necessary - starting from your checkout and ending with the end of the analysis?
Hey Ann,
Apologies as I may have not been clear. We cannot share the entire log redacted as it is quite a large log and we cannot go line by line to redact it to be posted publicly on a public forum.
Hence, I’m asking if there is a way we can share the log file with Sonar directly.
Since this is about detection of new code, how about just the portion related to checkout at the beginning of the job, and the part at the end of analysis around processing SCM data?
&1|15:46:35.428 INFO: Scanner configuration file: /sonar-scanner-4.4.0.2170-linux/conf/sonar-scanner.properties
&1|15:46:35.429 INFO: Project root configuration file: /xxprojxx/sonar-project.properties
&1|15:46:35.475 15:46:35.473 INFO: SonarScanner 4.4.0.2170
&1|15:46:35.476 15:46:35.475 INFO: Java 11.0.3 AdoptOpenJDK (64-bit)
&1|15:46:35.476 15:46:35.476 INFO: Linux 3.10.0-1160.66.1.el7.x86_64 amd64
&1|15:46:35.725 15:46:35.725 INFO: User cache: /root/.sonar/cache
&1|15:46:37.715 15:46:37.714 INFO: Scanner configuration file: /sonar-scanner-4.4.0.2170-linux/conf/sonar-scanner.properties
&1|15:46:37.715 15:46:37.715 INFO: Project root configuration file: /xxprojxx/sonar-project.properties
&1|15:46:37.718 15:46:37.716 INFO: Analyzing on SonarCloud
&1|15:46:37.718 15:46:37.717 INFO: Default locale: "en_US", source code encoding: "UTF-8"
&1|15:46:38.284 15:46:38.284 INFO: Load global settings
&1|15:46:38.529 15:46:38.529 INFO: Load global settings (done) | time=246ms
&1|15:46:38.535 15:46:38.534 INFO: Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
&1|15:46:38.551 15:46:38.551 INFO: User cache: /root/.sonar/cache
&1|15:46:38.558 15:46:38.558 INFO: Load/download plugins
&1|15:46:38.558 15:46:38.558 INFO: Load plugins index
&1|15:46:38.811 15:46:38.811 INFO: Load plugins index (done) | time=253ms
&1|15:46:47.222 15:46:47.221 INFO: Load/download plugins (done) | time=8663ms
&1|15:46:47.807 15:46:47.806 INFO: Loaded core extensions: developer-scanner
&1|15:46:48.356 15:46:48.356 INFO: Load project settings for component key: 'xxorgxx_xxprojxx'
&1|15:46:48.572 15:46:48.572 INFO: Load project settings for component key: 'xxorgxx_xxprojxx' (done) | time=215ms
&1|15:46:48.576 15:46:48.576 INFO: Process project properties
&1|15:46:48.582 15:46:48.582 INFO: Execute project builders
&1|15:46:48.584 15:46:48.584 INFO: Execute project builders (done) | time=2ms
&1|15:46:48.586 15:46:48.586 INFO: Project key: xxorgxx_xxprojxx
&1|15:46:48.586 15:46:48.586 INFO: Base dir: /xxprojxx
&1|15:46:48.586 15:46:48.586 INFO: Working dir: /xxprojxx/.scannerwork
&1|15:46:48.700 15:46:48.700 INFO: Load project branches
&1|15:46:48.924 15:46:48.924 INFO: Load project branches (done) | time=224ms
&1|15:46:48.928 15:46:48.928 INFO: Check ALM binding of project 'xxorgxx_xxprojxx'
&1|15:46:49.112 15:46:49.112 INFO: Detected project binding: BOUND
&1|15:46:49.113 15:46:49.112 INFO: Check ALM binding of project 'xxorgxx_xxprojxx' (done) | time=185ms
&1|15:46:49.115 15:46:49.115 INFO: Load project pull requests
&1|15:46:49.489 15:46:49.489 INFO: Load project pull requests (done) | time=374ms
&1|15:46:49.493 15:46:49.493 INFO: Load branch configuration
&1|15:46:49.876 15:46:49.876 INFO: Load branch configuration (done) | time=384ms
&1|15:46:49.909 15:46:49.909 INFO: Load quality profiles
&1|15:46:50.170 15:46:50.170 INFO: Load quality profiles (done) | time=261ms
&1|15:46:50.175 15:46:50.175 INFO: Load active rules
&1|15:46:52.851 15:46:52.850 INFO: Load active rules (done) | time=2675ms
&1|15:46:52.886 15:46:52.886 INFO: Organization key: xxorgxx
&1|15:46:52.887 15:46:52.887 INFO: Pull request 30671 for merge into development from hotfix/RGD-53296
&1|15:46:52.903 15:46:52.903 INFO: Load project repositories
&1|15:46:53.771 15:46:53.770 INFO: Load project repositories (done) | time=867ms
&1|15:46:53.772 15:46:53.772 INFO: SCM collecting changed files in the branch
&1|15:47:14.471 15:47:14.471 INFO: 17934 files indexed... (last one was tests/Codeception/Acceptance/Full/Comms/SmartBlogs/BlogsCest.php)
&1|15:47:17.127 15:47:17.127 INFO: 20529 files indexed
&1|15:47:17.127 15:47:17.127 INFO: 11314 files ignored because of inclusion/exclusion patterns
&1|15:47:17.127 15:47:17.127 INFO: 13 files ignored because of scm ignore settings
&1|15:47:17.129 15:47:17.128 INFO: Quality profile for js: Sonar way
&1|15:47:17.129 15:47:17.128 INFO: Quality profile for json: SonarQube Way
&1|15:47:17.129 15:47:17.128 INFO: Quality profile for php: XX - PHP
&1|15:47:17.129 15:47:17.128 INFO: Quality profile for web: XX - HTML
&1|15:47:17.129 15:47:17.128 INFO: Quality profile for yaml: Sonar way
&1|15:47:17.149 15:47:17.148 INFO: ------------- Run sensors on module xxorgxx_xxprojxx
&1|15:47:17.324 15:47:17.324 INFO: Load metrics repository
&1|15:47:17.547 15:47:17.547 INFO: Load metrics repository (done) | time=223ms
&1|15:47:20.596 15:47:20.596 INFO: Sensor IaC CloudFormation Sensor [iac]
&1|15:47:20.822 15:47:20.821 INFO: 0 source files to be analyzed
&1|15:47:20.902 15:47:20.902 INFO: 0/0 source files have been analyzed
&1|15:47:20.904 15:47:20.902 INFO: Sensor IaC CloudFormation Sensor [iac] (done) | time=307ms
&1|15:47:20.904 15:47:20.903 INFO: Sensor IaC Kubernetes Sensor [iac]
&1|15:47:21.073 15:47:21.073 INFO: 0 source files to be analyzed
&1|15:47:21.158 15:47:21.157 INFO: 0/0 source files have been analyzed
&1|15:47:21.158 15:47:21.158 INFO: Sensor IaC Kubernetes Sensor [iac] (done) | time=255ms
&1|15:47:21.158 15:47:21.158 INFO: Sensor C# Project Type Information [csharp]
&1|15:47:21.174 15:47:21.174 INFO: Sensor C# Project Type Information [csharp] (done) | time=16ms
&1|15:47:21.175 15:47:21.174 INFO: Sensor C# Analysis Log [csharp]
&1|15:47:21.194 15:47:21.193 INFO: Sensor C# Analysis Log [csharp] (done) | time=19ms
&1|15:47:21.194 15:47:21.194 INFO: Sensor C# Properties [csharp]
&1|15:47:21.194 15:47:21.194 INFO: Sensor C# Properties [csharp] (done) | time=0ms
&1|15:47:21.194 15:47:21.194 INFO: Sensor HTML [web]
&1|15:47:21.194 15:47:21.194 INFO: Sensor HTML is restricted to changed files only
&1|15:47:22.016 15:47:22.016 INFO: Sensor HTML [web] (done) | time=822ms
&1|15:47:22.016 15:47:22.016 INFO: Sensor Text Sensor [text]
&1|15:47:22.032 15:47:22.031 INFO: 20424 source files to be analyzed
&1|15:47:25.616 15:47:25.616 INFO: 20424/20424 source files have been analyzed
&1|15:47:25.617 15:47:25.616 INFO: Sensor Text Sensor [text] (done) | time=3600ms
&1|15:47:25.617 15:47:25.617 INFO: Sensor VB.NET Project Type Information [vbnet]
&1|15:47:25.628 15:47:25.628 INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=11ms
&1|15:47:25.629 15:47:25.628 INFO: Sensor VB.NET Analysis Log [vbnet]
&1|15:47:25.643 15:47:25.643 INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=15ms
&1|15:47:25.643 15:47:25.643 INFO: Sensor VB.NET Properties [vbnet]
&1|15:47:25.643 15:47:25.643 INFO: Sensor VB.NET Properties [vbnet] (done) | time=0ms
&1|15:47:25.644 15:47:25.644 INFO: Sensor JaCoCo XML Report Importer [jacoco]
&1|15:47:25.709 15:47:25.708 INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
&1|15:47:25.710 15:47:25.709 INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
&1|15:47:25.710 15:47:25.710 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=66ms
&1|15:47:25.710 15:47:25.710 INFO: Sensor JavaScript analysis [javascript]
&1|15:47:27.709 15:47:27.709 INFO: Deploying custom rules bundle jar:file:/root/.sonar/cache/985cd0bb0f213099aac4466b1eea8b9a/sonar-securityjsfrontend-plugin.jar!/js-vulnerabilities-rules-1.0.0.tgz to /xxprojxx/.scannerwork/.sonartmp/eslint-bridge-bundle/package/custom-rules16310703578825233898
&1|15:47:29.632 15:47:29.632 INFO: 66 source files to be analyzed
&1|15:47:35.120 15:47:35.120 INFO: 66/66 source files have been analyzed
&1|15:47:35.120 15:47:35.120 INFO: Hit the cache for 0 out of 66
&1|15:47:35.122 15:47:35.122 INFO: Miss the cache for 66 out of 66: FILE_NOT_IN_CACHE [66/66]
&1|15:47:35.122 15:47:35.122 INFO: Sensor JavaScript analysis [javascript] (done) | time=9412ms
&1|15:47:35.122 15:47:35.122 INFO: Sensor TypeScript analysis [javascript]
&1|15:47:35.141 15:47:35.141 INFO: No input files found for analysis
&1|15:47:35.141 15:47:35.141 INFO: Hit the cache for 0 out of 0
&1|15:47:35.141 15:47:35.141 INFO: Miss the cache for 0 out of 0
&1|15:47:35.141 15:47:35.141 INFO: Sensor TypeScript analysis [javascript] (done) | time=19ms
&1|15:47:35.142 15:47:35.142 INFO: Sensor JavaScript inside YAML analysis [javascript]
&1|15:47:35.153 15:47:35.153 INFO: 12 source files to be analyzed
&1|15:47:35.267 15:47:35.267 INFO: 12/12 source files have been analyzed
&1|15:47:35.267 15:47:35.267 INFO: Hit the cache for 0 out of 12
&1|15:47:35.268 15:47:35.267 INFO: Miss the cache for 12 out of 12: FILE_NOT_IN_CACHE [12/12]
&1|15:47:35.268 15:47:35.268 INFO: Sensor JavaScript inside YAML analysis [javascript] (done) | time=125ms
&1|15:47:35.268 15:47:35.268 INFO: Sensor CSS Rules [javascript]
&1|15:47:35.268 15:47:35.268 INFO: Sensor CSS Rules is restricted to changed files only
&1|15:47:35.296 15:47:35.296 INFO: 284 source files to be analyzed
&1|15:47:37.688 15:47:37.688 INFO: 284/284 source files have been analyzed
&1|15:47:37.688 15:47:37.688 INFO: Hit the cache for 0 out of 0
&1|15:47:37.689 15:47:37.688 INFO: Miss the cache for 0 out of 0
&1|15:47:37.689 15:47:37.688 INFO: Sensor CSS Rules [javascript] (done) | time=2420ms
&1|15:47:37.689 15:47:37.689 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
&1|15:47:37.728 15:47:37.727 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=38ms
&1|15:47:37.728 15:47:37.728 INFO: Sensor PHP sensor [php]
&1|15:47:37.773 15:47:37.773 INFO: Starting PHP symbol indexer
&1|15:47:37.793 15:47:37.793 INFO: 20310 source files to be analyzed
&1|15:52:24.044 15:52:24.044 INFO: 20310/20310 source files have been analyzed
&1|15:52:24.060 15:52:24.059 INFO: Starting PHP rules
&1|15:52:24.065 15:52:24.065 INFO: 20310 source files to be analyzed
&1|16:08:17.223 16:08:17.223 INFO: 20310/20310 source files have been analyzed
&1|16:08:17.226 16:08:17.225 INFO: Importing /var/tmp/tests_report.xml
&1|16:08:18.063 16:08:18.063 INFO: Importing /var/tmp/coverage_report.xml
&1|16:08:19.607 16:08:19.605 INFO: Sensor PHP sensor [php] (done) | time=1241877ms
&1|16:08:19.607 16:08:19.606 INFO: Sensor Analyzer for "php.ini" files [php]
&1|16:08:19.687 16:08:19.686 INFO: Sensor Analyzer for "php.ini" files [php] (done) | time=80ms
&1|16:08:19.687 16:08:19.687 INFO: Sensor Serverless configuration file sensor [security]
&1|16:08:19.703 16:08:19.703 INFO: 0 Serverless function entries were found in the project
&1|16:08:19.707 16:08:19.707 INFO: 0 Serverless function handlers were kept as entrypoints
&1|16:08:19.707 16:08:19.707 INFO: Sensor Serverless configuration file sensor [security] (done) | time=20ms
&1|16:08:19.707 16:08:19.707 INFO: Sensor AWS SAM template file sensor [security]
&1|16:08:19.726 16:08:19.726 INFO: Sensor AWS SAM template file sensor [security] (done) | time=19ms
&1|16:08:19.726 16:08:19.726 INFO: Sensor javabugs [dbd]
&1|16:08:19.727 16:08:19.727 INFO: Reading IR files from: /xxprojxx/.scannerwork/ir/java
&1|16:08:19.727 16:08:19.727 INFO: No IR files have been included for analysis.
&1|16:08:19.727 16:08:19.727 INFO: Sensor javabugs [dbd] (done) | time=1ms
&1|16:08:19.727 16:08:19.727 INFO: Sensor pythonbugs [dbd]
&1|16:08:19.729 16:08:19.729 INFO: Reading IR files from: /xxprojxx/.scannerwork/ir/python
&1|16:08:19.729 16:08:19.729 INFO: No IR files have been included for analysis.
&1|16:08:19.729 16:08:19.729 INFO: Sensor pythonbugs [dbd] (done) | time=2ms
&1|16:08:19.729 16:08:19.729 INFO: Sensor JavaSecuritySensor [security]
&1|16:08:19.731 16:08:19.730 INFO: Reading type hierarchy from: /xxprojxx/.scannerwork/ucfg2/java
&1|16:08:19.731 16:08:19.730 INFO: Read 0 type definitions
&1|16:08:19.733 16:08:19.733 INFO: Reading UCFGs from: /xxprojxx/.scannerwork/ucfg2/java
&1|16:08:19.734 16:08:19.733 INFO: No UCFGs have been included for analysis.
&1|16:08:19.734 16:08:19.734 INFO: Sensor JavaSecuritySensor [security] (done) | time=5ms
&1|16:08:19.734 16:08:19.734 INFO: Sensor CSharpSecuritySensor [security]
&1|16:08:19.734 16:08:19.734 INFO: Reading type hierarchy from: /xxprojxx/ucfg_cs2
&1|16:08:19.734 16:08:19.734 INFO: Read 0 type definitions
&1|16:08:19.734 16:08:19.734 INFO: Reading UCFGs from: /xxprojxx/ucfg_cs2
&1|16:08:19.734 16:08:19.734 INFO: No UCFGs have been included for analysis.
&1|16:08:19.734 16:08:19.734 INFO: Sensor CSharpSecuritySensor [security] (done) | time=0ms
&1|16:08:19.734 16:08:19.734 INFO: Sensor PhpSecuritySensor [security]
&1|16:08:19.734 16:08:19.734 INFO: Reading type hierarchy from: /xxprojxx/.scannerwork/ucfg2/php
&1|16:08:21.480 16:08:21.480 INFO: Read 14274 type definitions
&1|16:08:21.703 16:08:21.703 INFO: Reading UCFGs from: /xxprojxx/.scannerwork/ucfg2/php
&1|16:08:40.831 16:08:40.830 INFO: 16:08:40.827009 Running Tarjan on 737315 nodes
&1|16:08:41.990 16:08:41.990 INFO: 16:08:41.989647 Tarjan found 734079 components
&1|16:08:44.716 16:08:44.715 INFO: 16:08:44.715516 Variable type analysis: done
&1|16:08:44.720 16:08:44.720 INFO: 16:08:44.719215 Building Runtime Type propagation graph
&1|16:09:28.245 16:09:28.245 INFO: 16:09:28.245443 Running Tarjan on 923086 nodes
&1|16:09:32.874 16:09:32.873 INFO: 16:09:32.873592 Tarjan found 890076 components
&1|16:09:41.414 16:09:41.414 INFO: Skipping component with 30708 elements and 2183 types
&1|16:09:49.492 16:09:49.492 INFO: 16:09:49.492304 Variable type analysis: done
&1|16:09:49.493 16:09:49.492 INFO: 16:09:49.492912 Building Runtime Type propagation graph
&1|16:11:11.985 16:11:11.984 INFO: 16:11:11.984497 Running Tarjan on 916626 nodes
&1|16:11:16.837 16:11:16.837 INFO: 16:11:16.83752 Tarjan found 871065 components
&1|16:11:24.509 16:11:24.509 INFO: Skipping component with 40144 elements and 1788 types
&1|16:11:26.894 16:11:26.894 INFO: 16:11:26.894 Variable type analysis: done
&1|16:11:26.955 16:11:26.955 INFO: Analyzing 103384 ucfgs to detect vulnerabilities.
&1|16:13:42.557 16:13:42.556 INFO: All rules entrypoints : 2203
&1|16:13:42.557 16:13:42.557 INFO: Retained UCFGs : 20530
&1|16:13:46.629 16:13:46.628 INFO: Taint analysis starting. Entrypoints: 2203
&1|16:13:46.637 16:13:46.637 INFO: Running symbolic analysis for 'PHP'
&1|16:14:25.341 16:14:25.341 INFO: Taint analysis: done.
&1|16:14:25.341 16:14:25.341 INFO: Sensor PhpSecuritySensor [security] (done) | time=365607ms
&1|16:14:25.341 16:14:25.341 INFO: Sensor PythonSecuritySensor [security]
&1|16:14:25.341 16:14:25.341 INFO: Reading type hierarchy from: /xxprojxx/.scannerwork/ucfg2/python
&1|16:14:25.341 16:14:25.341 INFO: Read 0 type definitions
&1|16:14:25.342 16:14:25.341 INFO: Reading UCFGs from: /xxprojxx/.scannerwork/ucfg2/python
&1|16:14:25.342 16:14:25.341 INFO: No UCFGs have been included for analysis.
&1|16:14:25.342 16:14:25.341 INFO: Sensor PythonSecuritySensor [security] (done) | time=0ms
&1|16:14:25.342 16:14:25.342 INFO: Sensor JsSecuritySensor [security]
&1|16:14:25.342 16:14:25.342 INFO: Reading type hierarchy from: /xxprojxx/.scannerwork/ucfg2/js
&1|16:14:25.344 16:14:25.344 INFO: Read 0 type definitions
&1|16:14:25.344 16:14:25.344 INFO: Reading UCFGs from: /xxprojxx/.scannerwork/ucfg2/js
&1|16:14:25.387 16:14:25.387 INFO: 16:14:25.387455 Building Runtime Type propagation graph
&1|16:14:25.394 16:14:25.394 INFO: 16:14:25.393815 Running Tarjan on 465 nodes
&1|16:14:25.395 16:14:25.395 INFO: 16:14:25.395115 Tarjan found 465 components
&1|16:14:25.397 16:14:25.397 INFO: 16:14:25.397109 Variable type analysis: done
&1|16:14:25.397 16:14:25.397 INFO: 16:14:25.397472 Building Runtime Type propagation graph
&1|16:14:25.403 16:14:25.403 INFO: 16:14:25.40363 Running Tarjan on 465 nodes
&1|16:14:25.404 16:14:25.404 INFO: 16:14:25.404311 Tarjan found 465 components
&1|16:14:25.405 16:14:25.405 INFO: 16:14:25.405137 Variable type analysis: done
&1|16:14:25.405 16:14:25.405 INFO: Analyzing 107 ucfgs to detect vulnerabilities.
&1|16:14:25.637 16:14:25.637 INFO: Taint analysis starting. Entrypoints: 34
&1|16:14:25.637 16:14:25.637 INFO: Running symbolic analysis for 'JS'
&1|16:14:26.356 16:14:26.356 INFO: Taint analysis: done.
&1|16:14:26.356 16:14:26.356 INFO: Sensor JsSecuritySensor [security] (done) | time=1014ms
&1|16:14:26.365 16:14:26.365 INFO: ------------- Run sensors on project
&1|16:14:26.496 16:14:26.496 INFO: Sensor Analysis Warnings import [csharp]
&1|16:14:26.497 16:14:26.497 INFO: Sensor Analysis Warnings import [csharp] (done) | time=1ms
&1|16:14:26.497 16:14:26.497 INFO: Sensor Zero Coverage Sensor
&1|16:14:26.877 16:14:26.877 INFO: Sensor Zero Coverage Sensor (done) | time=380ms
&1|16:14:26.908 16:14:26.907 INFO: SCM Publisher SCM provider for this project is: git
&1|16:14:26.912 16:14:26.912 INFO: SCM Publisher 349 source files to be analyzed
&1|16:14:27.163 16:14:27.162 INFO: Blaming files using native implementation
&1|16:14:36.925 16:14:36.925 INFO: 39/349 source files have been analyzed
&1|16:14:46.925 16:14:46.925 INFO: 95/349 source files have been analyzed
&1|16:14:56.926 16:14:56.926 INFO: 166/349 source files have been analyzed
&1|16:15:06.926 16:15:06.926 INFO: 227/349 source files have been analyzed
&1|16:15:16.926 16:15:16.926 INFO: 290/349 source files have been analyzed
&1|16:15:31.996 16:15:31.996 INFO: CPD Executor CPD calculation finished (done) | time=4296ms
&1|16:15:33.170 16:15:33.170 INFO: SCM writing changed lines
&1|16:15:37.291 16:15:37.291 INFO: SCM writing changed lines (done) | time=4121ms
&1|16:15:38.803 16:15:38.802 INFO: Analysis report generated in 5378ms, dir size=26 MB
&1|16:16:01.963 16:16:01.963 INFO: Analysis report compressed in 23160ms, zip size=18 MB
&1|16:16:01.963 16:16:01.963 INFO: Analysis report generated in /xxprojxx/.scannerwork/scanner-report
&1|16:16:02.889 16:16:02.889 INFO: Analysis report uploaded in 924ms
&1|16:16:02.891 16:16:02.891 INFO: ANALYSIS SUCCESSFUL, you can find the results at: xxx
&1|16:16:02.891 16:16:02.891 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
&1|16:16:02.891 16:16:02.891 INFO: More about the report processing at xxx
&1|16:16:08.778 16:16:08.777 INFO: Analysis total time: 29:20.971 s
&1|16:16:08.779 16:16:08.779 INFO: ------------------------------------------------------------------------
&1|16:16:08.780 16:16:08.779 INFO: EXECUTION SUCCESS
&1|16:16:08.780 16:16:08.779 INFO: ------------------------------------------------------------------------
&1|16:16:08.780 16:16:08.779 INFO: Total time: 29:33.353s
&1|16:16:09.164 16:16:09.163 INFO: Final Memory: 140M/474M
&1|16:16:09.164 16:16:09.164 INFO: ------------------------------------------------------------------------
And if so, can you also provide the logs from the checkout / acquisition of the code for the pipeline?
Yes, sorry i should have mentioned that OP and me work in the same company.
Here is the full (as full as it can be) output of what we do and what we see as a result.
2 Screenshots are attached from BitBucket and SonarCloud UI respectfully as well. Screenshot 2022-09-28 at 9.56.112332×654 42.5 KBScreenshot 2022-09-28 at 9.56.302348×410 48.7 KB
#this is our sonar-project.properties file
# Organization and project keys are displayed in the right sidebar of the project homepage
sonar.organization=xxxx
sonar.projectKey=xxxx
sonar.host.url=https://sonarcloud.io
sonar.sources=xxxx
sonar.tests=./tests
sonar.language=php
sonar.sourceEncoding=UTF-8
sonar.php.coverage.reportPaths=/var/tmp/coverage_report.xml
sonar.php.tests.reportPath=/var/tmp/tests_report.xml
sonar.coverage.exclusions=xxxx
sonar.exclusions=xxxx
#end of our sonar-project.properties file
# clone repo
git clone xxGITURLxx && git fetch && git checkout hotfix/RGD-0000-BLABLA
.....
# do whatever changes youre doing, in this test case add a test commit with a dummy f()
git diff origin/development..HEAD
diff --git a/common/xxpath1xx/xxpath2xx/xxpath3xx/xxfile.class.phpxx b/common/xxpath1xx/xxpath2xx/xxpath3xx/xxfile.class.phpxx
index 6399982ac0f..044fc3bd051 100644
--- a/common/xxpath1xx/xxpath2xx/xxpath3xx/xxfile.class.phpxx
+++ b/common/xxpath1xx/xxpath2xx/xxpath3xx/xxfile.class.phpxx
@@ -2121,8 +2121,4 @@
+ public function dummy() {
+ echo 'This is a dummy f()';
+ }
#push the changes, and verify
git log origin/development..origin/hotfix/RGD-0000-BLABLA --oneline --no-merges
65243a1ac08 (HEAD -> hotfix/RGD-0000-BLABLA, origin/hotfix/RGD-0000-BLABLA) test
# at this moment we have only 1 change (1 file) between development and our branch
# generate code coverage report to use in sonar
/usr/bin/php -d memory_limit=-1 /xxPROJxx/vendor/bin/phpunit --coverage-clover /var/tmp/coverage_report.xml --log-junit /var/tmp/tests_report.xml
PHPUnit 9.3.11 by Sebastian Bergmann and contributors.
# run sonar with
/sonar-scanner-4.4.0.2170-linux/bin/sonar-scanner -Dsonar.verbose=true -Dsonar.login=xxsonarcredsxx -Dsonar.pullrequest.branch=hotfix/RGD-0000-BLABLA -Dsonar.pullrequest.key=30768 -Dsonar.pullrequest.base=development
09:17:07.193 INFO: SonarScanner 4.4.0.2170
09:17:07.195 INFO: Java 13.0.1 Oracle Corporation (64-bit)
09:17:07.195 INFO: Mac OS X 10.16 x86_64
09:17:08.118 INFO: Analyzing on SonarCloud
09:17:08.710 INFO: Load global settings
09:17:09.049 INFO: Load global settings (done) | time=339ms
09:17:09.057 INFO: Server id: 1BD809FA-AWHW8ct9-T_TB3XqouNu
09:17:09.075 INFO: Load/download plugins
09:17:09.076 INFO: Load plugins index
09:17:09.384 INFO: Load plugins index (done) | time=307ms
09:17:09.656 INFO: Load/download plugins (done) | time=581ms
09:17:10.140 INFO: Loaded core extensions: developer-scanner
09:17:10.723 INFO: Load project settings for component key: 'xxx'
09:17:10.990 INFO: Load project settings for component key: 'xxx' (done) | time=267ms
09:17:10.994 INFO: Process project properties
09:17:11.002 INFO: Execute project builders
09:17:11.004 INFO: Execute project builders (done) | time=1ms
09:17:11.006 INFO: Project key: xxx
09:17:18.854 INFO: Load project branches
09:17:19.138 INFO: Load project branches (done) | time=284ms
09:17:19.141 INFO: Check ALM binding of project 'xxx'
09:17:19.393 INFO: Detected project binding: BOUND
09:17:19.393 INFO: Check ALM binding of project 'xxx' (done) | time=252ms
09:17:19.395 INFO: Load project pull requests
09:17:19.946 INFO: Load project pull requests (done) | time=551ms
09:17:19.948 INFO: Load branch configuration
09:17:20.369 INFO: Load branch configuration (done) | time=421ms
09:17:20.398 INFO: Load quality profiles
09:17:20.720 INFO: Load quality profiles (done) | time=322ms
09:17:20.725 INFO: Load active rules
09:17:23.977 INFO: Load active rules (done) | time=3252ms
09:17:24.029 INFO: Organization key: xxx
09:17:24.029 INFO: Pull request 30768 for merge into development from hotfix/RGD-0000-BLABLA
09:17:24.046 INFO: Load project repositories
09:17:25.841 INFO: Load project repositories (done) | time=1795ms
09:17:25.842 INFO: SCM collecting changed files in the branch
09:17:26.635 INFO: SCM collecting changed files in the branch (done) | time=793ms
09:17:26.659 INFO: Indexing files...
09:17:26.660 INFO: Project configuration:
09:17:26.660 INFO: Excluded sources: xxx
09:17:26.660 INFO: Excluded sources for coverage: xxx
09:17:26.660 INFO: Excluded sources for duplication: xxx
09:17:36.661 INFO: 6013 files indexed... (last one was xxx)
09:17:46.662 INFO: 14669 files indexed... (last one was xxx)
09:17:54.008 INFO: 20605 files indexed
09:17:54.008 INFO: 11705 files ignored because of inclusion/exclusion patterns
09:17:54.008 INFO: 17 files ignored because of scm ignore settings
09:17:54.009 INFO: Quality profile for js: Sonar way
09:17:54.009 INFO: Quality profile for json: SonarQube Way
09:17:54.009 INFO: Quality profile for php: RG - PHP
09:17:54.009 INFO: Quality profile for web: RG - HTML
09:17:54.009 INFO: Quality profile for yaml: Sonar way
09:17:54.145 INFO: ------------- Run sensors on module xxx
09:17:54.303 INFO: Load metrics repository
09:17:54.577 INFO: Load metrics repository (done) | time=274ms
09:17:57.250 INFO: Sensor IaC CloudFormation Sensor [iac]
09:17:57.510 INFO: 0 source files to be analyzed
09:17:57.569 INFO: 0/0 source files have been analyzed
09:17:57.569 INFO: Sensor IaC CloudFormation Sensor [iac] (done) | time=320ms
09:17:57.569 INFO: Sensor IaC Kubernetes Sensor [iac]
09:17:57.642 INFO: 0 source files to be analyzed
09:17:57.724 INFO: 0/0 source files have been analyzed
09:17:57.724 INFO: Sensor IaC Kubernetes Sensor [iac] (done) | time=155ms
09:17:57.725 INFO: Sensor C# Project Type Information [csharp]
09:17:57.735 INFO: Sensor C# Project Type Information [csharp] (done) | time=10ms
09:17:57.735 INFO: Sensor C# Analysis Log [csharp]
09:17:57.754 INFO: Sensor C# Analysis Log [csharp] (done) | time=19ms
09:17:57.755 INFO: Sensor C# Properties [csharp]
09:17:57.755 INFO: Sensor C# Properties [csharp] (done) | time=0ms
09:17:57.755 INFO: Sensor HTML [web]
09:17:57.755 INFO: Sensor HTML is restricted to changed files only
09:17:58.376 INFO: Sensor HTML [web] (done) | time=621ms
09:17:58.376 INFO: Sensor Text Sensor [text]
09:17:58.385 INFO: 20500 source files to be analyzed
09:18:08.390 INFO: 18126/20500 files analyzed, current file: xxx
09:18:10.358 INFO: 20500/20500 source files have been analyzed
09:18:10.358 INFO: Sensor Text Sensor [text] (done) | time=11982ms
09:18:10.358 INFO: Sensor VB.NET Project Type Information [vbnet]
09:18:10.366 INFO: Sensor VB.NET Project Type Information [vbnet] (done) | time=8ms
09:18:10.366 INFO: Sensor VB.NET Analysis Log [vbnet]
09:18:10.385 INFO: Sensor VB.NET Analysis Log [vbnet] (done) | time=19ms
09:18:10.385 INFO: Sensor VB.NET Properties [vbnet]
09:18:10.386 INFO: Sensor VB.NET Properties [vbnet] (done) | time=1ms
09:18:10.386 INFO: Sensor JaCoCo XML Report Importer [jacoco]
09:18:10.419 INFO: 'sonar.coverage.jacoco.xmlReportPaths' is not defined. Using default locations: target/site/jacoco/jacoco.xml,target/site/jacoco-it/jacoco.xml,build/reports/jacoco/test/jacocoTestReport.xml
09:18:10.419 INFO: No report imported, no coverage information will be imported by JaCoCo XML Report Importer
09:18:10.419 INFO: Sensor JaCoCo XML Report Importer [jacoco] (done) | time=33ms
09:18:10.420 INFO: Sensor JavaScript analysis [javascript]
09:18:20.413 INFO: 66 source files to be analyzed
09:18:24.174 INFO: 66/66 source files have been analyzed
09:18:24.175 INFO: Hit the cache for 0 out of 66
09:18:24.175 INFO: Miss the cache for 66 out of 66: FILE_NOT_IN_CACHE [66/66]
09:18:24.176 INFO: Sensor JavaScript analysis [javascript] (done) | time=13755ms
09:18:24.176 INFO: Sensor TypeScript analysis [javascript]
09:18:24.186 INFO: No input files found for analysis
09:18:24.186 INFO: Hit the cache for 0 out of 0
09:18:24.186 INFO: Miss the cache for 0 out of 0
09:18:24.186 INFO: Sensor TypeScript analysis [javascript] (done) | time=10ms
09:18:24.187 INFO: Sensor JavaScript inside YAML analysis [javascript]
09:18:24.192 INFO: 12 source files to be analyzed
09:18:24.290 INFO: 12/12 source files have been analyzed
09:18:24.290 INFO: Hit the cache for 0 out of 12
09:18:24.290 INFO: Miss the cache for 12 out of 12: FILE_NOT_IN_CACHE [12/12]
09:18:24.290 INFO: Sensor JavaScript inside YAML analysis [javascript] (done) | time=103ms
09:18:24.290 INFO: Sensor CSS Rules [javascript]
09:18:24.290 INFO: Sensor CSS Rules is restricted to changed files only
09:18:24.309 INFO: 289 source files to be analyzed
09:18:26.453 INFO: 289/289 source files have been analyzed
09:18:26.453 INFO: Hit the cache for 0 out of 0
09:18:26.453 INFO: Miss the cache for 0 out of 0
09:18:26.453 INFO: Sensor CSS Rules [javascript] (done) | time=2163ms
09:18:26.454 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend]
09:18:26.509 INFO: Sensor ThymeLeaf template sensor [securityjavafrontend] (done) | time=55ms
09:18:26.510 INFO: Sensor PHP sensor [php]
09:18:26.540 INFO: Starting PHP symbol indexer
09:18:26.543 INFO: 20386 source files to be analyzed
09:18:36.548 INFO: 577/20386 files analyzed, current file: xxx
09:18:46.550 INFO: 1363/20386 files analyzed, current file: xxx
...........................................
09:23:03.611 INFO: 20386/20386 source files have been analyzed
09:23:03.628 INFO: Starting PHP rules
09:23:03.631 INFO: 20386 source files to be analyzed
09:23:13.636 INFO: 211/20386 files analyzed, current file: xxx
09:23:23.638 INFO: 451/20386 files analyzed, current file: xxx
...........................................
09:37:24.036 INFO: 20211/20386 files analyzed, current file: xxx
09:37:31.251 WARN: Failed to resolve 1 include/require statements like 'xxx' from 'admin'
09:37:31.282 INFO: 20386/20386 source files have been analyzed
09:37:31.288 INFO: Importing /var/tmp/tests_report.xml
09:37:31.501 WARN: Test cases must always be descendants of a file-based suite, skipping : xxx in xxx
09:37:31.501 WARN: Test cases must always be descendants of a file-based suite, skipping : xxx in xxx
09:37:31.502 WARN: Test cases must always be descendants of a file-based suite, skipping : xxx in xxx
09:37:34.701 INFO: Importing /var/tmp/coverage_report.xml
09:37:39.070 INFO: Sensor PHP sensor [php] (done) | time=1152560ms
09:37:39.071 INFO: Sensor Analyzer for "php.ini" files [php]
09:37:39.135 INFO: Sensor Analyzer for "php.ini" files [php] (done) | time=64ms
09:37:39.135 INFO: Sensor Serverless configuration file sensor [security]
09:37:39.147 INFO: 0 Serverless function entries were found in the project
09:37:39.151 INFO: 0 Serverless function handlers were kept as entrypoints
09:37:39.152 INFO: Sensor Serverless configuration file sensor [security] (done) | time=17ms
09:37:39.152 INFO: Sensor AWS SAM template file sensor [security]
09:37:39.175 INFO: Sensor AWS SAM template file sensor [security] (done) | time=23ms
09:37:39.175 INFO: Sensor javabugs [dbd]
09:37:39.181 INFO: No IR files have been included for analysis.
09:37:39.181 INFO: Sensor javabugs [dbd] (done) | time=6ms
09:37:39.182 INFO: Sensor pythonbugs [dbd]
09:37:39.184 INFO: No IR files have been included for analysis.
09:37:39.184 INFO: Sensor pythonbugs [dbd] (done) | time=2ms
09:37:39.184 INFO: Sensor JavaSecuritySensor [security]
09:37:39.185 INFO: Read 0 type definitions
09:37:39.188 INFO: No UCFGs have been included for analysis.
09:37:39.188 INFO: Sensor JavaSecuritySensor [security] (done) | time=4ms
09:37:39.188 INFO: Sensor CSharpSecuritySensor [security]
09:37:39.191 INFO: Read 0 type definitions
09:37:39.191 INFO: No UCFGs have been included for analysis.
09:37:39.191 INFO: Sensor CSharpSecuritySensor [security] (done) | time=3ms
09:37:39.192 INFO: Sensor PhpSecuritySensor [security]
09:37:42.178 INFO: Read 14326 type definitions
09:38:34.727 INFO: 09:38:34.726709 Running Tarjan on 738756 nodes
09:38:35.558 INFO: 09:38:35.557777 Tarjan found 735515 components
09:38:37.872 INFO: 09:38:37.872773 Variable type analysis: done
09:38:37.873 INFO: 09:38:37.873467 Building Runtime Type propagation graph
09:39:16.666 INFO: 09:39:16.665779 Running Tarjan on 924769 nodes
09:39:20.704 INFO: 09:39:20.704473 Tarjan found 891184 components
09:39:28.165 INFO: Skipping component with 31331 elements and 2189 types
09:39:36.019 INFO: 09:39:36.019446 Variable type analysis: done
09:39:36.021 INFO: 09:39:36.02122 Building Runtime Type propagation graph
09:40:42.698 INFO: 09:40:42.697735 Running Tarjan on 918516 nodes
09:40:47.004 INFO: 09:40:47.004148 Tarjan found 872306 components
09:40:53.451 INFO: Skipping component with 40899 elements and 1796 types
09:40:55.297 INFO: 09:40:55.29733 Variable type analysis: done
09:40:55.383 INFO: Analyzing 103649 ucfgs to detect vulnerabilities.
09:42:41.602 INFO: All rules entrypoints : 2088
09:42:41.602 INFO: Retained UCFGs : 19873
09:42:43.610 INFO: Taint analysis starting. Entrypoints: 2088
09:42:43.636 INFO: Running symbolic analysis for 'PHP'
at 193:74 - 194:105
at 183:79 - 183:179
at 90:28 - 90:95
at 91:28 - 91:95
at 92:28 - 92:95
at 93:49 - 93:116
at 128:107 - 128:160
at 179:71 - 197:5
at 61:22 - 61:124
at 64:22 - 64:123
at 68:22 - 68:131
at 140:22 - 140:124
at 143:22 - 143:123
at 147:22 - 147:131
at 29:74 - 29:170
at 30:70 - 35:33
at 87:31 - 87:58
at 75:70 - 80:17
at 53:34 - 54:61
at 66:34 - 68:40
at 91:167 - 91:320
at 100:166 - 100:313
at 30:89 - 30:135
at 127:90 - 127:147
at 54:91 - 54:165
at 231:54 - 231:121
at 25:114 - 25:184
at 25:285 - 25:351
at 460:49 - 460:155
at 462:49 - 462:148
at 463:49 - 463:148
at 464:49 - 464:150
at 465:49 - 465:149
at 95:140 - 95:163
at 165:25 - 165:54
at 222:28 - 222:57
at 198:145 - 198:165
at 165:25 - 165:54
at 222:28 - 222:57
at 198:145 - 198:165
at 165:25 - 165:54
at 222:28 - 222:57
at 198:145 - 198:165
at 165:25 - 165:54
at 222:28 - 222:57
at 198:145 - 198:165
at 820:71 - 820:117
at 85:12 - 88:13
at 151:16 - 151:142
at 551:16 - 551:47
at 552:16 - 552:37
at 470:12 - 470:85
at 101:36 - 101:83
at 111:36 - 111:83
array: Y(this.positions, object: D(thisVar: _this_(rg_uielement_formschema), ucfgId: rg_uielement_formschema::getglobalerrors ) )
dim(0): M(value: "" )
array: Y(this.positions, object: D(thisVar: _this_(rg_uielement_formschema), ucfgId: rg_uielement_formschema::getglobalerrors ) )
dim(0): M(value: "" )
at 542:8 - 542:28
at 27:29 - 27:68 ) {
}], type: ) {
H(value: _SONAR_WILDCARD_146 ) = Flow([<history>: ] M(value: "" ) )
M(value: "name" ) = Flow([<history>: ] E(id: 2685, instruction: vcall %7 = get[NO-DEFAULT] (%5, "ssoSettings", %6)
at 27:29 - 27:68 ) {
M(value: "type" ) = Flow([<history>: ] E(id: 2685, instruction: vcall %7 = get[NO-DEFAULT] (%5, "ssoSettings", %6)
at 27:29 - 27:68 ) {
}' produced a symbol of a different class: 'K(TAINTSOURCES = [E(SANITIZED = [COMMAND_INJECTION, LDAP_INJECTION, PATH_INJECTION, XPATH_INJECTION, REGEX_DOS, SQL_INJECTION, REFLECTED_XSS, DESERIALIZATION, SERVER_SIDE_REQUEST, LOG_INJECTION, HTTP_REDIRECT, NOSQL_INJECTION, HEADER_INJECTION, CODE_INJECTION, INCLUDE_INJECTION, DOM_XSS, ARGUMENT_INJECTION, ZIP_SLIP, DOM_OPEN_REDIRECT, REFLECTION_INJECTION, SESSION_FIXATION, ARGUMENT_CONSTRUCTION, INTENT_REDIRECTION, THREAD_SUSPENSION_DOS, JSON_OPERATIONS, XML_OPERATIONS], id: 2685, instruction: vcall %7 = get[NO-DEFAULT] (%5, "ssoSettings", %6)
at 27:29 - 27:68 ) {
}], SANITIZED = [COMMAND_INJECTION, LDAP_INJECTION, PATH_INJECTION, XPATH_INJECTION, REGEX_DOS, SQL_INJECTION, REFLECTED_XSS, DESERIALIZATION, SERVER_SIDE_REQUEST, LOG_INJECTION, HTTP_REDIRECT, NOSQL_INJECTION, HEADER_INJECTION, CODE_INJECTION, INCLUDE_INJECTION, DOM_XSS, ARGUMENT_INJECTION, ZIP_SLIP, DOM_OPEN_REDIRECT, REFLECTION_INJECTION, SESSION_FIXATION, ARGUMENT_CONSTRUCTION, INTENT_REDIRECTION, THREAD_SUSPENSION_DOS, JSON_OPERATIONS, XML_OPERATIONS], type: ) {
H(value: _SONAR_WILDCARD_146 ) = Flow([<history>: ] M(value: "" ) )
M(value: "name" ) = Flow([<history>: ] E(SANITIZED = [COMMAND_INJECTION, LDAP_INJECTION, PATH_INJECTION, XPATH_INJECTION, REGEX_DOS, SQL_INJECTION, REFLECTED_XSS, DESERIALIZATION, SERVER_SIDE_REQUEST, LOG_INJECTION, HTTP_REDIRECT, NOSQL_INJECTION, HEADER_INJECTION, CODE_INJECTION, INCLUDE_INJECTION, DOM_XSS, ARGUMENT_INJECTION, ZIP_SLIP, DOM_OPEN_REDIRECT, REFLECTION_INJECTION, SESSION_FIXATION, ARGUMENT_CONSTRUCTION, INTENT_REDIRECTION, THREAD_SUSPENSION_DOS, JSON_OPERATIONS, XML_OPERATIONS], id: 2685, instruction: vcall %7 = get[NO-DEFAULT] (%5, "ssoSettings", %6)
at 27:29 - 27:68 ) {
M(value: "type" ) = Flow([<history>: ] E(SANITIZED = [COMMAND_INJECTION, LDAP_INJECTION, PATH_INJECTION, XPATH_INJECTION, REGEX_DOS, SQL_INJECTION, REFLECTED_XSS, DESERIALIZATION, SERVER_SIDE_REQUEST, LOG_INJECTION, HTTP_REDIRECT, NOSQL_INJECTION, HEADER_INJECTION, CODE_INJECTION, INCLUDE_INJECTION, DOM_XSS, ARGUMENT_INJECTION, ZIP_SLIP, DOM_OPEN_REDIRECT, REFLECTION_INJECTION, SESSION_FIXATION, ARGUMENT_CONSTRUCTION, INTENT_REDIRECTION, THREAD_SUSPENSION_DOS, JSON_OPERATIONS, XML_OPERATIONS], id: 2685, instruction: vcall %7 = get[NO-DEFAULT] (%5, "ssoSettings", %6)
at 27:29 - 27:68 ) {
}'. Attempting to continue with the original symbol.
at 213:19 - 213:48
at 213:19 - 213:48
at 213:19 - 213:48
at 213:19 - 213:48
09:43:10.049 INFO: Taint analysis: done.
09:43:10.049 INFO: Sensor PhpSecuritySensor [security] (done) | time=330857ms
09:43:10.052 INFO: Sensor PythonSecuritySensor [security]
09:43:10.058 INFO: Read 0 type definitions
09:43:10.058 INFO: No UCFGs have been included for analysis.
09:43:10.058 INFO: Sensor PythonSecuritySensor [security] (done) | time=6ms
09:43:10.058 INFO: Sensor JsSecuritySensor [security]
09:43:10.062 INFO: Read 0 type definitions
09:43:10.170 INFO: 09:43:10.17064 Building Runtime Type propagation graph
09:43:10.174 INFO: 09:43:10.174876 Running Tarjan on 465 nodes
09:43:10.175 INFO: 09:43:10.175352 Tarjan found 465 components
09:43:10.176 INFO: 09:43:10.176119 Variable type analysis: done
09:43:10.176 INFO: 09:43:10.176258 Building Runtime Type propagation graph
09:43:10.179 INFO: 09:43:10.179442 Running Tarjan on 465 nodes
09:43:10.179 INFO: 09:43:10.179762 Tarjan found 465 components
09:43:10.180 INFO: 09:43:10.180107 Variable type analysis: done
09:43:10.180 INFO: Analyzing 107 ucfgs to detect vulnerabilities.
09:43:10.230 INFO: Taint analysis starting. Entrypoints: 34
09:43:10.230 INFO: Running symbolic analysis for 'JS'
09:43:10.521 INFO: Taint analysis: done.
09:43:10.521 INFO: Sensor JsSecuritySensor [security] (done) | time=463ms
09:43:10.530 INFO: ------------- Run sensors on project
09:43:10.614 INFO: Sensor Analysis Warnings import [csharp]
09:43:10.615 INFO: Sensor Analysis Warnings import [csharp] (done) | time=1ms
09:43:10.616 INFO: Sensor Zero Coverage Sensor
09:43:13.452 INFO: Sensor Zero Coverage Sensor (done) | time=2836ms
09:43:13.468 INFO: SCM Publisher SCM provider for this project is: git
09:43:13.473 INFO: SCM Publisher 392 source files to be analyzed
09:43:14.211 INFO: Blaming files using native implementation
09:43:23.476 INFO: 144/392 source files have been analyzed
09:43:33.476 INFO: 285/392 source files have been analyzed
09:43:39.724 INFO: Blaming files using native implementation (done) | time=25513ms
09:43:39.731 INFO: SCM Publisher 392/392 source files have been analyzed (done) | time=26257ms
09:43:40.755 INFO: CPD Executor 3377 files had no CPD blocks
09:43:40.756 INFO: CPD Executor Calculating CPD for 13356 files
09:43:49.030 INFO: CPD Executor CPD calculation finished (done) | time=8274ms
09:44:00.145 INFO: SCM writing changed lines
09:44:04.358 INFO: SCM writing changed lines (done) | time=4213ms
09:44:06.332 INFO: Analysis report generated in 15558ms, dir size=24 MB
09:44:44.709 INFO: Analysis report compressed in 38377ms, zip size=18 MB
09:44:51.605 INFO: Analysis report uploaded in 6894ms
09:44:51.608 INFO: ANALYSIS SUCCESSFUL, you can find the results at: https://sonarcloud.io/dashboard?id=xxxxx&pullRequest=30768
09:44:51.608 INFO: Note that you will be able to access the updated dashboard once the server has processed the submitted analysis report
09:44:51.608 INFO: More about the report processing at https://sonarcloud.io/api/ce/task?id=xxxx
09:45:05.479 INFO: Analysis total time: 27:55.338 s
09:45:05.508 INFO: ------------------------------------------------------------------------
09:45:05.508 INFO: EXECUTION SUCCESS
09:45:05.508 INFO: ------------------------------------------------------------------------
09:45:05.508 INFO: Total time: 27:58.391s
09:45:06.096 INFO: Final Memory: 142M/484M
09:45:06.096 INFO: ------------------------------------------------------------------------
In this case again i can see a lot more files being shown as changed in the sonar cloud UI than realistically changed.
Hopefully that is enough information, let me know if you need anything else.
Thanks for the logs. I’m a bit over my head at this point, so I’ve pinged the experts. Hopefully, they’ll be along soon.
Thanks a lot for the detailed investigation in your post.
I’ve looked into your case.
If I understood correctly, you’d expect to have a single file analyzed by the scanner.
According to the logs, the scanner finds 392 files.
09:43:13.473 INFO: SCM Publisher 392 source files to be analyzed
Does this match what you see in the UI?
To investigate this difference, would you be able to run with debug on?
You should be able to do it by passing -Dsonar.verbose=true to the scanner.
Could you then look for a log starting with Merge base sha1: ?
This should give you the reference of the commit the scanner is comparing against the HEAD reference of the currently checked out branch.
If I understood correctly, you would expect this reference to be the HEAD of the development branch. Could you confirm this is the case?
thanks
Yes, this is exaclty the problem we have.
The verbose log has too much information in it, some of it we consider confidential, thats why i havent shared it in full.
&1|16:15:33.309 16:15:33.309 DEBUG: Merge base sha1: e2b98ee336843df4c9041e60c0851110bf20923b
The commit that is on the hotfix branch is 65243a1ac084772bee9ffc14f15366dfb5f450e7 done on 2022-09-27 on a fresh clone of the repo.
The commit that sonar is referencing as HEAD of development branch (if i understand correctly) is e2b98ee336843df4c9041e60c0851110bf20923b done on 2022-09-09.
So i guess the question is why is sonar considering this (quite old commit) to be the HEAD? How does it figure this out?
The scanner checks the commits referred to by the following references:
refs/heads/development
refs/remotes/origin/development
refs/remotes/upstream/development
Could you check which commits these references point to and share whether they match HEAD of the development branch or the one used by the scanner?
According to the initial post in the thread, you are using GoCD, which is not a build system we commonly encounter and, as a consequence, specifically support.
We can’t exclude that the way the Git clone is performed does not create/update references the way we expect them to be.
Cheers,
Hey, I’ve outlined what is being run as commands in my post earlier.
Its basically a fresh clone each time (which by default checks out the destination branch). Then checkout to the source branch, then run everything else (again, its in my earlier post).
Ill have a look to see about the refs, but it doesnt make sense for them to not be uptodate considering its a fresh clone (not shallow) each time.
Is there a way to run a debug command alone to figure out which commit sonar will consider as head? Something like a dry-run i guess. Otherwise i have to wait for quite a bit for each test
Hristo Mitev:
Ill have a look to see about the refs, but it doesnt make sense for them to not be uptodate considering its a fresh clone (not shallow) each time.
I can’t offer an explanation. I don’t know your system.
Hristo Mitev:
Is there a way to run a debug command alone to figure out which commit sonar will consider as head? Something like a dry-run i guess. Otherwise i have to wait for quite a bit for each test
I don’t have such standalone program to provide. Running the scanner in debug is the most available.
I see you are able to modify your build script.
Maybe you could run the following Git commands right before running the scanner and share the output:
Thank you for the ideas, we will dig deeper on our end.
Can we just clarify that the merge base that sonar uses is taken from the sonar.pullrequest.base value, correct?
I.e. if i provide development, it will target development as a merge base, if i provide a different branch it will use that as a base?
Is there a way to ask sonar to use the remote instead of the local head?
And finally im guessing we can provide a commit hash too, instead of a named branch?
Hristo Mitev:
Can we just clarify that the merge base that sonar uses is taken from the sonar.pullrequest.base value, correct?
I.e. if i provide development, it will target development as a merge base, if i provide a different branch it will use that as a base?
When your SonarCloud project is bound to your Gitlab project (this is your case), the base branch is the Pull Request’s destination branch on BitbucketCloud.
Hristo Mitev:
Is there a way to ask sonar to use the remote instead of the local head?
Sonar takes the local head unless undefined, then the “remote” one, then the “upstream” one.
This is not configurable.
I suggest to first see what the references are before engaging into considering changes in Sonar.
cheers,
I had this issue as well. Thought I would reply here with my solution in case it helps anyone. What happens was that in the CI server machine in the git repo there was an old local branch of the same name as the branch provided to the sonar-scanner -Dsonar.pullrequest.base parameter. In my case it was a branch called “develop”. So even though at the time of the build the local “develop” was not checked out, per the comment above sonar-scanner gives priority first to checking refs/heads/develop to figure out the commit ID to use as the base reference. Once I deleted that local branch the scanner went back to using refs/remotes/origin/develop to get the commit id of “develop” and all was good.
Hristo Mitev: