Code Quality Testing code-quality-testing
Learn how code quality testing of pipelines works and how it can improve the quality of your deployments.
Introduction introduction
Code quality testing evaluates your application code based on a set of quality rules. It is the primary purpose of a code-quality only pipeline and is executed immediately following the build step in all production and non-production pipelines.
See Configuring Your CI-CD Pipeline to learn more about different types of pipelines.
Code Quality Rules understanding-code-quality-rules
Code quality testing scans the source code to ensure that it meets certain quality criteria. This is implemented by a combination of SonarQube and content package-level examination using OakPAL. There are over 100 rules, combining generic Java rules and AEM-specific rules. Some of the AEM-specific rules are created based on best practices from AEM Engineering and are referred to as custom code quality rules.
Three-Tiered Ratings three-tiered-gate
Issues identified by code quality testing are assigned to one of three categories.
-
Critical - These are issues which cause an immediate failure of the pipeline.
-
Important - These are issues which cause the pipeline to enter a paused state. A deployment manager, project manager, or business owner can either override the issues, in which case the pipeline proceeds, or they can accept the issues, in which case the pipeline stops with a failure.
-
Info - These are issues which are provided purely for informational purposes and have no impact on the pipeline execution
Ratings ratings
The results of this step is delivered as Ratings.
The following table summarizes the ratings and failure thresholds for each of the critical, important and information categories.
B = At least 1 minor vulnerability
C = At least 1 major vulnerability
D = At least 1 critical vulnerability
E = At least 1 blocker vulnerability
B = At least 1 minor bug
C = At least 1 major bug
D = At least 1 critical bug
E = At least 1 blocker bug
Defined by the outstanding remediation cost for code smells as a percentage of the time that has already gone into the application
- A = <=5%
- B = 6-10%
- C = 11-20%
- D = 21-50%
- E = >50%
Defined by a mix of unit test line coverage and condition coverage using the formula:Coverage = (CT + CF + LC)/(2*B + EL)
CT
= Conditions that have been evaluated astrue
at least once while running unit testsCF
= Conditions that have been evaluated asfalse
at least once while running unit testsLC
= Covered lines = lines_to_cover - uncovered_linesB
= total number of conditionsEL
= total number of executable lines (lines_to_cover)
Defined as the number of lines involved in duplicated blocks. A block of code is considered duplicated under the following conditions.
Non-Java Projects:
- There should be at least 100 successive and duplicated tokens.
- Those tokens should be spread over at least:
- 30 lines of code for COBOL
- 20 lines of code for ABAP
- 10 lines of code for other languages
Java Projects:
- There should be at least 10 successive and duplicated statements regardless of the number of tokens and lines.
Differences in indentation and in string literals are ignored when detecting duplicates.
Dealing with False Positives dealing-with-false-positives
The quality scanning process is not perfect and will sometimes incorrectly identify issues which are not actually problematic. This is referred to as a false positive.
In these cases, the source code can be annotated with the standard Java @SuppressWarnings
annotation specifying the rule ID as the annotation attribute. For example, one common false positive is that the SonarQube rule to detect hardcoded passwords can be aggressive about how a hardcoded password is identified.
The following code is fairly common in an AEM project, which has code to connect to some external service.
@Property(label = "Service Password")
private static final String PROP_SERVICE_PASSWORD = "password";
SonarQube will then raise a blocker vulnerability. But after reviewing the code, you recognize that this is not a vulnerability and can annotate the code with the appropriate rule ID.
@SuppressWarnings("squid:S2068")
@Property(label = "Service Password")
private static final String PROP_SERVICE_PASSWORD = "password";
However, if the code was actually this:
@Property(label = "Service Password", value = "mysecretpassword")
private static final String PROP_SERVICE_PASSWORD = "password";
Then the correct solution is to remove the hardcoded password.
@SuppressWarnings
annotation as specific as possible, that is, annotate only the specific statement or block causing the issue, it is possible to annotate at a class level.Content Package Scanning Optimization content-package-scanning-optimization
As part of the quality analysis process, Cloud Manager performs analysis of the content packages produced by the Maven build. Cloud Manager offers optimizations to accelerate this process, which are effective when certain packaging constraints are observed. Most significant is the optimization performed for projects that output a single content package, generally referred to as an “all” package, which contains several other content packages produced by the build, which are marked as skipped. When Cloud Manager detects this scenario, rather than unpack the “all” package, the individual content packages are scanned directly and sorted based on dependencies. For example, consider the following build output.
all/myco-all-1.0.0-SNAPSHOT.zip
(content-package)ui.apps/myco-ui.apps-1.0.0-SNAPSHOT.zip
(skipped-content-package)ui.content/myco-ui.content-1.0.0-SNAPSHOT.zip
(skipped-content-package)
If the only items inside myco-all-1.0.0-SNAPSHOT.zip
are the two skipped content packages, then the two embedded packages are scanned in lieu of the “all” content package.
For projects that produce dozens of embedded packages, this optimization has been shown to save upwards of 10 minutes per pipeline execution.
A special case can occur when the “all” content package contains a combination of skipped content packages and OSGi bundles. For example, if myco-all-1.0.0-SNAPSHOT.zip
contained the two embedded packages previously mentioned and one or more OSGi bundles, then a new, minimal content package is constructed with only the OSGi bundles. This package is always named cloudmanager-synthetic-jar-package
and the contained bundles are placed in /apps/cloudmanager-synthetic-installer/install
.
- This optimization does not impact the packages which are deployed to AEM.
- Because the matching between the embedded content packages and the skipped content packages is based on file names, this optimization cannot be performed if multiple skipped content packages have exactly the same file name or if the file name is changed while embedding.