Skip to content

Create new rule detect system() c - #21

Merged
DMarinhoCodacy merged 5 commits into
mainfrom
create-new-rule-detect-system()-c
May 26, 2026
Merged

DMarinhoCodacy merged 5 commits into
mainfrom
create-new-rule-detect-system()-c

Conversation

@DMarinhoCodacy

Copy link
Copy Markdown
Contributor

No description provided.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the CircleCI plugins-test orb version, lowers the confidence level of the standard system call detection rule in docs/codacy-rules.yaml, and shortens the expected error messages in results.xml. However, a critical issue was identified where the shortened error messages in results.xml no longer match the actual rule message defined in docs/codacy-rules.yaml. This mismatch will cause integration tests to fail unless the rule definition is also updated or the changes in results.xml are reverted.

Comment thread docs/multiple-tests/codacy-rules/results.xml
@codacy-production

Copy link
Copy Markdown

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.

Run reviewer

TIP This summary will be updated as you push new changes.

@codacy-production codacy-production Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

While the PR is technically up to Codacy standards, there are significant process and implementation concerns. The PR title claims to create a new rule, but the code actually modifies an existing security rule for system() calls. There is also a lack of documentation in the PR description explaining the rationale for these changes.

Crucially, although new detection patterns and exclusions (e.g., osutility::system()) were introduced to the configuration, no test files (C/C++ samples) were included to verify these patterns. This makes it impossible to confirm if the exclusion logic works as intended or if it creates false negatives. Furthermore, there is a technical disagreement between agents on whether lowering the confidence of a deterministic security risk like system() is appropriate; the consensus suggests that refining patterns is preferable to a global confidence downgrade.

About this PR

  • No test files (C/C++ samples) are included in the PR to verify that the patterns correctly identify target calls (global, std::, ::) and correctly respect the exclusion for osutility::system().
  • The PR description is empty, providing no context for the confidence downgrade or the orb upgrade. Additionally, the PR title 'Create new rule' contradicts the actual implementation, which modifies an existing rule.

Test suggestions

  • Detection of global system() calls
  • Detection of std::system() calls in C++
  • Detection of ::system() calls in C++
  • Exclusion of osutility::system() calls
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Detection of global system() calls
2. Detection of std::system() calls in C++
3. Detection of ::system() calls in C++
4. Exclusion of osutility::system() calls

TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback

Comment thread docs/codacy-rules.yaml
Ensure safe alternatives or proper input validation/sanitization is used.
impact: MEDIUM
confidence: HIGH No newline at end of file
confidence: LOW No newline at end of file

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 MEDIUM RISK

There are multiple concerns with this rule update:

  1. Identity Mismatch: The PR title suggests a new rule, but this modifies codacy.c.security.avoid-std-system. If a new rule was intended, use a unique ID; otherwise, update the title.
  2. Confidence Level: Lowering confidence to LOW for a deterministic security risk like system() is counter-intuitive. If the goal is to reduce noise from C++ member functions, it is better to refine the patterns while keeping the confidence HIGH.
  3. Documentation: If the change proceeds, consider updating the rule message to recommend safer alternatives like execve or posix_spawn to improve utility.

@DMarinhoCodacy
DMarinhoCodacy merged commit 22f515f into main May 26, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants