Skip to content

feat: enhance semantic search with multiple paths and pattern filtering#348

Merged
deepin-bot[bot] merged 1 commit into
linuxdeepin:masterfrom
Johnson-zs:master
Jul 14, 2026
Merged

feat: enhance semantic search with multiple paths and pattern filtering#348
deepin-bot[bot] merged 1 commit into
linuxdeepin:masterfrom
Johnson-zs:master

Conversation

@Johnson-zs

@Johnson-zs Johnson-zs commented Jul 14, 2026

Copy link
Copy Markdown
Contributor
  1. Added support for multiple custom paths in search rules by accepting
    QVariantList for custom_path field
  2. Implemented exclude_patterns to filter search results by file name
    patterns
  3. Added ruleMetadataById method to find rules without knowing their
    group
  4. Improved path expansion logic in ActionExtractor and
    LocationExtractor
  5. Enhanced semantic searcher to apply exclude_patterns when processing
    results

Log: Added support for multiple search paths and result filtering by
file patterns

Influence:

  1. Test search rules with multiple custom paths
  2. Verify exclude_patterns filters correct files from results
  3. Check backward compatibility with existing single-path rules
  4. Validate rule lookups work correctly with new ruleMetadataById
  5. Test performance impact with multiple paths and pattern filtering

feat: 增强语义搜索支持多路径和模式过滤

  1. 在搜索规则中添加对多个自定义路径的支持,支持custom_path字段使用
    QVariantList
  2. 实现exclude_patterns功能,按文件名模式过滤搜索结果
  3. 添加ruleMetadataById方法,无需知道规则组即可查找规则
  4. 改进ActionExtractor和LocationExtractor中的路径扩展逻辑
  5. 增强语义搜索器在处理结果时应用exclude_patterns

Log: 新增支持多个搜索路径及按文件模式过滤结果的功能

Influence:

  1. 测试使用多个自定义路径的搜索规则
  2. 验证exclude_patterns能正确过滤结果文件
  3. 检查与现有单一路径规则的向后兼容性
  4. 验证新的ruleMetadataById方法查找规则是否正确
  5. 测试多路径和模式过滤对性能的影响

Summary by Sourcery

Enhance semantic search to support multiple rule-defined search paths and filtering of results by file name patterns.

New Features:

  • Support multiple custom search paths per rule by allowing list-based custom_path metadata in action and location extraction.
  • Add exclude_patterns rule metadata to filter out search results matching specified file name glob patterns across engines.
  • Introduce ruleMetadataById to retrieve rule metadata by rule ID without requiring the rule group.

Enhancements:

  • Refactor semantic search engine completion and error handling into shared callbacks that apply result filtering and final aggregation.

1. Added support for multiple custom paths in search rules by accepting
QVariantList for custom_path field
2. Implemented exclude_patterns to filter search results by file name
patterns
3. Added ruleMetadataById method to find rules without knowing their
group
4. Improved path expansion logic in ActionExtractor and
LocationExtractor
5. Enhanced semantic searcher to apply exclude_patterns when processing
results

Log: Added support for multiple search paths and result filtering by
file patterns

Influence:
1. Test search rules with multiple custom paths
2. Verify exclude_patterns filters correct files from results
3. Check backward compatibility with existing single-path rules
4. Validate rule lookups work correctly with new ruleMetadataById
5. Test performance impact with multiple paths and pattern filtering

feat: 增强语义搜索支持多路径和模式过滤

1. 在搜索规则中添加对多个自定义路径的支持,支持custom_path字段使用
QVariantList
2. 实现exclude_patterns功能,按文件名模式过滤搜索结果
3. 添加ruleMetadataById方法,无需知道规则组即可查找规则
4. 改进ActionExtractor和LocationExtractor中的路径扩展逻辑
5. 增强语义搜索器在处理结果时应用exclude_patterns

Log: 新增支持多个搜索路径及按文件模式过滤结果的功能

Influence:
1. 测试使用多个自定义路径的搜索规则
2. 验证exclude_patterns能正确过滤结果文件
3. 检查与现有单一路径规则的向后兼容性
4. 验证新的ruleMetadataById方法查找规则是否正确
5. 测试多路径和模式过滤对性能的影响
@deepin-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: Johnson-zs

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@sourcery-ai

sourcery-ai Bot commented Jul 14, 2026

Copy link
Copy Markdown

Reviewer's Guide

Adds support for multiple custom paths in semantic search rules, introduces exclude_patterns-based filename filtering, factors engine completion/error handling into reusable helpers, and extends rule metadata lookup to work by rule ID across groups while updating extractors to handle QVariantList-based paths without breaking single-path behavior.

Sequence diagram for semantic search exclude_patterns filtering

sequenceDiagram
    actor User
    participant SemanticSearcher
    participant SemanticSearcherData
    participant SemanticRuleEngine
    participant SearchEngine

    User->>SemanticSearcher: search(naturalLanguage, dirs)
    SemanticSearcher->>SemanticSearcherData: doSearch(naturalLanguage, dirs)
    SemanticSearcherData->>SemanticSearcherData: intent.consumedSpans()
    loop for each MatchSpan
        SemanticSearcherData->>SemanticRuleEngine: ruleMetadataById(ruleId)
        SemanticRuleEngine-->>SemanticSearcherData: metadata
        SemanticSearcherData->>SemanticSearcherData: build excludeNamePatterns
    end
    SemanticSearcherData->>SearchEngine: createAndLaunchEngine(query, onEngineFinished, onEngineError)
    SearchEngine-->>SemanticSearcherData: searchFinished(results)
    SemanticSearcherData->>SemanticSearcherData: onEngineFinished(results)
    SemanticSearcherData->>SemanticSearcherData: filter by excludeNamePatterns
    SemanticSearcherData->>SemanticSearcherData: deduplicate by seenPaths
    SemanticSearcherData->>SemanticSearcherData: pendingFinishCount.fetch_sub(1)
    alt all engines finished
        SemanticSearcherData->>SemanticSearcher: searchFinished(allResults)
    end
    SearchEngine-->>SemanticSearcherData: errorOccurred(error)
    SemanticSearcherData->>SemanticSearcherData: onEngineError(error)
    SemanticSearcherData->>SemanticSearcherData: onEngineFinished({})
Loading

File-Level Changes

Change Details Files
Apply exclude_patterns from rule metadata to filter semantic search results by filename and refactor engine completion handling.
  • Collect exclude_patterns from consumed rule spans via ruleMetadataById before launching engines and store them in SemanticSearcherData::excludeNamePatterns.
  • Introduce SemanticSearcherData::onEngineFinished to centralize result aggregation, filename pattern filtering via QDir::match, path deduplication, maxResults truncation, and final status/signals.
  • Introduce SemanticSearcherData::onEngineError to log SearchError and delegate to onEngineFinished with an empty result list, preserving the previous error-as-finish behavior.
  • Wire engine signal handlers in doSearch to onEngineFinished/onEngineError instead of inline lambdas, keeping overall search lifecycle unchanged.
src/dfm-search/dfm-search-lib/semantic/semanticsearcher.cpp
src/dfm-search/dfm-search-lib/semantic/semanticsearcher_p.h
Support multiple custom_path entries (QVariantList) in semantic location-related rules while preserving single-string behavior.
  • In ActionExtractor::resolveImReceivedPaths, treat metadata["custom_path"] as either a QVariantList of relative paths or a single path, expanding each via LocationExtractor::expandCustomPath and deduplicating absolute results.
  • In LocationExtractor::extract, interpret metadata["custom_path"] similarly as a QVariantList or string, expanding each relative path under QDir::homePath, deduplicating, and falling back to xdg_type resolution when no custom path is provided.
  • Include QVariant headers where needed so QVariantList-based logic compiles.
src/dfm-search/dfm-search-lib/semantic/extractors/actionextractor.cpp
src/dfm-search/dfm-search-lib/semantic/extractors/locationextractor.cpp
Expose rule metadata lookup by rule ID across all semantic rule groups.
  • Add SemanticRuleEngine::ruleMetadataById, which linearly searches all RuleGroup entries and returns the first matching rule's metadata.
  • Declare the new API in semanticruleengine.h with documentation explaining its use when only a ruleId is available from consumedSpans.
src/dfm-search/dfm-search-lib/semantic/semanticruleengine.cpp
src/dfm-search/dfm-search-lib/semantic/semanticruleengine.h

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@deepin-ci-robot

Copy link
Copy Markdown

deepin pr auto review

★ 总体评分:92分

■ 【总体评价】

代码实现了custom_path多路径支持与exclude_patterns过滤机制,重构回调逻辑为成员函数,整体质量良好
逻辑正确且无安全漏洞,因存在跨文件代码重复及QStringList线性查找的轻微性能隐患扣8分

■ 【详细分析】

  • 1.语法逻辑(基本正确)✓

ActionExtractor::resolveImReceivedPathsLocationExtractor::extract中通过customPathVar.userType() == QMetaType::QVariantList判断类型,兼容字符串与列表两种形式,向后兼容性良好。SemanticSearcherData::onEngineFinishedstd::any_of配合QDir::match实现glob过滤,逻辑正确。onEngineError调用onEngineFinished({})以空结果触发计数递减,保持了原有的错误兜底语义。ruleMetadataById遍历所有group查找ruleId,返回逻辑正确。
潜在问题:onEngineFinished中并发访问seenPathsallResults未加锁保护,若多个引擎从不同线程以DirectConnection方式回调,存在数据竞争风险;excludeNamePatternsdoSearch中填充后于回调中读取,依赖Qt信号默认AutoConnection的线程亲和性保证安全,但若调用方手动指定DirectConnection则存在竞态
建议:若引擎可能跨线程回调,考虑在onEngineFinished中对共享数据加QMutex保护;或在文档中明确约束引擎回调必须通过队列连接

  • 2.代码质量(良好)✓

将lambda回调重构为onEngineFinishedonEngineError成员函数,提升了可读性与可维护性。ruleMetadataById的Doxygen注释清晰说明了与ruleMetadata的区别及使用场景。变量命名如customPathVarexcludeNamePatterns语义明确。JSON规则文件中exclude_patterns配置结构合理。
潜在问题:actionextractor.cppresolveImReceivedPathslocationextractor.cppextract中处理customPathVar为QVariantList的代码块几乎完全相同(约10行),存在明显代码重复
建议:将custom_path列表展开逻辑抽取为LocationExtractor的静态方法如expandCustomPathList(const QVariant &customPathVar),供两处调用复用

  • 3.代码性能(无性能问题)✓

onEngineFinished中对每个结果执行excludeNamePatternsstd::any_of匹配,复杂度为O(n×m)(n为结果数,m为模式数),对于典型场景(模式数<10,结果数<1000)性能可接受。excludeNamePatterns收集阶段使用contains去重,列表规模极小无影响。ruleMetadataById线性遍历所有group和rule,但仅在搜索初始化阶段调用一次,且规则数量有限。
建议:若未来exclude_patterns数量增长,可考虑预编译为QRegularExpression集合以加速匹配;对seenPaths可考虑使用QSet替代QStringList::contains以获得O(1)查找

  • 4.代码安全(存在0个安全漏洞)✓

漏洞对比统计:新增漏洞 0 个,减少漏洞 0 个,持平 0 个
本次改动涉及的custom_pathexclude_patterns均来源于应用内置的JSON规则配置文件(location_rules.json),属于可信数据源,不存在用户可控输入注入风险。QDir::match使用glob模式匹配而非正则表达式,不存在ReDoS风险。expandCustomPathQDir::homePath()为基路径展开相对路径,不存在路径遍历问题。无命令执行、SQL查询、反序列化等高危操作。

  • 建议:无需安全修复;建议在JSON规则文件加载阶段增加schema校验,防止配置错误导致意外行为

■ 【改进建议代码示例】

// locationextractor.h - 新增静态工具方法
static QStringList expandCustomPathList(const QVariant &customPathVar);

// locationextractor.cpp - 抽取公共逻辑
QStringList LocationExtractor::expandCustomPathList(const QVariant &customPathVar)
{
    QStringList resolved;
    if (customPathVar.userType() == QMetaType::QVariantList) {
        for (const QVariant &pathVar : customPathVar.toList()) {
            const QString relPath = pathVar.toString();
            if (relPath.isEmpty()) continue;
            const QStringList expanded = expandCustomPath(QDir::homePath(), relPath);
            for (const QString &p : expanded) {
                if (!resolved.contains(p)) resolved.append(p);
            }
        }
    } else {
        const QString customSubdir = customPathVar.toString();
        if (!customSubdir.isEmpty()) {
            resolved = expandCustomPath(QDir::homePath(), customSubdir);
        }
    }
    return resolved;
}

// actionextractor.cpp - 调用公共方法
QStringList ActionExtractor::resolveImReceivedPaths() const
{
    // ... 前置逻辑不变 ...
    const QVariant customPathVar = meta.value("custom_path");
    QStringList resolved = LocationExtractor::expandCustomPathList(customPathVar);
    
    if (resolved.isEmpty() && !xdgType.isEmpty()) {
        const QString path = LocationExtractor::resolveXdgPath(xdgType);
        if (!path.isEmpty()) {
            resolved = { path };
        }
    }
    // ... 后续逻辑不变 ...
}

// locationextractor.cpp - 同样调用公共方法
void LocationExtractor::extract(const QString &input, ParsedIntent &intent)
{
    // ... 前置逻辑不变 ...
    const QVariant customPathVar = metadata.value("custom_path");
    QStringList resolvedPaths = expandCustomPathList(customPathVar);
    
    if (resolvedPaths.isEmpty()) {
        const QString path = resolveXdgPath(xdgType);
        if (!path.isEmpty()) {
            resolvedPaths = { path };
        }
    }
    // ... 后续逻辑不变 ...
}

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • The logic for handling custom_path as either a single string or QVariantList is duplicated between ActionExtractor::resolveImReceivedPaths and LocationExtractor::extract; consider extracting a shared helper to keep behavior aligned and reduce future maintenance overhead.
  • SemanticRuleEngine::ruleMetadataById performs a linear scan over all groups and rules for each lookup; if this method is called frequently (e.g., once per consumed span), consider caching a ruleId→metadata (or ruleId→group) map during initialization to avoid repeated O(n) traversals.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- The logic for handling `custom_path` as either a single string or `QVariantList` is duplicated between `ActionExtractor::resolveImReceivedPaths` and `LocationExtractor::extract`; consider extracting a shared helper to keep behavior aligned and reduce future maintenance overhead.
- `SemanticRuleEngine::ruleMetadataById` performs a linear scan over all groups and rules for each lookup; if this method is called frequently (e.g., once per consumed span), consider caching a ruleId→metadata (or ruleId→group) map during initialization to avoid repeated O(n) traversals.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@Johnson-zs

Copy link
Copy Markdown
Contributor Author

/forcemerge

@deepin-bot

deepin-bot Bot commented Jul 14, 2026

Copy link
Copy Markdown

This pr force merged! (status: blocked)

@deepin-bot
deepin-bot Bot merged commit 6b50295 into linuxdeepin:master Jul 14, 2026
23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants