Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds first-pass CircleCI scanning support to Pipeleek and introduces a Docker Compose ELK example for shipping scan logs into Elasticsearch/Kibana.
Changes:
- Added a new
circle scancommand, CircleCI transport/helpers, normalization logic, and a standalonepipeleek-circlebinary. - Added unit and e2e tests for the new CircleCI scanner paths and discovery helpers.
- Added an
examples/compose-elk/stack plus related docs/config examples for centralized log ingestion and visualization.
Reviewed changes
Copilot reviewed 24 out of 25 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
tests/e2e/circle/scan/scan_test.go |
Adds CircleCI e2e coverage for direct project scans and org discovery. |
pkg/circle/scan/transport.go |
Implements CircleCI API transport methods and download helpers. |
pkg/circle/scan/transport_test.go |
Tests org/project discovery and v1/v2 fallback behavior. |
pkg/circle/scan/scanner.go |
Adds CircleCI scan orchestration, filtering, log/test/artifact scanning, and option init. |
pkg/circle/scan/scanner_test.go |
Covers normalization helpers, URL builders, and discovery hints. |
pkg/circle/scan/normalize.go |
Adds project/org slug normalization and discovery helper logic. |
pipeleek.example.yaml |
Documents CircleCI config options in the main sample config. |
Makefile |
Adds CircleCI build/test targets and cleanup entries. |
internal/cmd/root.go |
Registers the new CircleCI command group on the main CLI. |
internal/cmd/circle/scan/scan.go |
Defines the circle scan Cobra command and config binding. |
internal/cmd/circle/scan/scan_test.go |
Verifies Circle scan command flag registration. |
internal/cmd/circle/circle.go |
Adds the CircleCI root command. |
internal/cmd/circle/circle_test.go |
Tests CircleCI root command construction. |
go.sum |
Records checksums for the new CircleCI dependency set. |
go.mod |
Adds the CircleCI CLI dependency and related transitive updates. |
examples/compose-elk/start-elk.sh |
Adds a helper script to start ELK and bootstrap Kibana. |
examples/compose-elk/setup-kibana.sh |
Creates Kibana data views, saved searches, visualizations, and dashboard objects. |
examples/compose-elk/setup-elasticsearch.sh |
Installs an Elasticsearch index template for Pipeleek logs. |
examples/compose-elk/README.md |
Documents the ELK example workflow and dashboard usage. |
examples/compose-elk/pipeleek.shared.example.yaml |
Adds a shared multi-platform config example for the compose stack. |
examples/compose-elk/logstash/pipeline/logstash.conf |
Defines Logstash ingestion/parsing/mapping for Pipeleek JSON logs. |
examples/compose-elk/docker-compose.yml |
Adds the ELK stack and one-shot scan job services. |
docs/introduction/logging.md |
Links the new compose ELK example from logging docs. |
docs/introduction/configuration.md |
Adds CircleCI configuration documentation. |
cmd/pipeleek-circle/main.go |
Adds the standalone CircleCI-focused entrypoint binary. |
Comment on lines
+123
to
+125
| if hitTimeoutSeconds := config.GetInt("common.hit_timeout"); hitTimeoutSeconds > 0 { | ||
| options.HitTimeout = time.Duration(hitTimeoutSeconds) * time.Second | ||
| } |
Comment on lines
+581
to
+585
| v1Filter := orgName | ||
| if strings.HasPrefix(strings.ToLower(input.Organization), "circleci/") { | ||
| v1Filter = "" | ||
| } | ||
| fallbackProjects, fallbackErr := apiClient.ListAccessibleProjectsV1(context.Background(), input.VCS, v1Filter) |
Comment on lines
+160
to
+163
| if !matchesFilter(s.options.Statuses, pipeline.State) { | ||
| continue | ||
| } | ||
|
|
Comment on lines
+340
to
+341
| locationURL := circleAppWorkflowURL(workflow.ID) | ||
| jobURL := circleAppJobURL(project, pipeline.Number, workflow.ID, job.JobNumber, locationURL) |
Comment on lines
+484
to
+489
| content, err := s.options.APIClient.DownloadWithAuth(s.options.Context, artifact.URL) | ||
| if err != nil { | ||
| continue | ||
| } | ||
|
|
||
| if int64(len(content)) > s.options.MaxArtifactSize { |
| project: [my-org/my-repo] # circle scan --project (optional if org is set) | ||
| vcs: github # circle scan --vcs | ||
| org: my-org # circle scan --org (also enables org-wide discovery when project is omitted) | ||
| # --org accepts: my-org, github/my-org, or app URL forms like |
Comment on lines
+20
to
+21
| scan: | ||
| owner: my-github-org |
Comment on lines
+34
to
+35
| scan: | ||
| project: my-project |
Comment on lines
+40
to
+41
| scan: | ||
| owner: my-org |
Comment on lines
+27
to
+28
| scan: | ||
| workspace: my-workspace |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.