diff --git a/CLAUDE.md b/CLAUDE.md index b15fd87..61982f6 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,6 +24,7 @@ go install ./cmd/builder ./builder auth github # Authenticate with GitHub (OAuth device flow) ./builder init # Set up workflow in current repo ./builder ios build # Trigger build and download IPA to ./dist/ +./builder ios build --profile production # Build with a builder.json profile ./builder dev flutter # Flutter hot reload with MobAI ./builder dev rn # React Native hot reload with MobAI ./builder dev kmp # Kotlin Multiplatform install + launch (no hot reload) @@ -134,6 +135,30 @@ internal/ submodule commit that only exists locally fails checkout on the runner. - **Run Correlation**: `run-name` carries the build ID so concurrent builds cannot adopt each other's runs +- **Build Profiles**: `profiles.` in `builder.json` overrides `ios.configuration`, `ios.scheme`, + `ios.signing` and `provider`, and adds `env` and the reserved `distribution`. `ios build` and + `ios share` take `--profile`; without it `defaultProfile` applies, and without that the top-level + settings are used unchanged. `config.ResolveProfile` does the merge, `Coordinator.settings` layers + `--unsigned`/`--provider` on top, and `Progress.Settings` prints the result before dispatch. + `Profile.Signing` is a `*bool` so a profile's `false` can override a top-level `true`; the jq in + `Resolve parameters` needs an explicit `!= null` test for the same reason, since `//` treats + `false` as missing. The runner receives env as one JSON object: the `profile` dispatch input + (`{"name","env","distribution"}`, one input to stay under the ten-input limit) on GitHub, and + `BUILD_ENV` plus `DISTRIBUTION` variables for `runner.sh`. Each entry is base64-encoded per + key and value on the runner (jq drops NUL bytes, and a key with a space must not split), the + `$GITHUB_ENV` heredoc uses a random delimiter so no value line can end it early, names are + checked against `^[A-Za-z_][A-Za-z0-9_]*$`, and `ResolveProfile` rejects the names the runners + own (`reservedEnv` and `reservedEnvPrefixes` in `internal/config/profile.go`: the runner + parameters, the signing secrets, `PATH`/`HOME`/`DEVELOPER_DIR`, and the `GITHUB_`, `RUNNER_`, + `CM_`, `BITRISE_`, `BUILDER_` namespaces; keep that list in step with what `runner.sh` and the + workflows read). `profile` is only sent when a profile is selected (`--profile` or + `defaultProfile`), because a workflow file from before profiles rejects a dispatch with an input + it does not declare; `triggerError` turns that 422 into a "run `builder init`" message. On + GitHub the profile's env lands in `$GITHUB_ENV`, and step-level `env:` (the signing secrets, the + build parameters) takes precedence over it. `distribution` reaches the runner as the + `steps.params.outputs.distribution` output on GitHub and the `DISTRIBUTION` variable for + `runner.sh`; the export step is meant to consume it under those names. + `env` is build-time configuration, not secrets: it sits in `builder.json` and in the run's inputs - **Flutter Detection**: Auto-detects Flutter projects, runs `flutter pub get`, uses `Runner` scheme - **DerivedData Caching**: `restore` keys on `github.run_id` and only the prefix in `restore-keys` ever hits, so every run must pair with a `cache/save` step or later builds stay cold. `ios-share` @@ -178,14 +203,28 @@ internal/ "project": "MyApp", "platform": "ios", "github": { "owner": "username", "repo": "my-ios-app" }, - "ios": { "path": "ios", "scheme": "" } + "ios": { "path": "ios", "scheme": "" }, + "defaultProfile": "development", + "profiles": { + "development": { "configuration": "Debug", "signing": false }, + "preview": { "configuration": "Release", "signing": true, "env": { "API_URL": "https://staging.example.com" } }, + "production": { "configuration": "Release", "signing": true, "scheme": "MyApp", "provider": "codemagic", "distribution": "app-store" } + } } ``` +`profiles` and `defaultProfile` are optional. A profile's fields are `configuration`, `scheme`, +`signing`, `provider`, `env` (string map) and `distribution` (`development`, `ad-hoc`, `app-store`, +`enterprise`; reserved for the export step, passed through but not applied yet). `runner` and +`submit` are planned for the same struct (`config.Profile`) but not read. + ## Workflow Features The embedded workflow template (`internal/workflow/templates/ios-build.yml`): -- Triggered via `workflow_dispatch` with `build_id`, `snapshot_ref`, `ios_path`, `scheme` +- Triggered via `workflow_dispatch` with `build_id`, `snapshot_ref`, `ios_path`, `scheme`, + `use_signing`, `configuration`, `flutter_version`, `jdk_version` and `profile` (nine of the ten + inputs GitHub allows; the last slot is meant for item 5's `build_number`, so add nothing else + without combining) - Dispatch runs the workflow from the **default branch**, so edits to the workflow file itself only take effect once pushed there — unlike app sources, which come from the snapshot ref - Checks out `snapshot_ref` over the default-branch checkout when set @@ -193,7 +232,9 @@ The embedded workflow template (`internal/workflow/templates/ios-build.yml`): workflow) for environments without GitHub API access. Push events run the workflow file from the tagged commit, `inputs` are empty, so a `Resolve parameters` step reads `ios_path`, `scheme`, `use_signing`, `configuration`, `flutter_version` and `jdk_version` from `builder.json` in the - tagged tree; every later step reads `steps.params.outputs.*`, never `inputs.*`. The job deletes + tagged tree, applying the profile named by `defaultProfile` (a tag cannot pick one per run); + every later step reads `steps.params.outputs.*`, never `inputs.*`. The same step exports the + profile's `env` to `$GITHUB_ENV` and outputs `profile` and `distribution`. The job deletes the tag when it ends (`permissions: contents: write`). Any other workflow in the repo with an unfiltered `on: push` also fires on these tags. - Runs on `macos-latest` diff --git a/README.md b/README.md index d99627b..f6d9b34 100644 --- a/README.md +++ b/README.md @@ -96,7 +96,9 @@ The run is named after the tag. Build settings come from `builder.json` in the tagged commit (`ios.path`, `ios.scheme`, `ios.signing`, `ios.configuration`, `flutter.version`, `kmp.jdkVersion`), the simulator stays available for the default 30 minutes, and the tag is deleted when the run ends. The IPA is -attached to the run as an artifact. +attached to the run as an artifact. A tag carries no flags, so a tag build +cannot pick a [profile](#build-profiles) per run; it applies the profile named +by `defaultProfile`, if there is one. ## Additional macOS Providers @@ -174,6 +176,7 @@ builder update # Update builder to the latest release builder ios build # Trigger build and download IPA to ./dist/ builder ios build --unsigned # Build without code signing (if signing is configured) builder ios build --provider codemagic # Build on another provider (also: bitrise) +builder ios build --profile production # Build with a profile from builder.json # Simulator (free, needs a MOBAI_API_KEY secret) builder ios share # Try the build on a simulator in the MobAI app @@ -243,6 +246,69 @@ builder signing setup # Upload code signing secrets to GitHub | `ios.signing` | Sign the IPA with the uploaded certificate and profile | `false` | | `ios.configuration` | Xcode build configuration. **Builds are `Debug` unless you set `Release`**; Debug is faster and is what the dev commands expect | `Debug` | +### Build Profiles + +Profiles are named sets of build settings, in the spirit of `eas.json`, selected +with `--profile` on `ios build` and `ios share`: + +```json +{ + "ios": { "path": "ios", "configuration": "Debug" }, + "defaultProfile": "development", + "profiles": { + "development": { "configuration": "Debug", "signing": false }, + "preview": { "configuration": "Release", "signing": true, + "env": { "API_URL": "https://staging.example.com" } }, + "production": { "configuration": "Release", "signing": true, "scheme": "MyApp", + "provider": "codemagic", "distribution": "app-store" } + } +} +``` + +```bash +builder ios build --profile preview +builder ios share --profile preview +``` + +| Field | Description | +|-------|-------------| +| `configuration` | Overrides `ios.configuration` | +| `scheme` | Overrides `ios.scheme` | +| `signing` | Overrides `ios.signing`; `false` in a profile turns signing off even when the top level has it on | +| `provider` | Overrides the top-level `provider` (`github`, `codemagic`, `bitrise`) | +| `env` | String map exported as environment variables on the runner before dependencies are installed and the app is built, so `pod install`, `npm install`, `flutter pub get`, Gradle and xcodebuild all see them | +| `distribution` | Reserved: one of `development`, `ad-hoc`, `app-store`, `enterprise`. Validated and passed to the runner; the export step does not act on it yet | + +How a build's settings are resolved: + +- Without `--profile`, the profile named by `defaultProfile` applies. With + neither, the top-level `ios.*` and `provider` settings are used exactly as + before, so existing projects are unaffected. +- A profile only overrides the fields it sets; everything else comes from the + top level. An unknown profile name is an error that lists the available ones. +- `--unsigned` and `--provider` on the command line override the profile. +- The resolved settings (profile, configuration, scheme, signing, provider, env + names) are printed before anything is dispatched. +- `ios share` only takes the profile's scheme, provider and env: simulator + builds are always Debug and unsigned. + +**`env` values are build-time configuration, not secrets.** They are stored in +`builder.json`, sent to the CI provider as plain workflow inputs, and visible in +the run's inputs and logs. Keep tokens and passwords in the provider's secrets +instead (`gh secret set` on GitHub, or the [Codemagic / Bitrise secrets +guide](docs/provider-secrets.md)); the build reads those as environment +variables too. Names the runner owns are rejected: its own parameters +(`SCHEME`, `CONFIGURATION`, `USE_SIGNING`, `BUILD_ENV`, ...), the signing +secrets, `PATH`, `HOME`, `DEVELOPER_DIR`, and anything starting with `GITHUB_`, +`RUNNER_`, `CM_`, `BITRISE_` or `BUILDER_`. + +Selecting a profile, with `--profile` or `defaultProfile`, needs the workflow +files from this version of Builder, which declare a `profile` input; an older +committed workflow rejects the dispatch. Run `builder init` again to refresh +`.github/workflows/ios-build.yml` and `ios-share.yml` (or `builder init +--provider ...` for `runner.sh`) in a project set up earlier, then commit and +push them to the default branch. + ### MobAI Configuration | Field | Description | Default | diff --git a/cmd/builder/root.go b/cmd/builder/root.go index cfda5b9..7a8ce53 100644 --- a/cmd/builder/root.go +++ b/cmd/builder/root.go @@ -471,7 +471,7 @@ func runInit(cmd *cobra.Command, args []string) error { if buildErr == nil { fmt.Println() - return runBuild(context.Background(), cfg, build.BuildOptions{ + return runBuild(context.Background(), cfg, &build.BuildOptions{ OutputDir: "dist", Timeout: 30 * time.Minute, Remote: remoteName, @@ -551,15 +551,31 @@ func init() { iosBuildCmd.Flags().Bool("unsigned", false, "Build unsigned IPA (skip code signing even if configured)") iosBuildCmd.Flags().StringP("remote", "r", "origin", "Git remote to push the working-tree snapshot to") iosBuildCmd.Flags().String("provider", "", "Override CI provider (default github or builder.json provider)") + iosBuildCmd.Flags().String("profile", "", "Build profile from builder.json (default: defaultProfile, else the top-level ios settings)") iosCmd.AddCommand(iosBuildCmd) // iOS share command flags iosShareCmd.Flags().Duration("duration", 30*time.Minute, "How long the simulator stays available while unused") iosShareCmd.Flags().StringP("remote", "r", "origin", "Git remote to push the working-tree snapshot to") iosShareCmd.Flags().String("provider", "", "Override CI provider (default github or builder.json provider)") + iosShareCmd.Flags().String("profile", "", "Build profile from builder.json; its scheme, provider and env apply to the simulator build") iosCmd.AddCommand(iosShareCmd) } +// effectiveProvider is the --provider flag, else the selected profile's +// provider, else builder.json's. The coordinator resolves the same chain; this +// exists so the GitHub client and signal handling agree with it. +func effectiveProvider(cfg *config.Config, profile, flag string) (string, error) { + if flag != "" { + return flag, nil + } + s, err := cfg.ResolveProfile(profile) + if err != nil { + return "", err + } + return s.Provider, nil +} + func runIOSBuild(cmd *cobra.Command, args []string) error { cfg, err := loadConfig() if err != nil { @@ -574,13 +590,18 @@ func runIOSBuild(cmd *cobra.Command, args []string) error { timeout, _ := cmd.Flags().GetDuration("timeout") unsigned, _ := cmd.Flags().GetBool("unsigned") remote, _ := cmd.Flags().GetString("remote") - provider, _ := cmd.Flags().GetString("provider") + providerFlag, _ := cmd.Flags().GetString("provider") + profile, _ := cmd.Flags().GetString("profile") ctx := cmd.Context() if ctx == nil { ctx = context.Background() } + provider, err := effectiveProvider(cfg, profile, providerFlag) + if err != nil { + return err + } name, err := cfg.ProviderName(provider) if err != nil { return err @@ -590,8 +611,9 @@ func runIOSBuild(cmd *cobra.Command, args []string) error { ctx, stop = signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM) defer stop() } - return runBuild(ctx, cfg, build.BuildOptions{ + return runBuild(ctx, cfg, &build.BuildOptions{ Provider: provider, + Profile: profile, OutputDir: outputDir, Timeout: timeout, Unsigned: unsigned, @@ -610,7 +632,8 @@ func runIOSShare(cmd *cobra.Command, args []string) error { duration, _ := cmd.Flags().GetDuration("duration") remote, _ := cmd.Flags().GetString("remote") - provider, _ := cmd.Flags().GetString("provider") + providerFlag, _ := cmd.Flags().GetString("provider") + profile, _ := cmd.Flags().GetString("profile") ctx := cmd.Context() if ctx == nil { @@ -622,12 +645,17 @@ func runIOSShare(cmd *cobra.Command, args []string) error { ctx, stop := signal.NotifyContext(ctx, os.Interrupt, syscall.SIGTERM) defer stop() + provider, err := effectiveProvider(cfg, profile, providerFlag) + if err != nil { + return err + } ghClient, err := clientForProvider(cfg, provider) if err != nil { return err } result, err := build.NewCoordinator(cfg, ghClient).Share(ctx, build.ShareOptions{ Provider: provider, + Profile: profile, Duration: duration, Remote: remote, }) @@ -652,7 +680,7 @@ func runIOSShare(cmd *cobra.Command, args []string) error { return nil } -func runBuild(ctx context.Context, cfg *config.Config, opts build.BuildOptions) error { +func runBuild(ctx context.Context, cfg *config.Config, opts *build.BuildOptions) error { ghClient, err := clientForProvider(cfg, opts.Provider) if err != nil { return err diff --git a/docs/providers.md b/docs/providers.md index 34673cf..9ae7c37 100644 --- a/docs/providers.md +++ b/docs/providers.md @@ -112,8 +112,10 @@ builder ios build --provider bitrise --unsigned builder ios build --provider github # explicit override ``` -Provider selection is: command flag, then `builder.json`'s `provider`, then -`github`. Adding or logging into a provider does not change the default. +Provider selection is: command flag, then the selected build profile's +`provider` (see the README's Build Profiles section), then `builder.json`'s +`provider`, then `github`. Adding or logging into a provider does not change +the default. To change it, edit `provider`, or pass `--set-default` when configuring a provider. ```json diff --git a/internal/build/coordinator.go b/internal/build/coordinator.go index 4dfc97f..24b8040 100644 --- a/internal/build/coordinator.go +++ b/internal/build/coordinator.go @@ -10,6 +10,7 @@ import ( "io" "os" "path/filepath" + "strings" "time" "github.com/MobAI-App/ios-builder/internal/ci" @@ -56,13 +57,87 @@ func NewCoordinatorWithOutput(cfg *config.Config, gh *github.Client, w io.Writer // BuildOptions contains options for a build type BuildOptions struct { - Provider string // Override the configured CI provider + Provider string // Override the configured CI provider (and the profile's) + Profile string // builder.json profile to build with; empty uses defaultProfile, else the top-level settings OutputDir string Timeout time.Duration Unsigned bool // Skip code signing even if configured Remote string // Git remote to push the working-tree snapshot to } +// settings applies the selected profile, then the command flags, over +// builder.json. The returned name is the provider that will run the job. +func (c *Coordinator) settings(profile, provider string, unsigned bool) (*config.BuildSettings, string, error) { + s, err := c.config.ResolveProfile(profile) + if err != nil { + return nil, "", err + } + if provider != "" { + s.Provider = provider + } + name, err := c.config.ProviderName(s.Provider) + if err != nil { + return nil, "", err + } + if unsigned { + s.Signing = false + } + return &s, name, nil +} + +// workflowInputs maps the settings onto the workflow_dispatch inputs both +// GitHub workflows share. Empty values are left out so the declared defaults +// apply, and `profile` is only sent when one is selected: a workflow file from +// before profiles rejects a dispatch carrying an input it does not declare. +func (c *Coordinator) workflowInputs(buildID, ref string, s *config.BuildSettings) map[string]string { + inputs := map[string]string{ + "build_id": buildID, + "snapshot_ref": ref, + } + if c.config.IOS.Path != "" { + inputs["ios_path"] = c.config.IOS.Path + } + if s.Scheme != "" { + inputs["scheme"] = s.Scheme + } + // Pass Flutter version if configured (ensures SDK version match for hot reload) + if c.config.Flutter.Version != "" { + inputs["flutter_version"] = c.config.Flutter.Version + } + // Pass JDK version for Kotlin Multiplatform Gradle builds + if c.config.KMP.JDKVersion != "" { + inputs["jdk_version"] = c.config.KMP.JDKVersion + } + if p := s.ProfileInput(); p != "" { + inputs["profile"] = p + } + return inputs +} + +// buildInputs are the ios-build.yml inputs: the shared ones plus signing and +// configuration, which the simulator workflow has no use for. +func (c *Coordinator) buildInputs(buildID, ref string, s *config.BuildSettings) map[string]string { + inputs := c.workflowInputs(buildID, ref, s) + if s.Signing { + inputs["use_signing"] = "true" + } + // Pass build configuration (Debug is faster, Release for production) + if s.Configuration != "" { + inputs["configuration"] = s.Configuration + } + return inputs +} + +// triggerError explains a rejected dispatch. GitHub answers 422 "Unexpected +// inputs provided" when the committed workflow file does not declare an input, +// which for `profile` means the file predates build profiles. +func triggerError(err error, inputs map[string]string, file string) error { + if _, ok := inputs["profile"]; ok && strings.Contains(err.Error(), "Unexpected inputs") { + return fmt.Errorf("failed to trigger workflow: the committed .github/workflows/%s does not declare the `profile` input; run `builder init` to refresh it, then commit and push the workflow to the default branch: %w", file, err) + } + return fmt.Errorf("failed to trigger workflow: %w", err) +} + // BuildResult contains the result of a build type BuildResult struct { BuildID string @@ -73,13 +148,16 @@ type BuildResult struct { } // Build triggers a remote build and downloads the IPA artifact -func (c *Coordinator) Build(ctx context.Context, opts BuildOptions) (*BuildResult, error) { - name, err := c.config.ProviderName(opts.Provider) +func (c *Coordinator) Build(ctx context.Context, opts *BuildOptions) (*BuildResult, error) { + // Defaults below are filled in on a copy: opts belongs to the caller. + o := *opts + opts = &o + settings, name, err := c.settings(opts.Profile, opts.Provider, opts.Unsigned) if err != nil { return nil, err } if name != "github" || c.provider != nil { - return c.buildRemote(ctx, opts) + return c.buildRemote(ctx, opts, settings) } if c.github == nil { return nil, fmt.Errorf("GitHub client is required") @@ -98,6 +176,7 @@ func (c *Coordinator) Build(ctx context.Context, opts BuildOptions) (*BuildResul // Generate build ID buildID := uuid.New().String()[:8] c.progress.Start(buildID) + c.progress.Settings(settings, name) // Step 1: Snapshot the working tree so the build matches what's on disk c.progress.Update(PhaseSnapshot, "Snapshotting working tree...") @@ -117,37 +196,11 @@ func (c *Coordinator) Build(ctx context.Context, opts BuildOptions) (*BuildResul // Step 2: Trigger workflow c.progress.Update(PhaseTriggering, "Triggering GitHub Actions build...") - inputs := map[string]string{ - "build_id": buildID, - "snapshot_ref": ref, - } - // Add iOS-specific inputs if configured - if c.config.IOS.Path != "" { - inputs["ios_path"] = c.config.IOS.Path - } - if c.config.IOS.Scheme != "" { - inputs["scheme"] = c.config.IOS.Scheme - } - // Determine signing: use signing if configured and not explicitly disabled - useSigning := c.config.IOS.Signing && !opts.Unsigned - if useSigning { - inputs["use_signing"] = "true" - } - // Pass build configuration (Debug is faster, Release for production) - if c.config.IOS.Configuration != "" { - inputs["configuration"] = c.config.IOS.Configuration - } - // Pass Flutter version if configured (ensures SDK version match for hot reload) - if c.config.Flutter.Version != "" { - inputs["flutter_version"] = c.config.Flutter.Version - } - // Pass JDK version for Kotlin Multiplatform Gradle builds - if c.config.KMP.JDKVersion != "" { - inputs["jdk_version"] = c.config.KMP.JDKVersion - } + inputs := c.buildInputs(buildID, ref, settings) if err := c.github.TriggerWorkflow(ctx, c.config.GitHub.Owner, c.config.GitHub.Repo, WorkflowFile, inputs); err != nil { + err = triggerError(err, inputs, WorkflowFile) c.progress.Error(PhaseTriggering, err) - return nil, fmt.Errorf("failed to trigger workflow: %w", err) + return nil, err } c.progress.Complete(PhaseTriggering, "Workflow triggered") diff --git a/internal/build/inputs_test.go b/internal/build/inputs_test.go new file mode 100644 index 0000000..5ec62c8 --- /dev/null +++ b/internal/build/inputs_test.go @@ -0,0 +1,160 @@ +package build + +import ( + "bytes" + "encoding/json" + "errors" + "io" + "reflect" + "strings" + "testing" + + "github.com/MobAI-App/ios-builder/internal/config" +) + +func profiledConfig() *config.Config { + signed := true + return &config.Config{ + Project: "App", + GitHub: config.GitHubConfig{Owner: "owner", Repo: "repo"}, + IOS: config.IOSConfig{Path: "ios", Scheme: "App", Configuration: "Debug"}, + Flutter: config.FlutterConfig{Version: "3.24.0"}, + Profiles: map[string]config.Profile{ + "preview": { + Configuration: "Release", Signing: &signed, Scheme: "AppPreview", Distribution: "ad-hoc", + Env: map[string]string{"API_URL": "https://staging.example.com", "FLAGS": "a b"}, + }, + "ci": {Provider: "codemagic"}, + }, + Codemagic: config.CIConfig{AppID: "app", Branch: "main"}, + } +} + +func TestSettingsPrecedence(t *testing.T) { + c := NewCoordinatorWithOutput(profiledConfig(), nil, io.Discard) + s, name, err := c.settings("", "", false) + if err != nil || name != "github" || s.Profile != "" || s.Configuration != "Debug" || s.Signing { + t.Fatalf("top-level settings: %+v %s %v", s, name, err) + } + s, name, err = c.settings("preview", "", false) + if err != nil || name != "github" || !s.Signing || s.Configuration != "Release" { + t.Fatalf("profile settings: %+v %s %v", s, name, err) + } + // --unsigned beats the profile's signing. + if s, _, err = c.settings("preview", "", true); err != nil || s.Signing { + t.Fatalf("--unsigned ignored: %+v %v", s, err) + } + // The profile's provider applies, and --provider beats it. + if _, name, err = c.settings("ci", "", false); err != nil || name != "codemagic" { + t.Fatalf("profile provider: %s %v", name, err) + } + if _, name, err = c.settings("ci", "bitrise", false); err != nil || name != "bitrise" { + t.Fatalf("--provider ignored: %s %v", name, err) + } + if _, _, err = c.settings("nope", "", false); err == nil || !strings.Contains(err.Error(), "ci, preview") { + t.Fatalf("unknown profile: %v", err) + } +} + +func TestGitHubInputsMapping(t *testing.T) { + c := NewCoordinatorWithOutput(profiledConfig(), nil, io.Discard) + + s, _, _ := c.settings("", "", false) + got := c.buildInputs("abcdef12", "refs/ios-builder/jobs/abcdef12", s) + want := map[string]string{ + "build_id": "abcdef12", "snapshot_ref": "refs/ios-builder/jobs/abcdef12", + "ios_path": "ios", "scheme": "App", "configuration": "Debug", "flutter_version": "3.24.0", + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("without a profile the inputs must be unchanged:\n got %v\nwant %v", got, want) + } + + s, _, _ = c.settings("preview", "", false) + got = c.buildInputs("abcdef12", "ref", s) + if got["scheme"] != "AppPreview" || got["configuration"] != "Release" || got["use_signing"] != "true" { + t.Fatalf("profile not mapped: %v", got) + } + var profile struct { + Name string + Env map[string]string + Distribution string + } + if err := json.Unmarshal([]byte(got["profile"]), &profile); err != nil { + t.Fatalf("profile input is not JSON: %q %v", got["profile"], err) + } + if profile.Name != "preview" || profile.Distribution != "ad-hoc" || profile.Env["FLAGS"] != "a b" || len(profile.Env) != 2 { + t.Fatalf("profile input: %+v", profile) + } + if len(got) > 10 { + t.Fatalf("workflow_dispatch allows at most 10 inputs, sending %d", len(got)) + } + + share := c.workflowInputs("abcdef12", "ref", s) + for _, k := range []string{"use_signing", "configuration"} { + if _, ok := share[k]; ok { + t.Fatalf("simulator workflow does not declare %s", k) + } + } + if share["profile"] == "" || share["scheme"] != "AppPreview" { + t.Fatalf("share inputs: %v", share) + } + + s, _, _ = c.settings("", "", false) + share = c.workflowInputs("abcdef12", "ref", s) + want = map[string]string{"build_id": "abcdef12", "snapshot_ref": "ref", "ios_path": "ios", "scheme": "App", "flutter_version": "3.24.0"} + if !reflect.DeepEqual(share, want) { + t.Fatalf("without a profile the share inputs must be unchanged:\n got %v\nwant %v", share, want) + } +} + +func TestTriggerErrorExplainsOldWorkflow(t *testing.T) { + rejected := errors.New(`failed to trigger workflow (status 422): {"message":"Unexpected inputs provided: [\"profile\"]"}`) + err := triggerError(rejected, map[string]string{"profile": "{}"}, WorkflowFile) + if !strings.Contains(err.Error(), "builder init") || !strings.Contains(err.Error(), WorkflowFile) || !errors.Is(err, rejected) { + t.Fatalf("old workflow not explained: %v", err) + } + // Without the profile input the message is GitHub's, unchanged. + if err := triggerError(rejected, map[string]string{}, WorkflowFile); strings.Contains(err.Error(), "builder init") || !errors.Is(err, rejected) { + t.Fatalf("unrelated rejection rewritten: %v", err) + } +} + +func TestRemoteInputsMapping(t *testing.T) { + c := NewCoordinatorWithOutput(profiledConfig(), nil, io.Discard) + + s, _, _ := c.settings("", "", false) + got := c.inputs("abcdef12", "ref", "sha", s) + want := map[string]string{ + "BUILD_ID": "abcdef12", "SNAPSHOT_REF": "ref", "SNAPSHOT_SHA": "sha", "IOS_PATH": "ios", "SCHEME": "App", + "CONFIGURATION": "Debug", "FLUTTER_VERSION": "3.24.0", "JDK_VERSION": "17", "USE_SIGNING": "false", + "BUILDER_REPOSITORY": "owner/repo", + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("without a profile the runner variables must be unchanged:\n got %v\nwant %v", got, want) + } + + s, _, _ = c.settings("preview", "", false) + got = c.inputs("abcdef12", "ref", "sha", s) + if got["USE_SIGNING"] != "true" || got["SCHEME"] != "AppPreview" || got["CONFIGURATION"] != "Release" || got["DISTRIBUTION"] != "ad-hoc" { + t.Fatalf("profile mapping: %v", got) + } + var env map[string]string + if err := json.Unmarshal([]byte(got["BUILD_ENV"]), &env); err != nil || env["API_URL"] != "https://staging.example.com" { + t.Fatalf("BUILD_ENV: %q %v", got["BUILD_ENV"], err) + } +} + +func TestSettingsPrinted(t *testing.T) { + var out bytes.Buffer + p := NewProgress(&out) + p.Start("abcdef12") + p.Settings(&config.BuildSettings{Profile: "preview", Configuration: "Release", Signing: true, Env: map[string]string{"B": "2", "A": "1"}, Distribution: "ad-hoc"}, "github") + for _, want := range []string{"Profile: preview", "Configuration: Release", "Scheme: (auto-detected)", "Signing: signed", "Provider: github", "Env: A, B", "Distribution: ad-hoc"} { + if !strings.Contains(out.String(), want) { + t.Errorf("missing %q in:\n%s", want, out.String()) + } + } + if strings.Contains(out.String(), "staging") || strings.Contains(out.String(), "=1") { + t.Fatal("env values should not be printed, only names") + } +} diff --git a/internal/build/progress.go b/internal/build/progress.go index 956eb77..e1c2083 100644 --- a/internal/build/progress.go +++ b/internal/build/progress.go @@ -3,9 +3,13 @@ package build import ( "fmt" "io" + "maps" + "slices" "strings" "sync" "time" + + "github.com/MobAI-App/ios-builder/internal/config" ) // Phase represents a build phase @@ -63,7 +67,38 @@ func (p *Progress) Start(buildID string) { fmt.Fprintf(p.writer, "\n") fmt.Fprintf(p.writer, "🏗️ Builder - Remote iOS Build\n") - fmt.Fprintf(p.writer, " Build ID: %s\n", buildID) + fmt.Fprintf(p.writer, " Build ID: %s\n", buildID) +} + +// Settings prints what the job will run with, before anything is dispatched, +// so a wrong profile or flag is visible without opening the provider's logs. +// It completes the header that Start begins. +func (p *Progress) Settings(s *config.BuildSettings, provider string) { + p.mu.Lock() + defer p.mu.Unlock() + + orDefault := func(v, d string) string { + if v == "" { + return d + } + return v + } + signing := "unsigned" + if s.Signing { + signing = "signed" + } + fmt.Fprintf(p.writer, " Profile: %s\n", orDefault(s.Profile, "(none)")) + fmt.Fprintf(p.writer, " Configuration: %s\n", orDefault(s.Configuration, "Debug")) + fmt.Fprintf(p.writer, " Scheme: %s\n", orDefault(s.Scheme, "(auto-detected)")) + fmt.Fprintf(p.writer, " Signing: %s\n", signing) + fmt.Fprintf(p.writer, " Provider: %s\n", provider) + if len(s.Env) > 0 { + keys := slices.Sorted(maps.Keys(s.Env)) + fmt.Fprintf(p.writer, " Env: %s\n", strings.Join(keys, ", ")) + } + if s.Distribution != "" { + fmt.Fprintf(p.writer, " Distribution: %s\n", s.Distribution) + } fmt.Fprintf(p.writer, "\n") } diff --git a/internal/build/remote.go b/internal/build/remote.go index 8d35313..f41b530 100644 --- a/internal/build/remote.go +++ b/internal/build/remote.go @@ -61,10 +61,14 @@ func (c *Coordinator) remote(override string) (ci.Provider, config.CIConfig, err return RemoteProvider(c.config, override) } -func (c *Coordinator) inputs(buildID, ref, sha string) map[string]string { +// inputs are the variables runner.sh reads on Codemagic and Bitrise. The +// profile's env travels as one JSON object in BUILD_ENV, which the runner +// exports before installing dependencies; DISTRIBUTION is passed through for +// the export step. Both are only set when the profile provides them. +func (c *Coordinator) inputs(buildID, ref, sha string, s *config.BuildSettings) map[string]string { v := map[string]string{"BUILD_ID": buildID, "SNAPSHOT_REF": ref, "SNAPSHOT_SHA": sha, - "IOS_PATH": c.config.IOS.Path, "SCHEME": c.config.IOS.Scheme, - "CONFIGURATION": c.config.IOS.Configuration, "FLUTTER_VERSION": c.config.Flutter.Version, + "IOS_PATH": c.config.IOS.Path, "SCHEME": s.Scheme, + "CONFIGURATION": s.Configuration, "FLUTTER_VERSION": c.config.Flutter.Version, "JDK_VERSION": c.config.KMP.JDKVersion, "USE_SIGNING": "false", "BUILDER_REPOSITORY": c.config.GitHub.Owner + "/" + c.config.GitHub.Repo} if v["IOS_PATH"] == "" { @@ -76,11 +80,21 @@ func (c *Coordinator) inputs(buildID, ref, sha string) map[string]string { if v["JDK_VERSION"] == "" { v["JDK_VERSION"] = "17" } + if s.Signing { + v["USE_SIGNING"] = "true" + } + if env := s.EnvJSON(); env != "" { + v["BUILD_ENV"] = env + } + if s.Distribution != "" { + v["DISTRIBUTION"] = s.Distribution + } return v } -func (c *Coordinator) pushSnapshot(ctx context.Context, remote, buildID string) (string, string, error) { +func (c *Coordinator) pushSnapshot(ctx context.Context, remote, buildID string, s *config.BuildSettings, provider string) (string, string, error) { c.progress.Start(buildID) + c.progress.Settings(s, provider) c.progress.Update(PhaseSnapshot, "Snapshotting working tree...") sha, err := snapshot.Create(ctx, fmt.Sprintf("ios-builder snapshot %s", buildID)) if err != nil { @@ -94,11 +108,14 @@ func (c *Coordinator) pushSnapshot(ctx context.Context, remote, buildID string) return ref, sha, nil } -func (c *Coordinator) buildRemote(ctx context.Context, opts BuildOptions) (*BuildResult, error) { - p, cfgCI, err := c.remote(opts.Provider) +func (c *Coordinator) buildRemote(ctx context.Context, opts *BuildOptions, s *config.BuildSettings) (*BuildResult, error) { + p, cfgCI, err := c.remote(s.Provider) if err != nil { return nil, err } + // Defaults below are filled in on a copy: opts belongs to the caller. + o := *opts + opts = &o if opts.Timeout < 0 { return nil, fmt.Errorf("timeout must be positive") } @@ -115,14 +132,11 @@ func (c *Coordinator) buildRemote(ctx context.Context, opts BuildOptions) (*Buil defer cancel() started := time.Now() buildID := uuid.New().String()[:8] - ref, sha, err := c.pushSnapshot(ctx, opts.Remote, buildID) + ref, sha, err := c.pushSnapshot(ctx, opts.Remote, buildID, s, p.Name()) if err != nil { return nil, err } - v := c.inputs(buildID, ref, sha) - if c.config.IOS.Signing && !opts.Unsigned { - v["USE_SIGNING"] = "true" - } + v := c.inputs(buildID, ref, sha, s) c.progress.Update(PhaseTriggering, "Triggering "+p.Name()+" build...") run, err := p.Start(ctx, ci.Request{Workflow: cfgCI.BuildWorkflow, Variables: v}) if err != nil { @@ -263,8 +277,8 @@ func saveRemoteIPA(ctx context.Context, p ci.Provider, run ci.Run, a ci.Artifact return dest, n, nil } -func (c *Coordinator) shareRemote(ctx context.Context, opts ShareOptions) (*ShareResult, error) { - p, cfgCI, err := c.remote(opts.Provider) +func (c *Coordinator) shareRemote(ctx context.Context, opts ShareOptions, s *config.BuildSettings) (*ShareResult, error) { + p, cfgCI, err := c.remote(s.Provider) if err != nil { return nil, err } @@ -286,11 +300,11 @@ func (c *Coordinator) shareRemote(ctx context.Context, opts ShareOptions) (*Shar ctx, cancel := context.WithTimeout(ctx, opts.Timeout) defer cancel() buildID := uuid.New().String()[:8] - ref, sha, err := c.pushSnapshot(ctx, opts.Remote, buildID) + ref, sha, err := c.pushSnapshot(ctx, opts.Remote, buildID, s, p.Name()) if err != nil { return nil, err } - v := c.inputs(buildID, ref, sha) + v := c.inputs(buildID, ref, sha, s) v["DURATION"] = opts.Duration.String() run, err := p.Start(ctx, ci.Request{Workflow: cfgCI.ShareWorkflow, Variables: v}) if err != nil { diff --git a/internal/build/remote_test.go b/internal/build/remote_test.go index e2de200..7cc359b 100644 --- a/internal/build/remote_test.go +++ b/internal/build/remote_test.go @@ -163,7 +163,7 @@ func TestRemoteSnapshotLifecycle(t *testing.T) { if strings.HasSuffix(tt.name, "share") { _, err = c.Share(context.Background(), ShareOptions{}) } else { - result, err = c.Build(context.Background(), BuildOptions{OutputDir: filepath.Join(dir, "dist")}) + result, err = c.Build(context.Background(), &BuildOptions{OutputDir: filepath.Join(dir, "dist")}) } if tt.name == "success" || tt.name == "transient poll recovers" { if err != nil || result == nil { diff --git a/internal/build/share.go b/internal/build/share.go index 9e931b6..662c404 100644 --- a/internal/build/share.go +++ b/internal/build/share.go @@ -16,7 +16,8 @@ const ShareWorkflowFile = "ios-share.yml" // ShareOptions configures a simulator session. type ShareOptions struct { - Provider string // Override the configured CI provider + Provider string // Override the configured CI provider (and the profile's) + Profile string // builder.json profile; only its scheme, provider and env apply to a simulator build // Duration is how long the simulator stays available while unused. Using // it keeps it open past this. Duration time.Duration @@ -44,12 +45,16 @@ const sharePublishGrace = 30 * time.Second // Share builds the working tree for the simulator and publishes it to the // account's MobAI app, then returns while the job outlives the command. func (c *Coordinator) Share(ctx context.Context, opts ShareOptions) (*ShareResult, error) { - name, err := c.config.ProviderName(opts.Provider) + settings, name, err := c.settings(opts.Profile, opts.Provider, true) if err != nil { return nil, err } + // Simulator builds are always Debug, never signed and never exported, + // whatever the profile says. + settings.Configuration = "Debug" + settings.Distribution = "" if name != "github" || c.provider != nil { - return c.shareRemote(ctx, opts) + return c.shareRemote(ctx, opts, settings) } if c.github == nil { return nil, fmt.Errorf("GitHub client is required") @@ -65,6 +70,7 @@ func (c *Coordinator) Share(ctx context.Context, opts ShareOptions) (*ShareResul buildID := uuid.New().String()[:8] c.progress.Start(buildID) + c.progress.Settings(settings, name) c.progress.Update(PhaseSnapshot, "Snapshotting working tree...") sha, err := snapshot.Create(ctx, fmt.Sprintf("ios-builder snapshot %s", buildID)) @@ -82,26 +88,12 @@ func (c *Coordinator) Share(ctx context.Context, opts ShareOptions) (*ShareResul c.progress.Complete(PhaseSnapshot, fmt.Sprintf("Pushed %s", sha[:7])) c.progress.Update(PhaseTriggering, "Starting the simulator session...") - inputs := map[string]string{ - "build_id": buildID, - "snapshot_ref": ref, - "duration": opts.Duration.String(), - } - if c.config.IOS.Path != "" { - inputs["ios_path"] = c.config.IOS.Path - } - if c.config.IOS.Scheme != "" { - inputs["scheme"] = c.config.IOS.Scheme - } - if c.config.Flutter.Version != "" { - inputs["flutter_version"] = c.config.Flutter.Version - } - if c.config.KMP.JDKVersion != "" { - inputs["jdk_version"] = c.config.KMP.JDKVersion - } + inputs := c.workflowInputs(buildID, ref, settings) + inputs["duration"] = opts.Duration.String() if err := c.github.TriggerWorkflow(ctx, c.config.GitHub.Owner, c.config.GitHub.Repo, ShareWorkflowFile, inputs); err != nil { + err = triggerError(err, inputs, ShareWorkflowFile) c.progress.Error(PhaseTriggering, err) - return nil, fmt.Errorf("failed to trigger workflow: %w", err) + return nil, err } c.progress.Complete(PhaseTriggering, "Session starting") diff --git a/internal/config/profile.go b/internal/config/profile.go new file mode 100644 index 0000000..c700c7c --- /dev/null +++ b/internal/config/profile.go @@ -0,0 +1,155 @@ +package config + +import ( + "encoding/json" + "fmt" + "regexp" + "slices" + "sort" + "strings" +) + +// BuildSettings is what a build runs with once a profile has been applied over +// the top-level settings. Command flags (--unsigned, --provider) are applied by +// the caller on top of this. +type BuildSettings struct { + Profile string // selected profile name, empty when none applies + Configuration string + Scheme string + Signing bool + Provider string // profile provider, else the top-level provider; may be empty (GitHub) + Env map[string]string + Distribution string +} + +// Distributions are the accepted values of a profile's distribution field. +var Distributions = []string{"development", "ad-hoc", "app-store", "enterprise"} + +// reservedEnv names the variables the runners read their parameters and +// secrets from, and the ones the shell and the CI services own. A profile that +// set one of these would silently change the build, or on runner.sh replace a +// provider secret, since the env is exported before the signing step reads it. +var reservedEnv = []string{ + "BUILD_ID", "SNAPSHOT_REF", "SNAPSHOT_SHA", "IOS_PATH", "SCHEME", "CONFIGURATION", + "USE_SIGNING", "FLUTTER_VERSION", "JDK_VERSION", "BUILD_ENV", "DISTRIBUTION", + "DURATION", "PROJECT_TYPE", "EXPORT_METHOD", + "IOS_CERTIFICATE", "IOS_CERTIFICATE_PASSWORD", "IOS_PROVISIONING_PROFILE", "MOBAI_API_KEY", + "PATH", "HOME", "USER", "SHELL", "TMPDIR", "DEVELOPER_DIR", "NODE_OPTIONS", +} + +// reservedEnvPrefixes cover the runners' own namespaces: Builder's, GitHub +// Actions' (GITHUB_*, RUNNER_*, ACTIONS_*), Codemagic's (CM_*, FCI_*) and +// Bitrise's. +var reservedEnvPrefixes = []string{"BUILDER_", "GITHUB_", "RUNNER_", "ACTIONS_", "CM_", "FCI_", "BITRISE_"} + +var envNameRe = regexp.MustCompile(`^[A-Za-z_][A-Za-z0-9_]*$`) + +func reservedEnvName(name string) bool { + if slices.Contains(reservedEnv, name) { + return true + } + for _, prefix := range reservedEnvPrefixes { + if strings.HasPrefix(name, prefix) { + return true + } + } + return false +} + +// ProfileNames lists the configured profiles, sorted. +func (c *Config) ProfileNames() []string { + names := make([]string, 0, len(c.Profiles)) + for n := range c.Profiles { + names = append(names, n) + } + sort.Strings(names) + return names +} + +// ResolveProfile applies the named profile, or defaultProfile when name is +// empty, over the top-level ios.* and provider settings. With neither, the +// result is the top-level settings unchanged, so projects without profiles +// build exactly as before. +func (c *Config) ResolveProfile(name string) (BuildSettings, error) { + s := BuildSettings{ + Configuration: c.IOS.Configuration, + Scheme: c.IOS.Scheme, + Signing: c.IOS.Signing, + Provider: c.Provider, + } + source := "profile" + if name == "" { + name, source = c.DefaultProfile, "defaultProfile" + } + if name == "" { + return s, nil + } + p, ok := c.Profiles[name] + if !ok { + if len(c.Profiles) == 0 { + return s, fmt.Errorf("%s %q is not defined; builder.json has no profiles", source, name) + } + return s, fmt.Errorf("%s %q is not defined; available profiles: %s", source, name, strings.Join(c.ProfileNames(), ", ")) + } + if p.Distribution != "" && !slices.Contains(Distributions, p.Distribution) { + return s, fmt.Errorf("profile %q: distribution %q must be one of %s", name, p.Distribution, strings.Join(Distributions, ", ")) + } + for k := range p.Env { + if !envNameRe.MatchString(k) { + return s, fmt.Errorf("profile %q: env name %q is not a valid environment variable name", name, k) + } + if reservedEnvName(k) { + return s, fmt.Errorf("profile %q: env name %q is reserved for the runner", name, k) + } + } + s.Profile = name + if p.Configuration != "" { + s.Configuration = p.Configuration + } + if p.Scheme != "" { + s.Scheme = p.Scheme + } + if p.Signing != nil { + s.Signing = *p.Signing + } + if p.Provider != "" { + s.Provider = p.Provider + } + if len(p.Env) > 0 { + s.Env = p.Env + } + s.Distribution = p.Distribution + return s, nil +} + +// EnvJSON encodes the profile's environment as a JSON object, which is how it +// travels to the runner: workflow inputs and CI variables are strings, and JSON +// survives values with spaces, quotes and newlines. Empty when there is none. +func (s *BuildSettings) EnvJSON() string { + if len(s.Env) == 0 { + return "" + } + data, _ := json.Marshal(s.Env) // a map[string]string cannot fail to marshal + return string(data) +} + +// ProfileInput encodes the parts of the profile that are not workflow inputs of +// their own (name, env, distribution) as the single `profile` dispatch input, +// keeping the workflow under GitHub's limit of ten inputs. Empty when no +// profile is selected, so older workflow files keep receiving the inputs they +// declare. +func (s *BuildSettings) ProfileInput() string { + if s.Profile == "" { + return "" + } + env := s.Env + if env == nil { + env = map[string]string{} + } + data, _ := json.Marshal(struct { + Name string `json:"name"` + Env map[string]string `json:"env"` + Distribution string `json:"distribution"` + }{s.Profile, env, s.Distribution}) + return string(data) +} diff --git a/internal/config/profile_test.go b/internal/config/profile_test.go new file mode 100644 index 0000000..4217437 --- /dev/null +++ b/internal/config/profile_test.go @@ -0,0 +1,135 @@ +package config + +import ( + "encoding/json" + "strings" + "testing" +) + +func boolPtr(b bool) *bool { return &b } + +func profileConfig() *Config { + return &Config{ + Provider: "github", + IOS: IOSConfig{Path: "ios", Scheme: "Top", Signing: true, Configuration: "Debug"}, + Profiles: map[string]Profile{ + "development": {Configuration: "Debug", Signing: boolPtr(false)}, + "preview": {Configuration: "Release", Env: map[string]string{"API_URL": "https://staging.example.com"}}, + "production": {Configuration: "Release", Scheme: "MyApp", Provider: "codemagic", Distribution: "app-store"}, + }, + } +} + +func TestResolveProfile(t *testing.T) { + cfg := profileConfig() + for _, tt := range []struct { + name, profile string + want BuildSettings + }{ + {"no profile keeps top-level settings", "", BuildSettings{Configuration: "Debug", Scheme: "Top", Signing: true, Provider: "github"}}, + {"false overrides true", "development", BuildSettings{Profile: "development", Configuration: "Debug", Scheme: "Top", Signing: false, Provider: "github"}}, + {"unset fields inherit", "preview", BuildSettings{Profile: "preview", Configuration: "Release", Scheme: "Top", Signing: true, Provider: "github", Env: map[string]string{"API_URL": "https://staging.example.com"}}}, + {"every field overrides", "production", BuildSettings{Profile: "production", Configuration: "Release", Scheme: "MyApp", Signing: true, Provider: "codemagic", Distribution: "app-store"}}, + } { + t.Run(tt.name, func(t *testing.T) { + got, err := cfg.ResolveProfile(tt.profile) + if err != nil { + t.Fatal(err) + } + if got.Profile != tt.want.Profile || got.Configuration != tt.want.Configuration || got.Scheme != tt.want.Scheme || + got.Signing != tt.want.Signing || got.Provider != tt.want.Provider || got.Distribution != tt.want.Distribution || + len(got.Env) != len(tt.want.Env) || got.Env["API_URL"] != tt.want.Env["API_URL"] { + t.Fatalf("got %+v, want %+v", got, tt.want) + } + }) + } +} + +func TestResolveProfileDefault(t *testing.T) { + cfg := profileConfig() + cfg.DefaultProfile = "preview" + s, err := cfg.ResolveProfile("") + if err != nil || s.Profile != "preview" || s.Configuration != "Release" { + t.Fatalf("default profile not applied: %+v %v", s, err) + } + // An explicit --profile beats defaultProfile. + s, err = cfg.ResolveProfile("production") + if err != nil || s.Profile != "production" { + t.Fatalf("explicit profile lost to default: %+v %v", s, err) + } + cfg.DefaultProfile = "nightly" + if _, err := cfg.ResolveProfile(""); err == nil || !strings.Contains(err.Error(), `defaultProfile "nightly"`) { + t.Fatalf("unknown defaultProfile accepted or not named as the source: %v", err) + } +} + +func TestResolveProfileErrors(t *testing.T) { + cfg := profileConfig() + _, err := cfg.ResolveProfile("staging") + if err == nil || !strings.Contains(err.Error(), "development, preview, production") { + t.Fatalf("unknown profile should list the available names: %v", err) + } + if _, err := (&Config{}).ResolveProfile("staging"); err == nil || !strings.Contains(err.Error(), "no profiles") { + t.Fatalf("missing profiles section: %v", err) + } + for name, p := range map[string]Profile{ + "bad distribution": {Distribution: "adhoc"}, + "bad env name": {Env: map[string]string{"API-URL": "x"}}, + "env with equals": {Env: map[string]string{"A=B": "x"}}, + "reserved env": {Env: map[string]string{"SCHEME": "Other"}}, + "reserved secret": {Env: map[string]string{"IOS_CERTIFICATE": "x"}}, + "reserved PATH": {Env: map[string]string{"PATH": "/tmp"}}, + "GitHub namespace": {Env: map[string]string{"GITHUB_TOKEN": "x"}}, + "Codemagic space": {Env: map[string]string{"CM_BUILD_ID": "x"}}, + "Bitrise space": {Env: map[string]string{"BITRISE_GIT_BRANCH": "x"}}, + } { + cfg.Profiles["bad"] = p + if _, err := cfg.ResolveProfile("bad"); err == nil { + t.Errorf("%s accepted", name) + } + } +} + +func TestProfileJSONRoundTrip(t *testing.T) { + raw := `{"project":"App","github":{"owner":"o","repo":"r"},"defaultProfile":"preview", + "profiles":{"preview":{"configuration":"Release","signing":false,"env":{"API_URL":"https://staging.example.com"},"distribution":"ad-hoc"}}}` + var cfg Config + if err := json.Unmarshal([]byte(raw), &cfg); err != nil { + t.Fatal(err) + } + p := cfg.Profiles["preview"] + if p.Signing == nil || *p.Signing || p.Distribution != "ad-hoc" || cfg.DefaultProfile != "preview" { + t.Fatalf("parsed %+v", cfg) + } + out, err := json.Marshal(&Config{Project: "App"}) + if err != nil || strings.Contains(string(out), "profiles") || strings.Contains(string(out), "defaultProfile") { + t.Fatalf("empty profiles should be omitted: %s %v", out, err) + } +} + +func TestProfileEncodings(t *testing.T) { + s := BuildSettings{} + if s.EnvJSON() != "" || s.ProfileInput() != "" { + t.Fatal("no profile must produce no inputs") + } + s = BuildSettings{Profile: "preview", Env: map[string]string{"MSG": "line one\nline \"two\""}, Distribution: "ad-hoc"} + var env map[string]string + if err := json.Unmarshal([]byte(s.EnvJSON()), &env); err != nil || env["MSG"] != s.Env["MSG"] { + t.Fatalf("env encoding: %q %v", s.EnvJSON(), err) + } + var input struct { + Name string + Env map[string]string + Distribution string + } + if err := json.Unmarshal([]byte(s.ProfileInput()), &input); err != nil || input.Name != "preview" || input.Distribution != "ad-hoc" || input.Env["MSG"] != s.Env["MSG"] { + t.Fatalf("profile input: %q %v", s.ProfileInput(), err) + } + if strings.Contains(s.ProfileInput(), "\n") { + t.Fatal("profile input must be a single line") + } + noEnv := BuildSettings{Profile: "development"} + if got := noEnv.ProfileInput(); !strings.Contains(got, `"env":{}`) { + t.Fatalf("env should be an object even when empty: %s", got) + } +} diff --git a/internal/config/types.go b/internal/config/types.go index d67914d..16c39d3 100644 --- a/internal/config/types.go +++ b/internal/config/types.go @@ -18,6 +18,24 @@ type Config struct { ReactNative ReactNativeConfig `json:"reactNative,omitempty"` KMP KMPConfig `json:"kmp,omitempty"` MobAI MobAIConfig `json:"mobai,omitempty"` + // DefaultProfile is used when a command is run without --profile. Tag-triggered + // runs have no flags, so it is also the only way they can select a profile. + DefaultProfile string `json:"defaultProfile,omitempty"` + Profiles map[string]Profile `json:"profiles,omitempty"` +} + +// Profile is a named set of build settings, selected with --profile. Every +// field is optional and overrides the matching top-level setting; unset fields +// keep the top-level value. Runner and submit settings are planned here too. +type Profile struct { + Configuration string `json:"configuration,omitempty"` // overrides ios.configuration + Scheme string `json:"scheme,omitempty"` // overrides ios.scheme + Signing *bool `json:"signing,omitempty"` // overrides ios.signing; a pointer so false can override true + Provider string `json:"provider,omitempty"` // overrides provider + Env map[string]string `json:"env,omitempty"` // exported on the runner before dependencies and the build + // Distribution is reserved for the export step (development, ad-hoc, app-store, + // enterprise). It is validated and passed to the runner but not applied yet. + Distribution string `json:"distribution,omitempty"` } // CIConfig identifies an app already connected to the project's GitHub repository. diff --git a/internal/workflow/profile_test.go b/internal/workflow/profile_test.go new file mode 100644 index 0000000..7f199a2 --- /dev/null +++ b/internal/workflow/profile_test.go @@ -0,0 +1,197 @@ +package workflow + +import ( + "os" + "os/exec" + "path/filepath" + "runtime" + "strings" + "testing" + + "go.yaml.in/yaml/v3" +) + +// resolveStep returns the shell of the "Resolve parameters" step of a workflow +// template, which is where builder.json profiles are applied on the runner. +func resolveStep(t *testing.T, file string) string { + t.Helper() + data, err := GetTemplate(file) + if err != nil { + t.Fatal(err) + } + var wf struct { + Jobs map[string]struct { + Steps []struct { + Name string `yaml:"name"` + Run string `yaml:"run"` + } `yaml:"steps"` + } `yaml:"jobs"` + } + if err := yaml.Unmarshal(data, &wf); err != nil { + t.Fatal(err) + } + for _, job := range wf.Jobs { + for _, step := range job.Steps { + if step.Name == "Resolve parameters" { + return step.Run + } + } + } + t.Fatalf("%s has no Resolve parameters step", file) + return "" +} + +type resolved struct { + outputs map[string]string + env map[string]string + log string + err error +} + +// runResolve executes the step the way the runner does: bash, GITHUB_OUTPUT and +// GITHUB_ENV files, builder.json in the working directory. +func runResolve(t *testing.T, script, builderJSON string, env map[string]string) resolved { + t.Helper() + dir := t.TempDir() + if builderJSON != "" { + if err := os.WriteFile(filepath.Join(dir, "builder.json"), []byte(builderJSON), 0644); err != nil { + t.Fatal(err) + } + } + scriptPath := filepath.Join(dir, "resolve.sh") + if err := os.WriteFile(scriptPath, []byte(script), 0644); err != nil { + t.Fatal(err) + } + outPath, envPath := filepath.Join(dir, "output"), filepath.Join(dir, "env") + cmd := exec.Command("bash", "-e", scriptPath) + cmd.Dir = dir + cmd.Env = append(os.Environ(), "GITHUB_OUTPUT="+outPath, "GITHUB_ENV="+envPath, "GITHUB_REF_NAME=ios-build/abcdef12") + for k, v := range env { + cmd.Env = append(cmd.Env, k+"="+v) + } + out, err := cmd.CombinedOutput() + r := resolved{outputs: map[string]string{}, env: map[string]string{}, log: string(out), err: err} + if data, err := os.ReadFile(outPath); err == nil { + for _, line := range strings.Split(string(data), "\n") { + if k, v, ok := strings.Cut(line, "="); ok { + r.outputs[k] = v + } + } + } + if data, err := os.ReadFile(envPath); err == nil { + lines := strings.Split(string(data), "\n") + for i := 0; i < len(lines); i++ { + // GitHub's heredoc form: NAME< "$ENV_LOG" app="$dd/Build/Products/Debug-iphoneos/App.app" if [ "$settings" = true ]; then python3 - "$dd/Build/Products/Debug-iphoneos" <<'PY' @@ -166,10 +167,17 @@ fi scheme := `App's $(touch should-not-exist)` cmd := exec.Command("/bin/bash", script, "build") cmd.Dir = clone - cmd.Env = append(os.Environ(), "PATH="+bin+string(os.PathListSeparator)+os.Getenv("PATH"), "SNAPSHOT_REF="+ref, "SNAPSHOT_SHA="+sha, "BUILD_ID=abcdef12", "IOS_PATH=.", "USE_SIGNING=false", "CONFIGURATION=Debug", "SCHEME="+scheme, "SCHEME_LOG="+filepath.Join(dir, "scheme.log"), "BUILDER_CI_DIR="+filepath.Join(dir, "state")) + // The profile env arrives as one JSON object and must reach the build + // tools as ordinary variables, values intact. + buildEnv := `{"API_URL":"https://staging.example.com","NOTES":"line one\nline \"two\""}` + cmd.Env = append(os.Environ(), "PATH="+bin+string(os.PathListSeparator)+os.Getenv("PATH"), "SNAPSHOT_REF="+ref, "SNAPSHOT_SHA="+sha, "BUILD_ID=abcdef12", "IOS_PATH=.", "USE_SIGNING=false", "CONFIGURATION=Debug", "SCHEME="+scheme, "SCHEME_LOG="+filepath.Join(dir, "scheme.log"), "BUILDER_CI_DIR="+filepath.Join(dir, "state"), + "BUILD_ENV="+buildEnv, "DISTRIBUTION=ad-hoc", "ENV_LOG="+filepath.Join(dir, "env.log")) if out, err := cmd.CombinedOutput(); err != nil { t.Fatalf("runner: %s %v", out, err) } + if data, err := os.ReadFile(filepath.Join(dir, "env.log")); err != nil || string(data) != "https://staging.example.com|line one\nline \"two\"|ad-hoc" { + t.Fatalf("profile env did not reach the build: %q %v", data, err) + } if _, err := os.Stat(filepath.Join(clone, "build", "abcdef12.ipa")); err != nil { t.Fatal("runner produced no IPA:", err) } diff --git a/internal/workflow/templates/ios-build.yml b/internal/workflow/templates/ios-build.yml index e98bc2b..5a975b1 100644 --- a/internal/workflow/templates/ios-build.yml +++ b/internal/workflow/templates/ios-build.yml @@ -50,6 +50,13 @@ on: required: false type: string default: '17' + # One input for the profile fields that are not inputs of their own, so + # the workflow stays under the ten-input limit of workflow_dispatch. + profile: + description: 'Selected builder.json profile as JSON: {"name": "...", "env": {...}, "distribution": "..."}' + required: false + type: string + default: '{}' jobs: build: @@ -76,7 +83,8 @@ jobs: # Dispatch inputs arrive with their declared defaults. A tag push has no # inputs, so the values come from builder.json in the tagged commit and - # the build id is the tag name after the prefix. + # the build id is the tag name after the prefix. A tag build cannot pick + # a profile per run; it applies builder.json's defaultProfile, if any. - name: Resolve parameters id: params env: @@ -87,27 +95,79 @@ jobs: IN_CONFIGURATION: ${{ inputs.configuration }} IN_FLUTTER_VERSION: ${{ inputs.flutter_version }} IN_JDK_VERSION: ${{ inputs.jdk_version }} + IN_PROFILE: ${{ inputs.profile }} run: | set -e + PROFILE="" + if [ "$GITHUB_EVENT_NAME" != "workflow_dispatch" ] && [ -f builder.json ]; then + PROFILE=$(jq -r '.defaultProfile // empty' builder.json) + if [ -n "$PROFILE" ] && [ "$(jq -r --arg p "$PROFILE" '.profiles[$p] != null' builder.json)" != "true" ]; then + echo "::error::defaultProfile \"$PROFILE\" is not defined under profiles in builder.json" + exit 1 + fi + fi param() { # name dispatch-value jq-path default local v="" if [ "$GITHUB_EVENT_NAME" = "workflow_dispatch" ]; then v="$2" elif [ -f builder.json ]; then - v=$(jq -r "$3 // empty" builder.json) + v=$(jq -r --arg p "$PROFILE" "$3 // empty" builder.json) fi v="${v:-$4}" echo "$1=$v" >> "$GITHUB_OUTPUT" echo "$1=$v" } + # Profile fields override ios.*. signing needs the explicit null test: + # jq's // would let a profile's `false` fall through to ios.signing. param build_id "$IN_BUILD_ID" '""' "${GITHUB_REF_NAME##*/}" param ios_path "$IN_IOS_PATH" '.ios.path' '.' - param scheme "$IN_SCHEME" '.ios.scheme' '' - param use_signing "$IN_USE_SIGNING" '.ios.signing' 'false' - param configuration "$IN_CONFIGURATION" '.ios.configuration' 'Debug' + param scheme "$IN_SCHEME" '(.profiles[$p].scheme // .ios.scheme)' '' + param use_signing "$IN_USE_SIGNING" '(if .profiles[$p].signing != null then .profiles[$p].signing else .ios.signing end)' 'false' + param configuration "$IN_CONFIGURATION" '(.profiles[$p].configuration // .ios.configuration)' 'Debug' param flutter_version "$IN_FLUTTER_VERSION" '.flutter.version' '' param jdk_version "$IN_JDK_VERSION" '.kmp.jdkVersion' '17' + # The rest of the profile: name (for the summary), distribution (for + # the export step) and env, exported to every step from here on so + # dependency installs and the build see it. + if [ "$GITHUB_EVENT_NAME" = "workflow_dispatch" ]; then + PROFILE_JSON="$IN_PROFILE" + elif [ -f builder.json ]; then + PROFILE_JSON=$(jq -c --arg p "$PROFILE" '{name: $p, env: (.profiles[$p].env // {}), distribution: (.profiles[$p].distribution // "")}' builder.json) + fi + [ -n "${PROFILE_JSON:-}" ] || PROFILE_JSON='{}' + PROFILE=$(jq -r '.name // ""' <<< "$PROFILE_JSON") + DISTRIBUTION=$(jq -r '.distribution // ""' <<< "$PROFILE_JSON") + case "$DISTRIBUTION" in + ''|development|ad-hoc|app-store|enterprise) ;; + *) echo "::error::distribution \"$DISTRIBUTION\" must be development, ad-hoc, app-store or enterprise"; exit 1 ;; + esac + echo "profile=$PROFILE" >> "$GITHUB_OUTPUT" + echo "distribution=$DISTRIBUTION" >> "$GITHUB_OUTPUT" + echo "profile=${PROFILE:-(none)}" + echo "distribution=$DISTRIBUTION" + # Values are base64 per entry so newlines and quotes survive; the + # heredoc form of GITHUB_ENV then takes them verbatim, with a random + # delimiter so no value line can end it early. Names are checked so a + # value cannot smuggle in a second variable. + if [ "$(jq -r '.env // {} | type' <<< "$PROFILE_JSON")" != "object" ]; then + echo "::error::the profile's env must be a JSON object of variable names to string values"; exit 1 + fi + while IFS=' ' read -r key encoded; do + name=$(printf '%s' "$key" | base64 --decode) + if ! [[ "$name" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]]; then + echo "::error::env name \"$name\" is not a valid environment variable name"; exit 1 + fi + delim="BUILDER_ENV_${RANDOM}${RANDOM}${RANDOM}" + { + echo "$name<<$delim" + printf '%s' "$encoded" | base64 --decode + echo + echo "$delim" + } >> "$GITHUB_ENV" + echo "env: $name" + done < <(jq -r '.env // {} | to_entries[] | "\(.key | @base64) \(.value | tostring | @base64)"' <<< "$PROFILE_JSON") + - name: Setup Xcode uses: maxim-lobanov/setup-xcode@v1 with: @@ -643,11 +703,15 @@ jobs: env: USE_SIGNING: ${{ steps.params.outputs.use_signing }} CONFIGURATION: ${{ steps.params.outputs.configuration }} + PROFILE: ${{ steps.params.outputs.profile }} run: | echo "## Build Summary" >> $GITHUB_STEP_SUMMARY echo "" >> $GITHUB_STEP_SUMMARY echo "- **Build ID:** ${{ steps.params.outputs.build_id }}" >> $GITHUB_STEP_SUMMARY echo "- **Status:** ${{ job.status }}" >> $GITHUB_STEP_SUMMARY + if [ -n "$PROFILE" ]; then + echo "- **Profile:** $PROFILE" >> $GITHUB_STEP_SUMMARY + fi echo "- **Configuration:** $CONFIGURATION" >> $GITHUB_STEP_SUMMARY echo "- **DerivedData Cache:** ${{ steps.cache-deriveddata.outputs.cache-hit == 'true' && 'Hit' || 'Miss' }}" >> $GITHUB_STEP_SUMMARY echo "- **Pods Cache:** ${{ steps.pods-cache.outputs.cache-hit == 'true' && 'Hit' || 'Miss' }}" >> $GITHUB_STEP_SUMMARY diff --git a/internal/workflow/templates/ios-share.yml b/internal/workflow/templates/ios-share.yml index 56f757f..636b2de 100644 --- a/internal/workflow/templates/ios-share.yml +++ b/internal/workflow/templates/ios-share.yml @@ -53,6 +53,13 @@ on: required: false type: string default: '17' + # Same encoding as ios-build.yml. Only the env applies here: simulator + # builds are always Debug and unsigned, so distribution is ignored. + profile: + description: 'Selected builder.json profile as JSON: {"name": "...", "env": {...}, "distribution": "..."}' + required: false + type: string + default: '{}' jobs: simulator: @@ -81,7 +88,8 @@ jobs: # Dispatch inputs arrive with their declared defaults. A tag push has no # inputs, so the values come from builder.json in the tagged commit and - # the build id is the tag name after the prefix. + # the build id is the tag name after the prefix. A tag build cannot pick + # a profile per run; it applies builder.json's defaultProfile, if any. - name: Resolve parameters id: params env: @@ -91,14 +99,23 @@ jobs: IN_DURATION: ${{ inputs.duration }} IN_FLUTTER_VERSION: ${{ inputs.flutter_version }} IN_JDK_VERSION: ${{ inputs.jdk_version }} + IN_PROFILE: ${{ inputs.profile }} run: | set -e + PROFILE="" + if [ "$GITHUB_EVENT_NAME" != "workflow_dispatch" ] && [ -f builder.json ]; then + PROFILE=$(jq -r '.defaultProfile // empty' builder.json) + if [ -n "$PROFILE" ] && [ "$(jq -r --arg p "$PROFILE" '.profiles[$p] != null' builder.json)" != "true" ]; then + echo "::error::defaultProfile \"$PROFILE\" is not defined under profiles in builder.json" + exit 1 + fi + fi param() { # name dispatch-value jq-path default local v="" if [ "$GITHUB_EVENT_NAME" = "workflow_dispatch" ]; then v="$2" elif [ -f builder.json ]; then - v=$(jq -r "$3 // empty" builder.json) + v=$(jq -r --arg p "$PROFILE" "$3 // empty" builder.json) fi v="${v:-$4}" echo "$1=$v" >> "$GITHUB_OUTPUT" @@ -106,11 +123,44 @@ jobs: } param build_id "$IN_BUILD_ID" '""' "${GITHUB_REF_NAME##*/}" param ios_path "$IN_IOS_PATH" '.ios.path' '.' - param scheme "$IN_SCHEME" '.ios.scheme' '' + param scheme "$IN_SCHEME" '(.profiles[$p].scheme // .ios.scheme)' '' param duration "$IN_DURATION" '""' '30m' param flutter_version "$IN_FLUTTER_VERSION" '.flutter.version' '' param jdk_version "$IN_JDK_VERSION" '.kmp.jdkVersion' '17' + # The profile's env is exported to every step from here on so the + # dependency installs and the build see it. + if [ "$GITHUB_EVENT_NAME" = "workflow_dispatch" ]; then + PROFILE_JSON="$IN_PROFILE" + elif [ -f builder.json ]; then + PROFILE_JSON=$(jq -c --arg p "$PROFILE" '{name: $p, env: (.profiles[$p].env // {})}' builder.json) + fi + [ -n "${PROFILE_JSON:-}" ] || PROFILE_JSON='{}' + PROFILE=$(jq -r '.name // ""' <<< "$PROFILE_JSON") + echo "profile=$PROFILE" >> "$GITHUB_OUTPUT" + echo "profile=${PROFILE:-(none)}" + # Values are base64 per entry so newlines and quotes survive; the + # heredoc form of GITHUB_ENV then takes them verbatim, with a random + # delimiter so no value line can end it early. Names are checked so a + # value cannot smuggle in a second variable. + if [ "$(jq -r '.env // {} | type' <<< "$PROFILE_JSON")" != "object" ]; then + echo "::error::the profile's env must be a JSON object of variable names to string values"; exit 1 + fi + while IFS=' ' read -r key encoded; do + name=$(printf '%s' "$key" | base64 --decode) + if ! [[ "$name" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]]; then + echo "::error::env name \"$name\" is not a valid environment variable name"; exit 1 + fi + delim="BUILDER_ENV_${RANDOM}${RANDOM}${RANDOM}" + { + echo "$name<<$delim" + printf '%s' "$encoded" | base64 --decode + echo + echo "$delim" + } >> "$GITHUB_ENV" + echo "env: $name" + done < <(jq -r '.env // {} | to_entries[] | "\(.key | @base64) \(.value | tostring | @base64)"' <<< "$PROFILE_JSON") + # Starts the simulator booting in the background (cached, so later runs # boot fast) while the app builds. - name: Install mobai-ci + boot simulator diff --git a/internal/workflow/templates/runner.sh b/internal/workflow/templates/runner.sh index 7ed603e..b713a63 100644 --- a/internal/workflow/templates/runner.sh +++ b/internal/workflow/templates/runner.sh @@ -7,6 +7,23 @@ mkdir -p "$ci_dir" mode="${1:-build}" export IOS_PATH="${IOS_PATH:-.}" SCHEME="${SCHEME:-}" CONFIGURATION="${CONFIGURATION:-Debug}" export USE_SIGNING="${USE_SIGNING:-false}" JDK_VERSION="${JDK_VERSION:-17}" +# From the selected builder.json profile: DISTRIBUTION is reserved for the +# export step; BUILD_ENV is a JSON object exported by prepare(). +export DISTRIBUTION="${DISTRIBUTION:-}" BUILD_ENV="${BUILD_ENV:-}" + +# Exports the profile's env before any dependency install or build, as the +# GitHub workflows do. Values are base64 per entry so newlines and quotes +# survive; names are checked so a value cannot become a second variable. +export_build_env() { + [ -n "$BUILD_ENV" ] || return 0 + if [ "$(jq -r 'type' <<< "$BUILD_ENV" 2>/dev/null)" != "object" ]; then echo "BUILD_ENV must be a JSON object of variable names to string values" >&2; exit 1; fi + while IFS=' ' read -r key encoded; do + name=$(printf '%s' "$key" | base64 --decode) + if ! [[ "$name" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]]; then echo "Invalid env name in BUILD_ENV: $name" >&2; exit 1; fi + export "$name=$(printf '%s' "$encoded" | base64 --decode)" + echo "env: $name" + done < <(jq -r 'to_entries[] | "\(.key | @base64) \(.value | tostring | @base64)"' <<< "$BUILD_ENV") +} snapshot_checkout() { case "${SNAPSHOT_REF:-}" in @@ -39,6 +56,7 @@ prepare() { fi echo "Project type: $project_type" if ! command -v jq >/dev/null; then brew install jq; fi + export_build_env # Match the GitHub workflows' committed xcconfig-template convention. find . -path ./DerivedData -prune -o -type f \