[alinaliBQ]

Rationale for this change

GH-51694

Also see #49411 (comment) for context

What changes are included in this PR?

Upload unsigned macOS DYLIB and PKG to draft releases.

• An ARM64 base package is uploaded alongside DYLIB to skip rebuild of PKG.

Adds a check to skip upload process for universal PKG/MSI artifacts if they already exist.

To release macOS and Windows ODBC after this PR:

• Assumption: developer running dev/release/07-flightsqlodbc-upload.sh would be able to sign both Windows and macOS artifacts on the same machine.

1. Generate unsigned DYLIB, macOS base PKG, and DLL with CI

   • a macOS base PKG is uploaded to skip rebuild in step 4

2. Sign DYLIB and DLL and upload to GitHub release

3. Generate unsigned universal PKG (using base PKG) and MSI with CI

4. Sign universal PKG and MSI and upload to GitHub release

Are these changes tested?

Tested in fork

Are there any user-facing changes?

N/A

Was AI used for this PR?

In accordance to the AI generation guidelines, please disclose below whether and how AI was used in this PR.

PR code and description written by:

• Human
• AI

Reviewed before submission by:

• Human
• AI
• Not reviewed

• GitHub Issue: [FlightRPC][C++][FlightPRC][ODBC] macOS PKG installer upload CI #51694

[github-actions]

⚠️ GitHub issue #49411 has been automatically assigned in GitHub to PR creator.

[alinaliBQ]

@raulcd @amoeba cc @kou @lidavidm PR is ready for review.

[alinaliBQ]

Here are the suggested changes for 07-flightsqlodbc-upload.sh that I generated with help from Claude. I added TODO comments.

Assumption: PMC member running 07-flightsqlodbc-upload.sh would be able to sign both Windows and macOS artifacts on the same machine. rcodesign might be a good candidate as codesign only works on mac it seems..

diff --git a/dev/release/07-flightsqlodbc-upload.sh b/dev/release/07-flightsqlodbc-upload.sh
index c823bc84ae..bb924cc785 100755
--- a/dev/release/07-flightsqlodbc-upload.sh
+++ b/dev/release/07-flightsqlodbc-upload.sh
@@ -53,6 +53,7 @@ fi
 
 . "${SOURCE_DIR}/utils-env.sh"
 
+# TODO: Add checks for required macos signer storepass
 if [ -z "${ESIGNER_STOREPASS:-}" ]; then
   echo "ERROR: ESIGNER_STOREPASS is not set" >&2
   exit 1
@@ -73,12 +74,17 @@ dll_signed="arrow_flight_sql_odbc.dll"
 
 : "${GITHUB_REPOSITORY:=apache/arrow}"
 
+# In PHASE_BUILD_MSI step, both unsigned MSI and PKG will be built
 : "${PHASE_DEFAULT:=1}"
 : "${PHASE_SIGN_DLL:=${PHASE_DEFAULT}}"
+: "${PHASE_SIGN_DYLIB:=${PHASE_DEFAULT}}"
 : "${PHASE_BUILD_MSI:=${PHASE_DEFAULT}}"
 : "${PHASE_SIGN_MSI:=${PHASE_DEFAULT}}"
+: "${PHASE_SIGN_PKG:=${PHASE_DEFAULT}}"
 
-if [ "${PHASE_SIGN_DLL}" -eq 0 ] && [ "${PHASE_BUILD_MSI}" -eq 0 ] && [ "${PHASE_SIGN_MSI}" -eq 0 ]; then
+if [ "${PHASE_SIGN_DLL}" -eq 0 ] && [ "${PHASE_SIGN_DYLIB}" -eq 0 ] &&
+  [ "${PHASE_BUILD_MSI}" -eq 0 ] && [ "${PHASE_SIGN_MSI}" -eq 0 ] &&
+  [ "${PHASE_SIGN_PKG}" -eq 0 ]; then
   echo "No phases specified. Exiting."
   exit 1
 fi
@@ -102,7 +108,7 @@ if [ -e "${tmp_dir}" ]; then
 fi
 
 if [ "${PHASE_SIGN_DLL}" -gt 0 ]; then
-  echo "[1/8] Downloading ${dll_unsigned} from release..."
+  echo "[1/15] Downloading ${dll_unsigned} from release..."
   gh release download "${tag}" \
     --repo "${GITHUB_REPOSITORY}" \
     --pattern "${dll_unsigned}" \
@@ -112,7 +118,7 @@ if [ "${PHASE_SIGN_DLL}" -gt 0 ]; then
     exit 1
   fi
 
-  echo "[2/8] Signing ${dll_unsigned}..."
+  echo "[2/15] Signing ${dll_unsigned}..."
   echo "NOTE: Running jsign. You may be prompted for your OTP PIN..."
   jsign --storetype ESIGNER \
     --alias d97c5110-c66a-4c0c-ac0c-1cd6af812ee6 \
@@ -128,27 +134,66 @@ if [ "${PHASE_SIGN_DLL}" -gt 0 ]; then
     exit 1
   fi
 
-  echo "[3/8] Uploading signed DLL to GitHub Release..."
+  echo "[3/15] Uploading signed DLL to GitHub Release..."
   gh release upload "${tag}" \
     --repo "${GITHUB_REPOSITORY}" \
     --clobber \
     "${tmp_dir}/${dll_signed}"
 fi
 
+# The signed DYLIB must be on the release before PHASE_BUILD_MSI: the workflow
+# downloads it (libarrow_flight_sql_odbc.*.dylib) to build the universal PKG.
+if [ "${PHASE_SIGN_DYLIB}" -gt 0 ]; then
+  echo "[4/15] Downloading unsigned universal DYLIB from release..."
+  gh release download "${tag}" \
+    --repo "${GITHUB_REPOSITORY}" \
+    --pattern "libarrow_flight_sql_odbc_unsigned.*.dylib" \
+    --dir "${tmp_dir}"
+  dylib_unsigned="$(ls "${tmp_dir}"/libarrow_flight_sql_odbc_unsigned.*.dylib)"
+  dylib_unsigned_name="$(basename "${dylib_unsigned}")"
+  dylib_signed="${tmp_dir}/${dylib_unsigned_name/_unsigned/}"
+  if is_dylib_signed "${dylib_unsigned}"; then
+    echo "ERROR: ${dylib_unsigned_name} is already signed" >&2
+    exit 1
+  fi
+
+  echo "[5/15] Signing ${dylib_unsigned_name}..."
+  # Rename before signing: codesign derives the signature identifier from the
+  # file name, so signing it as *_unsigned.* would leak that into the identifier.
+  mv "${dylib_unsigned}" "${dylib_signed}"
+  # TODO: replace example `codesign` command with ASF approved way to sign DYLIB
+  codesign --force \
+    --timestamp \
+    --options runtime \
+    --sign "${APPLE_CODESIGN_IDENTITY}" \
+    "${dylib_signed}"
+  if ! is_dylib_signed "${dylib_signed}"; then
+    echo "ERROR: $(basename "${dylib_signed}") is not signed" >&2
+    exit 1
+  fi
+
+  echo "[6/15] Uploading signed DYLIB to GitHub Release..."
+  gh release upload "${tag}" \
+    --repo "${GITHUB_REPOSITORY}" \
+    --clobber \
+    "${dylib_signed}"
+fi
+
+# This also builds the universal macOS PKG, using the signed DYLIB above.
 if [ "${PHASE_BUILD_MSI}" -gt 0 ]; then
-  echo "[4/8] Triggering odbc_release_step in package_odbc.yml workflow..."
+  echo "[7/15] Triggering odbc_release_step in package_odbc.yml workflow..."
   gh workflow run package_odbc.yml \
     --repo "${GITHUB_REPOSITORY}" \
     --ref "${tag}" \
     --field odbc_release_step=true
 
-  echo "[5/8] Waiting for workflow to complete. This can take a very long time..."
+  echo "[8/15] Waiting for workflow to complete. This can take a very long time..."
   REPOSITORY="${GITHUB_REPOSITORY}" \
     "${SOURCE_DIR}/utils-watch-gh-workflow.sh" "${tag}" package_odbc.yml
 fi
 
 if [ "${PHASE_SIGN_MSI}" -gt 0 ]; then
-  echo "[6/8] Downloading unsigned MSI..."
+  echo "[9/15] Downloading unsigned MSI..."
   gh release download "${tag}" \
     --repo "${GITHUB_REPOSITORY}" \
     --pattern "Apache-Arrow-Flight-SQL-ODBC-${version}-win64.msi" \
@@ -159,7 +204,7 @@ if [ "${PHASE_SIGN_MSI}" -gt 0 ]; then
     exit 1
   fi
 
-  echo "[7/8] Signing MSI..."
+  echo "[10/15] Signing MSI..."
   echo "NOTE: Running jsign. You may be prompted for your OTP PIN..."
   jsign --storetype ESIGNER \
     --alias d97c5110-c66a-4c0c-ac0c-1cd6af812ee6 \
@@ -174,9 +219,53 @@ if [ "${PHASE_SIGN_MSI}" -gt 0 ]; then
     exit 1
   fi
 
-  echo "[8/8] Uploading signed MSI to GitHub Release..."
+  echo "[11/15] Uploading signed MSI to GitHub Release..."
   gh release upload "${tag}" \
     --repo "${GITHUB_REPOSITORY}" \
     --clobber \
     "${msi}"
 fi
+
+if [ "${PHASE_SIGN_PKG}" -gt 0 ]; then
+  echo "[12/15] Downloading unsigned PKG..."
+  gh release download "${tag}" \
+    --repo "${GITHUB_REPOSITORY}" \
+    --pattern "ArrowFlightSQLODBC-*_unsigned.pkg" \
+    --dir "${tmp_dir}"
+  pkg_unsigned="$(ls "${tmp_dir}"/ArrowFlightSQLODBC-*_unsigned.pkg)"
+  pkg_name="$(basename "${pkg_unsigned}" _unsigned.pkg)"
+  pkg_signed="${tmp_dir}/${pkg_name}.pkg"
+  if is_pkg_signed "${pkg_unsigned}"; then
+    echo "ERROR: PKG is already signed" >&2
+    exit 1
+  fi
+
+  echo "[13/15] Signing PKG..."
+  # TODO: replace example `productsign` command with ASF approved way to sign PKG
+  productsign \
+    --timestamp \
+    --sign "${APPLE_INSTALLER_IDENTITY}" \
+    "${pkg_unsigned}" \
+    "${pkg_signed}"
+  if ! is_pkg_signed "${pkg_signed}"; then
+    echo "ERROR: PKG is not signed" >&2
+    exit 1
+  fi
+
+  echo "[14/15] Notarizing and stapling PKG. This can take a while..."
+  # TODO: replace example `xcrun` command with ASF approved way to notarize PKG
+  xcrun notarytool submit "${pkg_signed}" \
+    --keychain-profile "${APPLE_NOTARY_PROFILE}" \
+    --wait
+  # Stapling fails if there is no notarization ticket, so a rejected submission
+  # can't slip through even if notarytool exits successfully.
+  xcrun stapler staple "${pkg_signed}"
+  xcrun stapler validate "${pkg_signed}"
+  spctl --assess --type install --verbose "${pkg_signed}"
+
+  echo "[15/15] Uploading signed PKG to GitHub Release..."
+  gh release upload "${tag}" \
+    --repo "${GITHUB_REPOSITORY}" \
+    --clobber \
+    "${pkg_signed}"
+fi