macOS with CMake + Ninja unfortunately still doesn't work 100%

[chris-se]

Yesterday I reported #5768, indicating that building OpenBLAS on macOS with Ninja worked for me in the "out of the box" configuration, so the workaround would only be needed for the Makefile variant.

A colleague of mine unfortunately ran into a problem (with slightly different toolchain, CMake and Ninja versions than I had) and was able to trigger the "argument list too long" issue even with Ninja. It appears to be the case that CMake+Ninja doesn't default to using response files in this case, and because I had an older version of the XCode installed, where OpenBLAS enables less variants, and didn't run into that with Ninja.

But since this is now the second time of me running into this, I decided to investigate this further. I found the following:

• There is an option to force CMake+Ninja to use response files. This works in this case for shared libraries. This must be enabled before the add_library() calls are made though, otherwise it will have no effect.
• For static libraries CMake+Ninja will try to use a response file, but ar does not accept them, and then the builds fails.
• Whereas for CMake+Unix Makefiles the response files will work for shared libraries (if they are enabled), so we don't need the other workaround if only the shared library is built.
• The workaround doesn't work for Ninja as-is though because the generated response file is different.

I've now created a patch (I'll open a pull request after this ticket) that does the following:

• Before the add_library() calls it sets the appropriate options to ensure that response files are used on macOS (regardless of the generator).
• When building static libraries (regardless of whether shared libraries are also built) the previous workaround is used for both Unix Makefiles and Ninja, but with a different response file list in each case.

After doing some test builds in the configurations make/static, make/shared, make/both, ninja/static, ninja/shared, and ninja/both I've verified that the CMake build system now properly generates the link commands in each case with the response file:

make, static

[ 98%] Linking C static library lib/libopenblas.a
/opt/homebrew/bin/cmake -P CMakeFiles/openblas_static.dir/cmake_clean_target.cmake
/opt/homebrew/bin/cmake -E cmake_link_script CMakeFiles/openblas_static.dir/link.txt --verbose=1
ar: creating archive [...]/OpenBLAS-0.3.32-test/build-make/static/lib/libopenblas.a
sh -c 'cat [...]/OpenBLAS-0.3.32-test/build-make/static/CMakeFiles/openblas_static.dir/objects*.rsp | xargs -n 1024 /usr/bin/ar -ru [...]/OpenBLAS-0.3.32-test/build-make/static/lib/libopenblas.a && exit 0' 
sh -c '/usr/bin/ar -rs [...]/OpenBLAS-0.3.32-test/build-make/static/lib/libopenblas.a [...]/OpenBLAS-0.3.32-test/build-make/static/driver/others/CMakeFiles/driver_others.dir/xerbla.c.o && exit 0' 
[ 98%] Built target openblas_static

make, shared

[ 98%] Linking C shared library lib/libopenblas.dylib
/opt/homebrew/bin/cmake -E cmake_link_script CMakeFiles/openblas_shared.dir/link.txt --verbose=1
/usr/bin/cc   -DNO_SME -DHAVE_C11 -Wall -fPIC -DGEMM_GEMV_FORWARD -DSBGEMM_GEMV_FORWARD -DBGEMM_GEMV_FORWARD -DSMALL_MATRIX_OPT -DDYNAMIC_ARCH -DNO_WARMUP -DMAX_CPU_NUMBER=6 -DMAX_PARALLEL_NUMBER=1 -DMAX_STACK_ALLOC=2048 -DNO_AFFINITY -DVERSION="\"0.3.32\"" -DBUILD_SINGLE -DBUILD_DOUBLE -DBUILD_COMPLEX -DBUILD_COMPLEX16 -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -current_version 0.3.0 -o lib/libopenblas.0.3.dylib -install_name @rpath/libopenblas.0.dylib @CMakeFiles/openblas_shared.dir/objects1.rsp @CMakeFiles/openblas_shared.dir/objects2.rsp @CMakeFiles/openblas_shared.dir/objects3.rsp @CMakeFiles/openblas_shared.dir/objects4.rsp @CMakeFiles/openblas_shared.dir/objects5.rsp @CMakeFiles/openblas_shared.dir/objects6.rsp @CMakeFiles/openblas_shared.dir/objects7.rsp @CMakeFiles/openblas_shared.dir/objects8.rsp @CMakeFiles/openblas_shared.dir/linkLibs.rsp
/opt/homebrew/bin/cmake -E cmake_symlink_library lib/libopenblas.0.3.dylib lib/libopenblas.0.dylib lib/libopenblas.dylib
[ 98%] Built target openblas_shared

make, static & shared

[ 98%] Linking C static library lib/libopenblas.a
/opt/homebrew/bin/cmake -P CMakeFiles/openblas_static.dir/cmake_clean_target.cmake
/opt/homebrew/bin/cmake -E cmake_link_script CMakeFiles/openblas_static.dir/link.txt --verbose=1
ar: creating archive [...]/OpenBLAS-0.3.32-test/build-make/both/lib/libopenblas.a
sh -c 'cat [...]/OpenBLAS-0.3.32-test/build-make/both/CMakeFiles/openblas_static.dir/objects*.rsp | xargs -n 1024 /usr/bin/ar -ru [...]/OpenBLAS-0.3.32-test/build-make/both/lib/libopenblas.a && exit 0' 
sh -c '/usr/bin/ar -rs [...]/OpenBLAS-0.3.32-test/build-make/both/lib/libopenblas.a [...]/OpenBLAS-0.3.32-test/build-make/both/driver/others/CMakeFiles/driver_others.dir/xerbla.c.o && exit 0' 
[ 98%] Built target openblas_static
[ 99%] Linking C shared library lib/libopenblas.dylib
/opt/homebrew/bin/cmake -E cmake_link_script CMakeFiles/openblas_shared.dir/link.txt --verbose=1
/usr/bin/cc   -DNO_SME -DHAVE_C11 -Wall -fPIC -DGEMM_GEMV_FORWARD -DSBGEMM_GEMV_FORWARD -DBGEMM_GEMV_FORWARD -DSMALL_MATRIX_OPT -DDYNAMIC_ARCH -DNO_WARMUP -DMAX_CPU_NUMBER=6 -DMAX_PARALLEL_NUMBER=1 -DMAX_STACK_ALLOC=2048 -DNO_AFFINITY -DVERSION="\"0.3.32\"" -DBUILD_SINGLE -DBUILD_DOUBLE -DBUILD_COMPLEX -DBUILD_COMPLEX16 -DADD_ -DCBLAS -arch arm64 -Wl,-search_paths_first -Wl,-headerpad_max_install_names @CMakeFiles/xscblat1.dir/objects1.rsp -o xscblat1 @CMakeFiles/xscblat1.dir/linkLibs.rsp
sh -c '/usr/bin/cc -fpic -shared -Wl,-all_load -Wl,-force_load,[...]/OpenBLAS-0.3.32-test/build-make/both/lib/libopenblas.a -o [...]/OpenBLAS-0.3.32-test/build-make/both/lib/libopenblas.0.3.dylib '
/opt/homebrew/bin/cmake -E cmake_symlink_library lib/libopenblas.0.3.dylib lib/libopenblas.0.dylib lib/libopenblas.dylib
[ 99%] Built target openblas_shared

ninja, static

[12771/12785] : && sh -c 'cat [...]/OpenBLAS-0.3.32-test/build-ninja/static/CMakeFiles/openblas_static.rsp | xargs -n 1024 /usr/bin/ar -ru [...]/OpenBLAS-0.3.32-test/build-ninja/static/lib/libopenblas.a && exit 0'  && sh -c '/usr/bin/ar -rs [...]/OpenBLAS-0.3.32-test/build-ninja/static/lib/libopenblas.a [...]/OpenBLAS-0.3.32-test/build-ninja/static/driver/others/CMakeFiles/driver_others.dir/xerbla.c.o && exit 0'  && :
ar: creating archive [...]/OpenBLAS-0.3.32-test/build-ninja/static/lib/libopenblas.a

ninja, shared

[12660/12786] : && /usr/bin/cc -DNO_SME -DHAVE_C11 -Wall -fPIC -DGEMM_GEMV_FORWARD -DSBGEMM_GEMV_FORWARD -DBGEMM_GEMV_FORWARD -DSMALL_MATRIX_OPT -DDYNAMIC_ARCH -DNO_WARMUP -DMAX_CPU_NUMBER=6 -DMAX_PARALLEL_NUMBER=1 -DMAX_STACK_ALLOC=2048 -DNO_AFFINITY -DVERSION="\"0.3.32\"" -DBUILD_SINGLE -DBUILD_DOUBLE -DBUILD_COMPLEX -DBUILD_COMPLEX16 -arch arm64 -dynamiclib -Wl,-headerpad_max_install_names -current_version 0.3.0 -o lib/libopenblas.0.3.dylib -install_name @rpath/libopenblas.0.dylib @CMakeFiles/openblas_shared.rsp && :

ninja, static & shared

[12771/12787] : && sh -c 'cat [...]/OpenBLAS-0.3.32-test/build-ninja/both/CMakeFiles/openblas_static.rsp | xargs -n 1024 /usr/bin/ar -ru [...]/OpenBLAS-0.3.32-test/build-ninja/both/lib/libopenblas.a && exit 0'  && sh -c '/usr/bin/ar -rs [...]/OpenBLAS-0.3.32-test/build-ninja/both/lib/libopenblas.a [...]/OpenBLAS-0.3.32-test/build-ninja/both/driver/others/CMakeFiles/driver_others.dir/xerbla.c.o && exit 0'  && :
ar: creating archive [...]/OpenBLAS-0.3.32-test/build-ninja/both/lib/libopenblas.a
[12777/12787] : && sh -c '/usr/bin/cc -fpic -shared -Wl,-all_load -Wl,-force_load,[...]/OpenBLAS-0.3.32-test/build-ninja/both/lib/libopenblas.a -o [...]/OpenBLAS-0.3.32-test/build-ninja/both/lib/libopenblas.0.3.dylib ' && :
[12778/12787] /opt/homebrew/bin/cmake -E cmake_symlink_library lib/libopenblas.0.3.dylib  lib/libopenblas.0.dylib lib/libopenblas.dylib && :