Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion exaudfclient/.env.template
Original file line number Diff line number Diff line change
Expand Up @@ -6,4 +6,6 @@ export PROTOBUF_LIBRARY_PREFIX=
export PROTOBUF_INCLUDE_PREFIX=
export OPENSSL_LIBRARY_PREFIX=
export OPENSSL_INCLUDE_PREFIX=
export VERBOSE_BUILD= #"--subcommands --verbose_failures"
export VERBOSE_BUILD= #"--subcommands --verbose_failures"
export ZMQ_LIBRARY_PREFIX=
export ZMQ_INCLUDE_PREFIX=
11 changes: 6 additions & 5 deletions exaudfclient/BUILD
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@

load("//:variables.bzl", "VM_ENABLED_DEFINES")

load("@rules_cc//cc:defs.bzl", "cc_binary", "cc_library")
load("@rules_shell//shell:sh_library.bzl", "sh_library")


VM_ENABLED_DEPS=select({
Expand Down Expand Up @@ -74,11 +75,11 @@ cc_binary(
## The purpose of the static binaries is to verify if the linker namespace test work correctly.
## It is crucial that all linker symbols of exaudflib and all it's dependency are loaded into a new linker namespace,
## so that the user can load it's own versions of those dependencies without any conflict.
## With this binary we simulate an error in our build system, that is a direct depedency to protobuf/zmq,
## With this binary we simulate an error in our build system, that is a direct dependency to protobuf/zmq,
## which then must be detected with the linker namespace tests:
## test/linker_namespace_sanity/linker_namespace_sanity.py checks the wrong configuration
## Besides this the test under test/python3/all/linker_namespace.py checks the normal build, which expects
## not to find any occurence of the dependencies (protobuf/zmq) in the primary linker namespace.
## not to find any occurrence of the dependencies (protobuf/zmq) in the primary linker namespace.
##
## We need to explicitly declare the dependency of protobuf/zmq here, as the exaudflib is a static lib (//exaudflib:exaudflib)
## and hence does not contain dependency information. We cannot declare the shared lib (:exaudflib_complete.so)
Expand All @@ -99,11 +100,11 @@ cc_binary(
data = ["@exaudfclient_base//:libexaudflib_complete.so"],
)

# Workarround for the hardcoded paths in exaudfclient for libexaudflib_complete.so and python_ext_dataframe.cc
# Workaround for the hardcoded paths in exaudfclient for libexaudflib_complete.so and python_ext_dataframe.cc
# - libexaudflib_complete.so and python_ext_dataframe.cc get dynamically loaded, therefore the exaudfclient needs to know their paths
# - Most flexible way to provides these paths would environment variables
# - The exasol database can't provide these paths, because they depend on the container
# - A workarround to provide these paths would be wrapper bash script which set these environment variables
# - A workaround to provide these paths would be wrapper bash script which set these environment variables
# - For performance reason, we can not wrap the binary into a shell script, as such this is only for local debugging and testing
# and we hardcode the paths for the production container

Expand Down
4 changes: 3 additions & 1 deletion exaudfclient/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@ bazel_dep(name = "exaudfclient_base", version = "1.0.0")
#For now, the Bazel Modul "exaudfclient_base" is stored in subdirectory "base".
#Later we might fetch it via Http or Git (after splitting up script-languages repository)
local_path_override(module_name="exaudfclient_base", path="base")
bazel_dep(name = "rules_jvm_external", version = "6.2")
bazel_dep(name = "rules_cc", version = "0.2.22")
bazel_dep(name = "rules_jvm_external", version = "6.7")
bazel_dep(name = "rules_shell", version = "0.8.0")

numpy_local_repository = use_repo_rule("@exaudfclient_base//:python_repository.bzl", "numpy_local_repository")
numpy_local_repository(name = "numpy")
Expand Down
2 changes: 1 addition & 1 deletion exaudfclient/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ For the language support:

## Start a build

The exaudfclient is a multi-language projects. Therefore, we are using [Bazel](https://docs.bazel.build/versions/master/bazel-overview.html) as build system, because it provide build support for many languages and allows to mix these languages. Because the exaudfclient has language bindings to Python 2/3, Java and R, we need specify where Bazel can find the correponding library- and header-files. This is done by Environment Variables.
The exaudfclient is a multi-language project. Therefore, we are using [Bazel](https://docs.bazel.build/versions/master/bazel-overview.html) as build system, because it provides build support for many languages and allows to mix these languages. Because the exaudfclient has language bindings to Python 2/3, Java and R, we need specify where Bazel can find the correponding library- and header-files. This is done by Environment Variables.

For executing the build locally, you can use the script

Expand Down
1 change: 1 addition & 0 deletions exaudfclient/base/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ exports_files(["filter_swig_code.py", "build_integrated.py",
"replace_swig_import_helper.py"])

load("//:variables.bzl", "VM_ENABLED_DEFINES")
load("@rules_cc//cc:defs.bzl", "cc_binary", "cc_library")

config_setting(
name = "benchmark",
Expand Down
1 change: 1 addition & 0 deletions exaudfclient/base/MODULE.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ module(name="exaudfclient_base", version = "1.0")

bazel_dep(name = "bazel_skylib", version = "1.7.1")
bazel_dep(name = "googletest", version = "1.15.0")
bazel_dep(name = "rules_cc", version = "0.2.22")
bazel_dep(name = "rules_jvm_external", version = "6.2")

# The following is necessary to build the UDF client for Java
Expand Down
2 changes: 2 additions & 0 deletions exaudfclient/base/benchmark_container/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "benchmark_container",
srcs = ["benchmark_container.cc","benchmark_container.h"],
Expand Down
9 changes: 8 additions & 1 deletion exaudfclient/base/exaudfclient.template.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,4 +4,11 @@ SCRIPT_DIR="$(dirname "$(readlink -f "${BASH_SOURCE[0]}")")"
echo "Changing to script directory $SCRIPT_DIR"
cd "$SCRIPT_DIR" || return 1
export LIBPYEXADATAFRAME_DIR="$SCRIPT_DIR/external/exaudfclient_base+/python/python3"
export LD_LIBRARY_PATH="/opt/conda/cuda-compat/:$LD_LIBRARY_PATH" #Temporary hack for the Cuda ML flavor(s)

if [[ -d /opt/conda ]]
then
export CONDA_DEFAULT_ENV=base
export CONDA_PREFIX=/opt/conda
export MAMBA_ROOT_PREFIX=$CONDA_PREFIX
export LD_LIBRARY_PATH="$CONDA_PREFIX/cuda-compat/:$LD_LIBRARY_PATH" # Temporary hack for the Cuda ML flavor(s)
fi
1 change: 1 addition & 0 deletions exaudfclient/base/exaudflib/BUILD
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package(default_visibility = ["//visibility:public"])
exports_files(["exascript.i"])

load("//:variables.bzl", "VM_ENABLED_DEFINES")
load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "script_data_transfer_objects",
Expand Down
13 changes: 7 additions & 6 deletions exaudfclient/base/java_repository.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,14 @@ load("@bazel_skylib//lib:paths.bzl", "paths")
# Problem Description:
# - Bazel creates for all external local repositories symlinks into its build directory
# - Bazel than writes during compilation these paths to the rpath of the binary
# - JNI/JVM uses the rpath for loading additonal shared libraries,
# but for a unknown reason it seems to truncate these pathes.
# - JNI/JVM uses the rpath for loading additional shared libraries,
# but for a unknown reason it seems to truncate these paths.
# - We assume two possible reason, the symlink paths of bazel contain a @, or
# the paths are to long and get for that reason truncated
# - In the end, it we wasn't able to convinve JNI/JVM from other pathes
# for its libraries, as the original pathes where apt installed the files.
# - In the end, it we wasn't able to convince JNI/JVM from other paths
# for its libraries, as the original paths where apt installed the files.
# Solution:
# - We solved the problem by adding the original pahtes for the libraries into the rpath via the linkopts
# - We solved the problem by adding the original paths for the libraries into the rpath via the linkopts

def _find_shared_libraries(prefix,library,p_repository_ctx):
command_result = p_repository_ctx.execute(["find", '%s'%prefix,'-name','%s'%library]) #TODO only one result
Expand All @@ -32,6 +32,8 @@ def _java_local_repository_impl(repository_ctx):

defines = '"ENABLE_JAVA_VM"'
build_file_content = """
load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "java",
srcs = glob(["{prefix}/include/*.h"], allow_empty=False),
Expand All @@ -50,4 +52,3 @@ java_local_repository = repository_rule(
implementation=_java_local_repository_impl,
local = True,
environ = ["JAVA_PREFIX"])

4 changes: 3 additions & 1 deletion exaudfclient/base/javacontainer/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

genrule(
name = "exascript_java_tmp_cc",
cmd = """
Expand Down Expand Up @@ -98,7 +100,7 @@ cc_library(
copts= ["-O0","-fno-lto"],
# We limit this target to -O0 (no optimization) and -fno-lto. because otherwise we get compiler warnings of the sort
# note: "code may be misoptimized unless -fno-strict-aliasing is used."
# Normally, this indicates that compiling with higher optimizations levels may break your progrom. Unitl now, we don't have enough tests
# Normally, this indicates that compiling with higher optimizations levels may break your program. Until now, we don't have enough tests
# to verify if higher levels of optimizations might work. We needed to deactivate link time optimization, too, because this delay the compilation
# until the exaudfclient binary is build and causes than there the same problem.
alwayslink=True,
Expand Down
1 change: 1 addition & 0 deletions exaudfclient/base/javacontainer/script_options/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "java_script_option_lines",
Expand Down
3 changes: 2 additions & 1 deletion exaudfclient/base/protobuf_repository.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ def _protobuf_local_repository_impl(repository_ctx):
fail("Environment Variable PROTOBUF_INCLUDE_PREFIX not found")
print("protobuf include prefix in environment specified; %s"%include_prefix)
build_file_content = """
load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "protobuf",
srcs = glob(["protobuf/lib/**/libprotobuf*.so"]),
Expand All @@ -28,4 +30,3 @@ protobuf_local_repository = repository_rule(
implementation=_protobuf_local_repository_impl,
local = True,
environ = ["PROTOBUF_LIBRARY_PREFIX","PROTOBUF_BIN","PROTOBUF_INCLUDE_PREFIX"])

3 changes: 3 additions & 0 deletions exaudfclient/base/python/BUILD
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

exports_files([
"exascript_python_preset_core.py",
"exascript_python_wrap.py",
Expand Down
8 changes: 5 additions & 3 deletions exaudfclient/base/python/python3/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_binary", "cc_library")

genrule(
name = "exascript_python_tmp_cc",
cmd = """
Expand All @@ -11,7 +13,7 @@ genrule(
#Run replace_swig_import_helper.py because the generated exascript_python.py is not compatible with Python 3.12
#TODO: Update Swig to a new version which produces code compatible with Python 3.12
cd ..
python3 "$(location //:replace_swig_import_helper.py)" "$(location exascript_python.py)" "$(location exascript_python.py)"
$$PYTHON3_PREFIX/bin/$$PYTHON3_VERSION "$(location //:replace_swig_import_helper.py)" "$(location exascript_python.py)" "$(location exascript_python.py)"
""",
outs = ["exascript_python_tmp.cc", "exascript_python.py"],
srcs = ["//exaudflib:exascript.i","//exaudflib:swig/script_data_transfer_objects_wrapper.h"],
Expand Down Expand Up @@ -44,7 +46,7 @@ genrule(
name = "exascript_python_int",
cmd = """
cp $(SRCS) .
python3 $(location //:build_integrated.py) "$(location exascript_python_int.h)" "exascript_python.py" "exascript_python_wrap.py" "exascript_python_preset.py"
$$PYTHON3_PREFIX/bin/$$PYTHON3_VERSION $(location //:build_integrated.py) "$(location exascript_python_int.h)" "exascript_python.py" "exascript_python_wrap.py" "exascript_python_preset.py"
""",
outs = ["exascript_python_int.h"],
srcs = [":exascript_python_tmp_cc", "//python:exascript_python_wrap.py", ":extend_exascript_python_preset_py"],
Expand All @@ -56,7 +58,7 @@ genrule(
cmd = """
ACTUAL_PYTHON_VERSION=`$$PYTHON3_PREFIX/bin/$$PYTHON3_VERSION -c 'import sys; print(".".join(map(str, sys.version_info[:3])))'`
if [[ $$ACTUAL_PYTHON_VERSION == 2* ]] ; then
python3 $(location //:filter_swig_code.py) "$@" "$<"
python $(location //:filter_swig_code.py) "$@" "$<"
else
Comment thread
tomuben marked this conversation as resolved.
cp "$<" "$@"
fi
Expand Down
5 changes: 4 additions & 1 deletion exaudfclient/base/python_repository.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,8 @@ def _python_local_repository_impl(repository_ctx):
defines.append('"ENABLE_PYTHON3"')
defines_str = ",".join(defines)
build_file_content = """
load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "{name}",
srcs = glob(["{lib_glob}"]),
Expand Down Expand Up @@ -118,6 +120,8 @@ def _numpy_local_repository_impl(repository_ctx):
numpy_include_dir =_get_numpy_include_dir(binary,repository_ctx)
hdrs = numpy_include_dir + "/*/*.h"
build_file_content = """
load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "numpy",
srcs = [],
Expand All @@ -135,4 +139,3 @@ numpy_local_repository = repository_rule(
implementation=_numpy_local_repository_impl,
local = True,
environ = ["PYTHON3_PREFIX","PYTHON3_VERSION"])

2 changes: 2 additions & 0 deletions exaudfclient/base/script_options_parser/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "exception",
hdrs = ["exception.h"]
Expand Down
2 changes: 2 additions & 0 deletions exaudfclient/base/script_options_parser/ctpg/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "script_option_lines_parser_ctpg",
hdrs = ["script_option_lines_ctpg.h"],
Expand Down
2 changes: 2 additions & 0 deletions exaudfclient/base/script_options_parser/legacy/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "script_option_lines_parser_legacy",
hdrs = ["script_option_lines.h"],
Expand Down
2 changes: 2 additions & 0 deletions exaudfclient/base/streaming_container/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "streamingcontainer",
srcs = ["streamingcontainer.cc","streamingcontainer.h"],
Expand Down
1 change: 1 addition & 0 deletions exaudfclient/base/swig_factory/BUILD
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "swig_factory_if",
Expand Down
2 changes: 2 additions & 0 deletions exaudfclient/base/utils/BUILD
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@

package(default_visibility = ["//visibility:public"])

load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "utils",
hdrs = [
Expand Down
3 changes: 2 additions & 1 deletion exaudfclient/base/zmq_repository.bzl
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ def _zmq_local_repository_impl(repository_ctx):
fail("Environment Variable ZMQ_INCLUDE_PREFIX not found")
print("zmq include prefix in environment specified; %s"%include_prefix)
build_file_content = """
load("@rules_cc//cc:defs.bzl", "cc_library")

cc_library(
name = "zmq",
srcs = glob(["zmq/lib/**/libzmq.so"]),
Expand All @@ -28,4 +30,3 @@ zmq_local_repository = repository_rule(
implementation=_zmq_local_repository_impl,
local = True,
environ = ["ZMQ_LIBRARY_PREFIX","ZMQ_INCLUDE_PREFIX"])

Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,11 @@ build_steps:
- conda-forge
packages:
- name: protobuf
version: =6.31.1
version: =7.35.1
- name: pcre
version: '=8.45'
- name: bazel
version: =8.3.1
version: =9.2.0
- name: make
version: '=4.3'
- name: chrpath
Expand Down
Original file line number Diff line number Diff line change
@@ -1,13 +0,0 @@
CVE-2025-47907 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-47906 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2020-11979 #Caused by cuda-nvvp 12.9.79 which is already latest version. Can be removed when updating CUDA Version. Ignoring for now.
CVE-2025-47912 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-58183 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-58185 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-58186 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-58187 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-58188 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-58189 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-61723 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-61724 #This affects nsight-compute version 2025.2.1. Ignoring for now
CVE-2025-61725 #This affects nsight-compute version 2025.2.1. Ignoring for now
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
scan:
skip-files:
- /usr/bin/trivy
# pip v26.2.1: SBOM lists packages affected by CVEs which are not installed or have been updated separately
- "**/pip/_vendor/bom.cdx.json"
# go binary - the included stdlib in v1.26.1 is affected by 20 CVEs
- "**/nsight-compute-2026.2.1/host/target-linux-x64/plugins/efa_metrics/nic_sampler"
ignore-policy: /trivy.rego
Loading