diff --git a/README.md b/README.md index abfea573d98..da4232e9bab 100644 --- a/README.md +++ b/README.md @@ -42,7 +42,9 @@ The following describes the layout of the repository and its different artifacts ## Building Sources To build all the artifacts, simply invoke the command `mvn install` at the root of this repository (or -the Maven command of your choice). +the Maven command of your choice). It is also possible to build artifacts with support for MKL enabled with +`mvn install -Djavacpp.platform.extension=-mkl` or CUDA with `mvn install -Djavacpp.platform.extension=-gpu` +or both with `mvn install -Djavacpp.platform.extension=-mkl-gpu`. Note that in some cases, if a version of the TensorFlow runtime library is not found for your environment, this process will fetch TensorFlow sources and trigger a build of all the native code (which can take @@ -55,7 +57,9 @@ read [TensorFlow documentation](https://www.tensorflow.org/install/source) for m To include TensorFlow in your Maven application, you first need to add a dependency on either the `tensorflow-core` or `tensorflow-core-platform` artifacts. The former could be included multiple times for different targeted systems by their classifiers, while the later includes them as dependencies for -`linux-x86_64`, `macosx-x86_64`, and `windows-x86_64`, with more to come in the future. +`linux-x86_64`, `macosx-x86_64`, and `windows-x86_64`, with more to come in the future. There are also +`tensorflow-core-platform-mkl`, `tensorflow-core-platform-gpu`, and `tensorflow-core-platform-mkl-gpu` +artifacts that depend on artifacts with MKL and/or CUDA support enabled. For example, for building a JAR that uses TensorFlow and is targeted to be deployed only on Linux systems, you should add the following dependencies: @@ -69,7 +73,7 @@ systems, you should add the following dependencies: org.tensorflow tensorflow-core-api 0.1.0-SNAPSHOT - linux-x86_64 + linux-x86_64${javacpp.platform.extension} ``` @@ -85,29 +89,30 @@ native dependencies as follows: org.tensorflow tensorflow-core-api 0.1.0-SNAPSHOT - linux-x86_64 + linux-x86_64${javacpp.platform.extension} org.tensorflow tensorflow-core-api 0.1.0-SNAPSHOT - macosx-x86_64 + macosx-x86_64${javacpp.platform.extension} org.tensorflow tensorflow-core-api 0.1.0-SNAPSHOT - windows-x86_64 + windows-x86_64${javacpp.platform.extension} ``` In some cases, pre-configured starter artifacts can help to automatically include all versions of -the native library for a given configuration. For example, the `tensorflow-core-platform` artifact includes -transitively all the artifacts above as a single dependency: +the native library for a given configuration. For example, the `tensorflow-core-platform`, +`tensorflow-core-platform-mkl`, `tensorflow-core-platform-gpu`, or `tensorflow-core-platform-mkl-gpu` +artifact includes transitively all the artifacts above as a single dependency: ```xml org.tensorflow - tensorflow-core-platform + tensorflow-core-platform${javacpp.platform.extension} 0.1.0-SNAPSHOT ``` diff --git a/tensorflow-core/pom.xml b/tensorflow-core/pom.xml index fe150f02680..ef5568bcadb 100644 --- a/tensorflow-core/pom.xml +++ b/tensorflow-core/pom.xml @@ -33,7 +33,7 @@ tensorflow-core-generator tensorflow-core-api - tensorflow-core-platform + tensorflow-core-platform${javacpp.platform.extension} @@ -51,8 +51,8 @@ macosx-x86_64${javacpp.platform.extension} windows-x86${javacpp.platform.extension} windows-x86_64${javacpp.platform.extension} - 1.5.1 - 0.20-${javacpp.version} + 1.5.2 + 0.21.2-${javacpp.version} @@ -337,7 +337,7 @@ - linux-armhf + linux-armhf${javacpp.platform.extension} @@ -349,7 +349,7 @@ - linux-arm64 + linux-arm64${javacpp.platform.extension} @@ -361,7 +361,7 @@ - linux-ppc64le + linux-ppc64le${javacpp.platform.extension} @@ -373,7 +373,7 @@ - linux-x86 + linux-x86${javacpp.platform.extension} @@ -385,7 +385,7 @@ - linux-x86_64 + linux-x86_64${javacpp.platform.extension} @@ -397,7 +397,7 @@ - macosx-x86_64 + macosx-x86_64${javacpp.platform.extension} @@ -409,7 +409,7 @@ - windows-x86 + windows-x86${javacpp.platform.extension} @@ -421,7 +421,7 @@ - windows-x86_64 + windows-x86_64${javacpp.platform.extension} @@ -434,7 +434,7 @@ linuxarm - linux-armhf + linux-armhf${javacpp.platform.extension} @@ -447,7 +447,7 @@ linuxarmhf - linux-armhf + linux-armhf${javacpp.platform.extension} @@ -460,7 +460,7 @@ linuxaarch64 - linux-arm64 + linux-arm64${javacpp.platform.extension} @@ -473,7 +473,7 @@ linuxarmv8 - linux-arm64 + linux-arm64${javacpp.platform.extension} @@ -486,7 +486,7 @@ linuxarm64 - linux-arm64 + linux-arm64${javacpp.platform.extension} @@ -499,7 +499,7 @@ linuxppc64le - linux-ppc64le + linux-ppc64le${javacpp.platform.extension} @@ -512,7 +512,7 @@ linuxamd64 - linux-x86_64 + linux-x86_64${javacpp.platform.extension} @@ -525,7 +525,7 @@ linuxx86-64 - linux-x86_64 + linux-x86_64${javacpp.platform.extension} @@ -538,7 +538,7 @@ linuxx86_64 - linux-x86_64 + linux-x86_64${javacpp.platform.extension} @@ -551,7 +551,7 @@ mac os xamd64 - macosx-x86_64 + macosx-x86_64${javacpp.platform.extension} @@ -564,7 +564,7 @@ mac os xx86-64 - macosx-x86_64 + macosx-x86_64${javacpp.platform.extension} @@ -577,7 +577,7 @@ mac os xx86_64 - macosx-x86_64 + macosx-x86_64${javacpp.platform.extension} @@ -590,7 +590,7 @@ windowsamd64 - windows-x86_64 + windows-x86_64${javacpp.platform.extension} @@ -603,7 +603,7 @@ windowsx86-64 - windows-x86_64 + windows-x86_64${javacpp.platform.extension} @@ -616,7 +616,7 @@ windowsx86_64 - windows-x86_64 + windows-x86_64${javacpp.platform.extension} diff --git a/tensorflow-core/tensorflow-core-api/build.sh b/tensorflow-core/tensorflow-core-api/build.sh index b1662e18230..058715af6c0 100755 --- a/tensorflow-core/tensorflow-core-api/build.sh +++ b/tensorflow-core/tensorflow-core-api/build.sh @@ -17,8 +17,16 @@ else export PYTHON_BIN_PATH=$(which python3) fi +if [[ "${EXTENSION:-}" == *mkl* ]]; then + export BUILD_FLAGS="$BUILD_FLAGS --config=mkl" +fi + +if [[ "${EXTENSION:-}" == *gpu* ]]; then + export BUILD_FLAGS="$BUILD_FLAGS --config=cuda" +fi + # Build C API of TensorFlow itself including a target to generate ops for Java -bazel build $BUILD_FLAGS --python_path="$PYTHON_BIN_PATH" --config=monolithic --config=mkl --output_filter=DONT_MATCH_ANYTHING --verbose_failures @org_tensorflow//tensorflow:tensorflow :java_op_gen_sources +bazel build $BUILD_FLAGS --python_path="$PYTHON_BIN_PATH" --config=monolithic --output_filter=DONT_MATCH_ANYTHING --verbose_failures @org_tensorflow//tensorflow:tensorflow :java_op_gen_sources # Normalize some paths with symbolic links TENSORFLOW_SO=(bazel-bin/external/org_tensorflow/tensorflow/libtensorflow.so.?.?.?) diff --git a/tensorflow-core/tensorflow-core-api/pom.xml b/tensorflow-core/tensorflow-core-api/pom.xml index 35bed91f824..4e26ca21853 100644 --- a/tensorflow-core/tensorflow-core-api/pom.xml +++ b/tensorflow-core/tensorflow-core-api/pom.xml @@ -20,11 +20,6 @@ javacpp ${javacpp.version} - - org.bytedeco - mkl-dnn - ${mkl-dnn.version} - org.tensorflow tensorflow-core-generator @@ -174,6 +169,9 @@ bash ${project.basedir}/build.sh + + ${javacpp.platform.extension} + ${project.basedir} diff --git a/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/AbstractTF_Tensor.java b/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/AbstractTF_Tensor.java index 4c0c5127a27..dabee8841c0 100644 --- a/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/AbstractTF_Tensor.java +++ b/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/AbstractTF_Tensor.java @@ -32,14 +32,7 @@ protected static class DeleteDeallocator extends TF_Tensor implements Pointer.De /** TensorFlow crashes if we don't pass it a deallocator, so... */ protected static Deallocator_Pointer_long_Pointer dummyDeallocator = new Deallocator_Pointer_long_Pointer() { @Override public void call(Pointer data, long len, Pointer arg) { } - }; - - static { - PointerScope s = PointerScope.getInnerScope(); - if (s != null) { - s.detach(dummyDeallocator); - } - } + }.retainReference(); /** A reference to prevent deallocation. */ protected Pointer pointer; diff --git a/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/presets/tensorflow.java b/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/presets/tensorflow.java index 9721705e39e..f1b41509736 100644 --- a/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/presets/tensorflow.java +++ b/tensorflow-core/tensorflow-core-api/src/main/java/org/tensorflow/c_api/presets/tensorflow.java @@ -84,6 +84,10 @@ "C:/Program Files (x86)/Microsoft Visual Studio 14.0/VC/redist/x64/Microsoft.VC140.OpenMP/", "C:/Program Files (x86)/Windows Kits/10/Redist/ucrt/DLLs/x64/" } + ), + @Platform( + value = {"linux", "macosx", "windows"}, + extension = {"-mkl", "-gpu", "-mkl-gpu"} ) }, target = "org.tensorflow.c_api", @@ -92,8 +96,69 @@ public class tensorflow implements LoadEnabled, InfoMapper { @Override public void init(ClassProperties properties) { String platform = properties.getProperty("platform"); + String extension = properties.getProperty("platform.extension"); + List preloads = properties.get("platform.preload"); + List resources = properties.get("platform.preloadresource"); List preloadpaths = properties.get("platform.preloadpath"); + // Only apply this at load time + if (!Loader.isLoadLibraries()) { + return; + } + + // Let users enable loading of the full version of MKL + String load = System.getProperty("org.bytedeco.openblas.load", + System.getProperty("org.bytedeco.mklml.load", "")).toLowerCase(); + + int i = 0; + if (load.equals("mkl") || load.equals("mkl_rt")) { + String[] libs = {"iomp5", "libiomp5md", "mkl_core", "mkl_avx", "mkl_avx2", "mkl_avx512", "mkl_avx512_mic", + "mkl_def", "mkl_mc", "mkl_mc3", "mkl_intel_lp64", "mkl_intel_thread", "mkl_gnu_thread", "mkl_rt"}; + for (i = 0; i < libs.length; i++) { + preloads.add(i, libs[i] + "#" + libs[i]); + } + load = "mkl_rt"; + resources.add("/org/bytedeco/mkl/"); + } + + if (load.length() > 0) { + if (platform.startsWith("linux")) { + preloads.add(i, load + "#mklml_intel"); + } else if (platform.startsWith("macosx")) { + preloads.add(i, load + "#mklml"); + } else if (platform.startsWith("windows")) { + preloads.add(i, load + "#mklml"); + } + } + + // Only apply this at load time since we don't want to copy the CUDA libraries here + if (!Loader.isLoadLibraries() || extension == null || !extension.endsWith("-gpu")) { + return; + } + String[] libs = {"cudart", "cublasLt", "cublas", "cufft", "curand", "cusolver", "cusparse", "cudnn", "nccl", "nvinfer"}; + for (String lib : libs) { + switch (platform) { + case "linux-arm64": + case "linux-ppc64le": + case "linux-x86_64": + case "macosx-x86_64": + lib += lib.equals("cudnn") ? "@.7" : lib.equals("nccl") ? "@.2" : lib.equals("nvinfer") ? "@.5" : lib.equals("cudart") ? "@.10.1" : "@.10"; + break; + case "windows-x86_64": + lib += lib.equals("cudnn") ? "64_7" : lib.equals("cudart") ? "64_101" : "64_10"; + break; + default: + continue; // no CUDA + } + if (!preloads.contains(lib)) { + preloads.add(i++, lib); + } + } + if (i > 0) { + resources.add("/org/bytedeco/cuda/"); + resources.add("/org/bytedeco/tensorrt/"); + } + String vcredistdir = System.getenv("VCToolsRedistDir"); if (vcredistdir != null && vcredistdir.length() > 0) { switch (platform) { diff --git a/tensorflow-core/tensorflow-core-platform-gpu/pom.xml b/tensorflow-core/tensorflow-core-platform-gpu/pom.xml new file mode 100644 index 00000000000..16292dfaece --- /dev/null +++ b/tensorflow-core/tensorflow-core-platform-gpu/pom.xml @@ -0,0 +1,99 @@ + + + + 4.0.0 + + + org.tensorflow + tensorflow-core + 0.1.0-SNAPSHOT + + tensorflow-core-platform-gpu + TensorFlow Core API Library Platform GPU + + + tensorflow-core-api + -gpu + + + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.linux-x86_64} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.macosx-x86_64} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.windows-x86_64} + + + + + + + maven-jar-plugin + + + default-jar + + + + ${javacpp.moduleId}.jar ${javacpp.moduleId}-linux-x86_64-gpu.jar ${javacpp.moduleId}-macosx-x86_64-gpu.jar ${javacpp.moduleId}-windows-x86_64-gpu.jar + + + + + + empty-javadoc-jar + + jar + + + javadoc + + + + empty-sources-jar + + jar + + + sources + + + + + + + + diff --git a/tensorflow-core/tensorflow-core-platform-mkl-gpu/pom.xml b/tensorflow-core/tensorflow-core-platform-mkl-gpu/pom.xml new file mode 100644 index 00000000000..b1d88d80538 --- /dev/null +++ b/tensorflow-core/tensorflow-core-platform-mkl-gpu/pom.xml @@ -0,0 +1,104 @@ + + + + 4.0.0 + + + org.tensorflow + tensorflow-core + 0.1.0-SNAPSHOT + + tensorflow-core-platform-mkl-gpu + TensorFlow Core API Library Platform MKL GPU + + + tensorflow-core-api + -mkl-gpu + + + + + org.bytedeco + mkl-dnn-platform + ${mkl-dnn.version} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.linux-x86_64} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.macosx-x86_64} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.windows-x86_64} + + + + + + + maven-jar-plugin + + + default-jar + + + + ${javacpp.moduleId}.jar ${javacpp.moduleId}-linux-x86_64-mkl-gpu.jar ${javacpp.moduleId}-macosx-x86_64-mkl-gpu.jar ${javacpp.moduleId}-windows-x86_64-mkl-gpu.jar + + + + + + empty-javadoc-jar + + jar + + + javadoc + + + + empty-sources-jar + + jar + + + sources + + + + + + + + diff --git a/tensorflow-core/tensorflow-core-platform-mkl/pom.xml b/tensorflow-core/tensorflow-core-platform-mkl/pom.xml new file mode 100644 index 00000000000..ba92be191fe --- /dev/null +++ b/tensorflow-core/tensorflow-core-platform-mkl/pom.xml @@ -0,0 +1,104 @@ + + + + 4.0.0 + + + org.tensorflow + tensorflow-core + 0.1.0-SNAPSHOT + + tensorflow-core-platform-mkl + TensorFlow Core API Library Platform MKL + + + tensorflow-core-api + -mkl + + + + + org.bytedeco + mkl-dnn-platform + ${mkl-dnn.version} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.linux-x86_64} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.macosx-x86_64} + + + ${project.groupId} + ${javacpp.moduleId} + ${project.version} + ${javacpp.platform.windows-x86_64} + + + + + + + maven-jar-plugin + + + default-jar + + + + ${javacpp.moduleId}.jar ${javacpp.moduleId}-linux-x86_64-mkl.jar ${javacpp.moduleId}-macosx-x86_64-mkl.jar ${javacpp.moduleId}-windows-x86_64-mkl.jar + + + + + + empty-javadoc-jar + + jar + + + javadoc + + + + empty-sources-jar + + jar + + + sources + + + + + + + + diff --git a/tensorflow-core/tensorflow-core-platform/pom.xml b/tensorflow-core/tensorflow-core-platform/pom.xml index b72990ba16b..8d6ace0948b 100644 --- a/tensorflow-core/tensorflow-core-platform/pom.xml +++ b/tensorflow-core/tensorflow-core-platform/pom.xml @@ -32,11 +32,6 @@ - - org.bytedeco - mkl-dnn-platform - ${mkl-dnn.version} - ${project.groupId} ${javacpp.moduleId}