TornadoVM Enables Java on GPUs

Software developer coding on dual monitors in a well-lit modern office, focused and engaged. Photo: Zayed Hossain/Pexels

TornadoVM, an open-source framework for Java, has reached version 7.3, enabling GPU-native workloads from Java applications. This major milestone allows Java programs to automatically accelerate on multi-core CPUs and GPUs, providing a rich GPU programming ecosystem.

The framework works alongside existing JVMs, including OpenJDK, Graalvm, Temurin, and Zulu, and offloads Java code to hardware backends, managing memory between the Java heap and accelerators. This makes it a useful building block for modern cloud and machine learning workloads.

Architectural Changes

Version 7 brings several architectural changes, including the use of the Foreign Function & Memory (FFM) API in place of the Java Native Interface (JNI). The experimental JVM Compiler Interface (JVMCI) dependency has also been removed, and TornadoVM now sources all compiler metadata through a reflection + ASM + Unsafe layer on every supported JDK.

The release also deepens integration with the Nvidia ecosystem through an optimized CUDA backend, including support for the cuTile programming model and the ability to call CUDA libraries such as cuBLAS and CUTLASS directly from within TornadoVM TaskGraphs.

Integration and Support

TornadoVM adds support for multi-GPU execution and has library integrations with Quarkus, Lang4j, Flink, Spring AI, and others. The framework is also available through SDKMan, making it easily accessible to developers.

For example, the integration of Apache Flink with TornadoVM allows developers to write standard queries as they normally would with the Table API, and a configuration flag can be used to indicate that the query should potentially be accelerated on a GPU.

When the computation reaches the task manager, the framework checks if there is a GPU available in the node it is executing on and, if so, it connects to a library provider, which analyzes the intermediate representation (IR) of the query to see if the computation is actually suitable for GPU execution.

Programming Nvidia GPUs

TornadoVM offers four ways to program Nvidia GPUs as a Java programmer, including the parallel API, kernel context API, and TileContext. The parallel API is a simple annotation that annotates a loop, providing a hint for the compiler that the loop can be potentially parallelized.

Alongside the 7.3 release, the TornadoVM team introduced jitLLM v2.0, its next-generation Java AI inference engine built on TornadoVM. The team positions jitLLM as a direct competitor to llama.cpp and Ollama in performance, and it ships as a single JAR file.

Both TornadoVM and jitLLM are being commercialized through Beestream.ai, a spin-off founded by the TornadoVM team. Further details can be found on the TornadoVM site, and also in recent videos from Devoxx Belgium, including “Tap into the NVIDIA Ecosystem from Java with TornadoVM’s Hybrid API”. The TornadoVM team has made these resources available to help developers get started with the framework and its various features.