Investigate the Feasibility of Implementing the Heatmap as a GPU Shader

[hdo]

Investigate the feasibility of implementing the heatmap functionality in Bookmap using GPU shaders. Even entry-level GPUs, such as the 4060 Ti, are now available with 16GB of VRAM, providing ample memory for data-intensive operations. Additionally, Apple's M-series chips have unified memory architecture, which allows the CPU and GPU to share the same memory pool. Currently, full viewport updates with lots of historic data can take 2-5 seconds on heavy tickers like NVDA. Leveraging the parallel processing capabilities of modern GPUs can significantly enhance performance, achieving real-time updates, smoother rendering, and reduced CPU load.

#### Objectives

- Real-Time Performance: Achieve real-time updates for heatmap rendering to improve user interaction and responsiveness.
- CPU Load Reduction: Free up CPU resources by offloading heatmap computations to the GPU, allowing the CPU to handle other critical tasks.
- Cross-Platform Compatibility: Ensure the solution works seamlessly across different operating systems and hardware configurations.

#### Approaches

Approach 1: Partial GPU Computation

1. Optimized Data Format:
   - Preprocess MBO data on the CPU to include only the information relevant to the heatmap.
   - Use data structures like ByteBuffer or FloatBuffer in Java for optimized transfer to the GPU.

2. CPU-GPU Data Transfer:
   - Utilize OpenGL/Vulkan APIs for data transfer. LWJGL or JOGL can assist in setting up buffers and textures to hold the heatmap data.

3. Heatmap Rendering on GPU:
   - Implement the heatmap rendering using fragment shaders to take advantage of the GPU’s parallelism.
   - Map the preprocessed data to textures and use these textures in shaders to render the heatmap.

4. CPU Indicator Calculations:
   - Keep complex indicator calculations on the CPU to maintain compatibility with existing add-ons.

Approach 2: Full GPU Computation

1. Uploading MBO Data to GPU:
   - Optimize the MBO data for GPU processing using structures like float arrays or texture2D for efficient memory access.
   - Use OpenGL or Vulkan for efficient data transfer between CPU and GPU. Java bindings for these APIs (e.g., LWJGL or JOGL) can facilitate this.

2. Shader Language:
   - Use GLSL (OpenGL Shading Language) or SPIR-V (Vulkan) to implement the heatmap rendering and indicator computations.
   - Leverage compute shaders for non-graphical computations to perform indicator calculations.

3. Integration with Java:
   - Utilize libraries like LWJGL (Lightweight Java Game Library) or JOGL (Java OpenGL) to interact with OpenGL or Vulkan.

#### Risks and Mitigations

- Risk: Compatibility with existing Add-ons.
  - Mitigation: Implement Approach 1 initially. Gradually explore methods to fully integrate GPU-based computations while maintaining support for non-GPU add-ons.
- Risk: Compatibility issues across different GPUs and operating systems.
  - Mitigation: Extensive testing and use of cross-platform libraries.
- Risk: Potential performance bottlenecks during data transfer between CPU and GPU.
  - Mitigation: Optimize data formats and transfer methods.

#### Dependencies and Requirements

- Hardware: Modern GPUs with OpenGL or Vulkan support.
- Software: LWJGL or JOGL libraries for Java, OpenGL/Vulkan APIs.
- Team: Developers with experience in GPU programming and Java.

[Svyatoslav]

Hi! Thank you for suggestion. The main challenge with this approach is complexity, of both development and debugging (especially in option 2). But we'll hopefully get there at some point