11.1. Release Notes

Java Unified Neural Orchestration: is a pure-Java distributed LLM inference/fine-tuning engine: GGUF loading, pipeline/tensor parallel across JVM nodes via gRPC, CUDA/ROCm acceleration through the Panama FFI (java.lang.foreign), LoRA training built in, and an OpenAI-compatible REST API via Javalin. Requires JDK 25+, Maven 3.9+.

License: Apache 2.0

Release 0.1.1 — LoRA on GPU, finally fast¶

This is the LoRA release: real GPU training, multi-arch adapters, a rebuilt docs site, AI disclosure on outputs, and Windows parity for API / play / merge.

/train-qa — measured on TinyLlama¶

Command (both releases):

GPU execution:

./juno lora --model-path models/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf

CPU execution:

./juno lora --model-path models/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf --cpu

Prompt: /train-qa What is my name? A: Juno

• 0.1.1 CUDA vs CPU: ~19× (30 s vs 571 s).

• 0.1.1 CUDA vs 0.1.0 “GPU”: ~48× wall time for the same fact (30 s vs 1455 s) — 0.1.0 did not yet have the frozen-weight GPU training path.

• 0.1.1 also reaches target in 12 passes instead of 31, with completion-only loss, AdamW, schedules, and a loss→target progress bar.

Startup in 0.1.1 makes the path explicit: Training on CUDA (auto-selected) · microbatch size 8 (or CPU with a clear warning when --cpu is set).

What else shipped¶

• LoRA toolkit: rsLoRA, DoRA, AdamW, LR schedules, dropout, validation, LoRA+, checkpoint v2, microbatch GEMM + VRAM fallback.

• Architectures: LoRA training for Qwen2, Phi-3, dense Qwen3 (plus LLaMA-family).

• Workflow: /train-file-qa, POST /v1/lora/train-file-qa, JFR LoRA metrics, cleaner /reset.

• Docs: juno-documentation (MyST book).

• Compliance UX: “The replies are generated by an AI system.”

• Windows: --api-port, --lora-play, merge on juno.bat / run.bat.

Release 0.1.0 Highlights¶

Distributed inference¶

• Pipeline parallel: contiguous layer blocks across JVM nodes; activations flow serially over gRPC.

• Tensor parallel: full depth on each node with head/FFN slices; coordinator AllReduce on logits.

• Zero sidecar processes: coordinator (juno-master) and workers (juno-node) are shaded JVM jars.

GPU acceleration¶

• NVIDIA CUDA 12.x / cuBLAS and AMD ROCm 6+ / rocBLAS via Panama FFI (java.lang.foreign).

• Auto-selection at startup: CUDA → ROCm → CPU. Override with -Djuno.gpu.backend=cuda|rocm|auto.

• Device-resident FP16 weights; automatic CPU quantised fallback on VRAM OOM.

LoRA fine-tuning¶

• In-process training REPL: ./juno lora

• Inference overlay: --lora-play PATH (local, cluster, AWS)

• Native merge to standalone GGUF: ./juno merge (patched tensors stored as F32)

• Train-file scheduling: --lora-chunk-tokens (default 32; recommend 128 for files), --lora-max-train-tokens seeded corpus caps

• Train device: --lora-train-device auto|gpu|cpu (gpu fails closed if unavailable)

• Microbatch: --lora-microbatch N / LORA_MICROBATCH (default 8; 1 = FP16 sequential); VRAM OOM auto-retries FP16 then CPU under auto

• GPU LoRA training (LLaMA/Qwen2): resident FP32 forward/transpose + microbatched GEMM (default batch 8); adapters/Adam on host. See Performance methodology.

• Multi-arch resident GPU transpose: LLaMA-family, Qwen2, Phi-3 (fused physical), dense Qwen3 via shared LoraResidentWeights (FP32→FP16→CPU VRAM ladder under auto)

OpenAI-compatible REST¶

• POST /v1/chat/completions (blocking + SSE)

• GET /v1/models, GET /v1/models/{model}

• Enable with --api-port N on ./juno local or cluster mode

• Juno extensions: x_juno_priority, x_juno_session_id, x_juno_top_k

JVM integration¶

• Maven BOM: cab.ml:juno-bom:0.1.0

• Facade API: JunoPlayer, LoraTrainer, JunoHttpClient

• See Quickstart: JVM embedding

Observability¶

• Custom JFR events across matmul, forward pass, token generation, LoRA training

• Health dashboard with per-node CPU load, coordinator P99 latency, node throughput

• Performance matrix: juno_test_matrix.html

Requirements¶

CPU-only inference requires no GPU stack. The ./juno launcher enforces JDK 25 at startup.

Supported models¶

GGUF with LLaMA-compatible architectures.

Quantizations: F32, F16, BF16, Q8_0, Q4_0, Q2_K, Q3_K, Q4_K, Q5_K, Q6_K.

Chat templates: llama3, mistral, gemma, tinyllama/zephyr, chatml, phi3. phi3 (Phi-3 / Phi-3.5) is supported via a dedicated handler and template. Gemma, Qwen 2, Qwen3, and Qwen3.5 (gemma, qwen2, qwen3, qwen3moe, qwen35) are under development: template and handler groundwork exists for some paths; end-to-end validation is in progress. Limitations for work in flight: no LoRA on Gemma/Qwen, no thinking-mode template, no fused QKV GGUFs on Qwen.

Quick start¶

mvn clean package -DskipTests

# Download a GGUF, then:
./juno local --model-path models/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf

# With OpenAI-compatible API:
./juno local --model-path models/tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf --api-port 8080

Full reference: docs/index.md

Known limitations (0.1.0)¶

• Text only: image or multimodal message content is not supported.

• OpenAI n > 1: rejected with HTTP 400; only single completions.

• Partial OpenAI compatibility: stop, presence_penalty, logit_bias, user, seed are ignored for client compatibility.

• No built-in auth or TLS on the REST server; configure at the reverse proxy or network layer for production.

• LoRA merge / redistribution may trigger model-license obligations; see LoRA and merge licensing.

• EU AI Act: compliance-oriented features (AI disclosure, audit logging, auth) are not yet built in; see EU AI Act compliance.

Documentation map¶

Developer session history: Changelog

Upgrade / migration¶

This is the first public release. No prior version migration path.

Artifacts publish under cab.ml at version 0.1.0 on Maven Central. Import the BOM:

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>cab.ml</groupId>
      <artifactId>juno-bom</artifactId>
      <version>0.1.0</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>

<- 10.6 Contributors  |  Table of Contents  |  11.2 Changelog ->