Tianshu Libs

Update Notes

This update primarily upgrades the underlying JJML inference runtime and completes the adaptation for MTP / Vulkan inference paths. The overall goal is to enhance the stability, long-text task recovery capability, and GPU inference controllability of local large language models within the Minecraft environment.

Major Changes

• Upgraded the underlying JJML version to adapt to the new native runtime library and Vulkan backend loading process.
• Enhanced compatibility for Qwen3.5 series GGUF models, still running pure text inference by default without actively enabling visual/mmproj paths.
• Added MTP speculative decoding support, which can detect if the current model supports MTP and provides automatic calibration capabilities.
• MTP is now a "request-level switch", allowing upper layers to decide whether to enable it on a per-conversation/task basis; automatically falls back to regular inference if the model doesn't support it or initialization fails.
• When background TASK is interrupted by CHAT or high-priority TASK, supports a more reliable cold recovery process, releasing native context / KV cache memory during suspension.
• Cold recovery changed to save token IDs and context checkpoint, avoiding boundary errors caused by re-tokenizing raw text.
• MTP cold recovery has been connected to the new JJML's restored-target recovery interface, making long task recovery more stable.
• Added Vulkan inference priority parameters, providing foundational capabilities for future "inference yielding to game rendering/main thread load".
• Enhanced GPU device selection parameters, supporting selection of specific Vulkan devices through configuration.
• Updated long context testing and MTP calibration logic, constructing heavy workloads based on actual token budgets to avoid prompts directly filling the context window.

Compatibility Notes

• When new runtime options are not passed, the original chat, chatStream, task, and taskStream calling methods remain unchanged.
• contextSize is still the fixed context window size at model loading time.
• maxTokens in requests represents the maximum response length; in scenarios with long prompts, it needs to be reasonably configured together with context.
• For regular players, MTP can be understood as an experimental acceleration capability; if the current model or device is not suitable, it will automatically downgrade without affecting basic chat functionality.

Validation Status

Smoke testing has been completed using Qwen3.5-2B-Q4_K_M.gguf in a Vulkan GPU environment, including regular inference, MTP inference, CHAT interrupting TASK, high-priority TASK preemption, MTP cold recovery, and 8k context long prompt scenario testing.