Tianshu Libraries v1.0.5

This version primarily enhances the stability of LLM inference, long contexts, embedding, and usage statistics in production environments. Upgrading is highly recommended for all upper-level projects utilizing this library.

Key Updates

• Upgraded JJML to 2.1.2.0021-49a8eba.
• Fixed an issue where long prompt prefilling in large context scenarios could trigger underlying batch boundary errors.
• Fixed an issue where calling BGE embedding models could trigger a native assert or JVM exit.
• Embedding context no longer forcefully specifies MEAN pooling; this is now left to automatic selection by JJML / GGUF metadata.
• BGE bert.pooling_type=2 now automatically uses CLS pooling.
• Incorrect API calls for embedding by non-embedding models now throw Java-visible exceptions instead of crashing the JVM.
• Maintained the TASK cold suspension mechanism: suspended tasks do not retain additional GPU KV cache; they only retain text/token states and replay upon recovery.
• Fully verified CHAT interrupting TASK, cold recovery events, request-level MTP enablement, and embedding/search paths.

Usage Statistics Validation

This version focused on reviewing the accuracy of LlmTokenUsage:

• countChatPromptTokens is consistent with the actual inference promptTokens.
• chatWithUsage, taskWithUsage, and stream finish usage paths are consistent.
• Long-context production-scale validation passed:
  • ctx=32768
  • promptTokens=25735
  • maxTokens=2048
  • Actual completionTokens=2048
  • totalTokens=27783
  • Completion error after re-tokenizing the final text is 0.

Validation Status

Verified items:

• Qwen3 0.6B standard inference, streaming, cancellation, queue cancellation, and TASK interrupted by CHAT.
• Qwen3.5 4B MTP model request-level MTP inference.
• 32K context long prompt prefilling.
• TASK cold suspension and recovery events.
• BGE large zh embedding and search.
• Illegal sampler / embedding / search parameter interception.
• Requests exceeding the context limit throw Java-visible exceptions without crashing the JVM.

Upgrade Recommendations

Before submitting long-context requests, upper-level mods are advised to use: