Local Inference API

By
KURONAMl

Run local choice-based inference in dependent Minecraft mods with no Python or model setup for players.

Run local choice-based inference from your Minecraft mod without asking players to install Python, download a model separately, or configure an external service.

Local Inference API is a shared library. Send a text context, a question and a list of choices; receive a candidate index or an abstention asynchronously. It does not add gameplay on its own.

The full MOD contains the model and runtime. Dependent mods share one inference worker in each game process. A separate lightweight developer kit contains the compile-only API and example sources, so compiling a consumer does not require downloading the model.

var request = new DecisionRequest(
    "I cannot remember my login password.",
    "What does this person need?",
    List.of("Reporting a lost card", "Resetting a password")
);
LocalInference.decide(request).thenAccept(result -> {
    // result.selected() is a zero-based index, or null for abstention.
    // Schedule game changes on the game thread and recheck their targets.
});

Requests support 1–24 choices and a maximum of 512 tokens. Excess input is rejected rather than truncated. One request runs at a time with up to 16 waiting; failure, timeout or overload completes the future exceptionally. Consumers must handle both failures and abstentions.

The bundled model is an experimental INT8 derivative of Verdict 151M. It can misclassify instructions, and Japanese and gameplay decision quality are not generally established. Evaluate your task before using it. Scores are not permission checks or guarantees of correctness. Keybind Oracle uses it for action-category hints while fixed rules handle key allocation.

The current distribution is approximately 392–394 MB. The worker stays resident after use until disconnection or shutdown; one measured macOS run reached approximately 850 MB worker RSS, excluding Minecraft. Windows x64 and macOS Apple Silicon have been exercised. Linux native libraries are included but untested; Intel Macs are not supported by this bundle.

Implementation, API and examples are MIT licensed. Bundled models and dependencies retain their upstream licenses and notices. The model was created by its upstream authors.

Bugs and questions: comment on the CurseForge page, or DM @kuronami333 on X.