Researchers at Privatemode have published a new approach to turn standard large language models like GLM-5.3-Flash into fast decision-making systems, matching the performance of TypeSafe's Jev decision model in both accuracy and speed. The work addresses one of the key bottlenecks in deploying LLMs for real-time software decisions.
The core idea is to craft the input prompt so that the first output token answers the question directly, producing typed decisions with a probability for every option in a single forward pass. This eliminates the need for multi-token generation, JSON parsing, or structured output validation that typically slows down LLM-based decisions and adds latency to production systems.
Using a benchmark constructed from public datasets, the team showed that GLM-5.3-Flash running on Privatemode's infrastructure delivers results on par with Jev in decision accuracy and speed. As a bonus, the setup enables typed decisions on images, which is not possible with Jev — opening up multimodal decision use cases that were previously out of reach for this class of system.
The approach addresses a fundamental challenge in deploying LLMs for software decisions: most applications need fast, structured outputs from a pre-defined set (like routing a support ticket, classifying a contract clause, or triaging an incident), but standard LLM inference is optimized for generating free-form text. By constraining the output to a single token, the system achieves decision-making speed comparable to purpose-built classifiers while retaining the flexibility and generalization of a general-purpose language model. The team notes this could be particularly valuable for edge deployments where latency and cost matter.