Which LLM Can You Run On-Prem When Your IP Is Worth Eight Figures?

Console manufacturers do not care about your developer productivity. When they distribute an early SDK for unreleased hardware, the non-disclosure agreement dictates exactly where that code can exist. Pasting a proprietary graphics API call into a hosted AI service to debug a rendering issue is a breach of contract. This is usually why a game studio starts looking at on-premise large language models. It is not about protecting their own unannounced titles. It is about maintaining their developer licenses with the hardware platforms.

Once the decision is made to run a model locally, the first filter is not a coding benchmark. It is the license. “Open weights” means you can download the file. It does not mean you can do whatever you want with it. Many models ship with community licenses that include acceptable-use policies restricting military, medical, or vaguely defined harmful applications. If the studio’s next title is a military shooter, legal will need a written interpretation of that clause. If the vendor cannot provide one, the model is dead on arrival. A model that is fifteen percent worse at code completion but carries an unambiguous Apache 2.0 or MIT license is the one that gets deployed. The unit of approval is a specific model release and its license, not a model family.

Evaluating open-weight models is disguised as an AI research task, but for infrastructure teams, it is purely an exercise in calculating VRAM and power delivery. A 70-billion parameter model requires over 140 gigabytes of VRAM to run at full precision. To fit it on enterprise workstation GPUs, the infrastructure team applies quantization, reducing the weights to 8-bit or 4-bit. This shrinks the memory footprint but causes a measurable drop in the model’s ability to reason through complex C++ templates. Quantization is a quality decision disguised as a hardware decision. The project quickly shifts from reading model cards to sourcing multi-GPU nodes that fit the memory requirements without exceeding the server room’s power delivery capacity.

Hardware sizing also dictates adoption. A technical director might test a 13-billion parameter model on a single 24GB card. It returns autocomplete in milliseconds but fails at multi-step reasoning, so developers abandon it after two weeks. The 70-billion model on the shared inference box is smarter, but it develops a queue during a release push. If the internal tool takes ten seconds to return a code completion, developers will go back to copying and pasting into external browsers. The bottleneck in local code generation is rarely the model’s intelligence; it is the latency between the keystroke and the inference server. Adoption is a security control. The on-prem model must be the path of least resistance.

When it is time to evaluate coding ability, generic benchmarks tell you almost nothing. A model that writes good Python boilerplate may be useless at HLSL, GLSL, or a proprietary scripting language. The evaluation must use the studio’s own codebase, languages, and patterns. A patch that looks right but fails the console build is not a successful coding answer. The test set should include small bug fixes, build-log diagnosis, and questions where the right answer is that there is not enough context.

Running the model on local metal solves the data exfiltration problem at the network boundary, but it introduces a software licensing and data retention layer. On-prem describes where inference runs, not where information goes. The airgap is a policy, not a network diagram. A build engineer might paste a failed console build log into an internal chat. The log contains paths into a restricted SDK and a command line with a credential. The model server is on-prem, but the chat gateway retains prompts for troubleshooting. That retention setting is where the security review starts. Keeping prompts off a hosted service does not keep them out of logs.

The same applies to retrieval-augmented generation. An art-tools developer might want the assistant to search design documents and asset metadata. If the documents carry different project access controls, a single shared vector index will let the model surface a restricted project name to an employee who cannot open the original file. You have to give the model the same project boundaries you give the engineer using it, including boundaries inside the search index.

The training data question has no clean answer. You cannot audit a model’s training data. You can document that you reviewed the license, evaluated the model on your own code, and selected it based on criteria your legal team approved. That documentation is the defense. Model licenses change, acceptable-use policies update, and indexed repositories evolve. The evaluation performed six months ago may no longer hold. Put a review on the calendar.