Training-Time vs Context-Injected Knowledge: How LLMs Actually "Know" Things

Modern LLMs have two distinct sources of knowledge, with meaningfully different reliability profiles.

Training-time knowledge is baked into model weights during training. Broad and general, but frozen at a cutoff date. More susceptible to hallucination, especially when the training data was sparse, conflicting, or outdated. The model cannot update this knowledge without retraining.

Context-injected knowledge is information you provide in the prompt or conversation. Far more reliably attended to than training knowledge — but it strongly shapes output, and incorrect injected information can override correct training knowledge. The model tends to treat what is in context as ground truth.

An important caveat: There are an increasing number of systems that attempt to extract and hold long-term information, and then re-inject it later on when some additional method (usually vector-similarity or another LLM call) deems it relevant. And there are a lot of agent systems that hang onto the near-past history/current conversation, and perhaps summarize it when it reaches some pre-determined token limit. So this can "mimic" training-time knowledge, and users can become confused about how the system "knows about their situation". But this is in fact context-injected knowledge, dependent on those additional systems to add it at LLM run-time.

Practical implication: for anything factual about your current scenario, or obscurely domain-specific, provide the relevant information directly in the prompt if you can, rather than relying on the model to recall it. However, for general or more broadly-domain-specific stuff (e.g. "how to code in python"), it is likely that modern models are sufficiently trained that they only require detailed code descriptions.