does it give the full history to the LLM each time?
It’s limited to the context size supported by given model. You can give the model 100k tokens of history but if it’s configured for less, it will just truncate it before processing (usually by removing oldest tokens first)
It’s limited to the context size supported by given model. You can give the model 100k tokens of history but if it’s configured for less, it will just truncate it before processing (usually by removing oldest tokens first)
Good news for tokenmaxxers: frontier models now have 1M context
Easy way to blow through ridiculous amounts of tokens.