Modify amount of context being sent to the model.
Sometimes the model will allways return error that the number of allowed token are exceeded the amount allowed, even when i try to say "HI".
it's possible that the environment is automatically including a large amount of data as context with every request, even simple ones. This could be anything from the entire project's codebase to a very long history of interactions.
it would be good to have an option to control how much information is sent to the model with each request.
1
vote
Razgar Abedi
shared this idea
-
or somehow effectively reset or delete the conversations,
one other idea is to reset the context to a rolled backed version, so when you rollback the contex of the conversations also rolledback to the state where it was.