Batch your requests together
The second most important thing I think is you want to batch your requests together.
You have a conversation open and you begin… maybe you start by inputting this, a big contract that you want summarized. The chat will take in that document. And then it’ll use everything it knows about you and put that all into the request that will be your huge input and that will cost 10,000, 15,000 tokens.
So when you actually submit that prompt, what you instead want to do is start batching your requests together.
And what you do by grouping these prompts and these commands in is that instead of every single time you ask it to do one of those things, it has to re-summarize the document. It has to re-put in all of the tokens in the input box. And what that ends up doing is that’s costing you money every time for every prompt.
So I think batching is very good. Putting in all this stuff into the input box like 10 times for each command, you’re putting it in once and the agent is able to go through your list and do it one by one.
Turn down thinking or turn it off
Number three, turn down thinking or turn off thinking entirely.
And what it essentially does is the AI will take its output and actually reread it and make sure it aligns to what the prompt is. And what we see is that this is okay, but all of the thinking that it does is done in the output box, right?
So when it does all of its thinking in the output window, it’s going to be costing way more than if it did all of the thinking in the input window.
So this goes back to picking the right model. But once you’re in the ballpark of what model you want to use, you can go ahead and then experiment with a high level of thinking, then the medium level of thinking and then low level thinking. And basically keep going down until it’s no longer producing the outputs that you want.
Ask for shorter answers
Tell it to be precise in the prompt, in whatever you give it, whatever kind of answer you want it to return to you, tell it to be more precise.
They give you a whole essay to read when you wanted a one-line response. Tell it to process it beforehand, condense the idea into one paragraph or a few sentences.
By telling it to actually reduce and concise its answer, it’s producing less output tokens, which are more expensive. So by producing less of them, it’s costing you less.
Start a new chat when you change subjects
I’ll be a little hesitant by saying this one because I think people can go overboard and they’ll create a new thread or a new task per different command that they want to give their chat.
But let’s say you’re working on one project, right? You have this project and you want the AI to understand it first and then help you make changes to this one project. Some people might say you need a new task or a new chat window for each time.
But what you don’t want to do for that specific case is have it reanalyze your entire project, every new chat, because the context of that project will disappear with a new chat.
So it’s important to keep your context windows a little more concise by starting new chats and new sessions before you switch off subject or switch to a different subject. That’s a good instance of creating a new chat or a new session because that will have its own concise context window. And it won’t be bloated with all the details of this other project, which is just feeding into your input tokens.
But if you’re working on the same project, don’t just keep making new chats.
I think this one’s a little overblown because input tokens are going to be way cheaper anyway.