I’ve spent thousands in AI spend across the companies that I’ve worked with, which are small to medium-sized businesses.
I’m going to go over five ways that I’ve ranked as being the best to worst for reducing your AI spend and making your monthly bill a lot cheaper.
Number one, model choice. This is definitely the most important one.
Every model has a different use
Every different model will have different uses. A million tokens might not seem to accrue really quickly, but it definitely does.
As far as choosing the model, you want to make sure that you’re using the lowest qualified model for the task that you want to do.
So if it’s a very complicated multi-step task and it’s maybe client facing, it’s very important to get right, you want to maybe use the top of the line model.
Whereas if it’s not so important and it’s maybe used internally and maybe it’s just… or a gut check on a summary of a document or something like that, you maybe don’t need a top-of-the-line model, and you want to instead go for an open-source model or something with comparable performance but at a fraction of the cost.
Use the least viable model
Model choice is really important. And what you want to do is, like I said, use the least viable model for that task.
Because if you’re starting to spam the really high models for relatively simple tasks, then you’re paying for no reason. And you’re essentially getting the same answer as you would from a lesser quality model.
How to test it
So how would I test this? I would just take whatever model you’re using now. Look up a chart. It’s just a website that compares different models in real time.
And so I would start reducing the quality or the power of the model that you’re using until it’s producing outputs that no longer make sense or are starting to break your systems.
That cut alone has across many of the systems that we’re talking about has saved me three to four times the cost.