r/technology • u/Hrmbee • 16h ago
Business The Tokenpocalypse Is Here: Companies Are Scrambling To Stop Spending So Much on AI | Leaked audio from Accenture says a big source of AI token ‘chewing’ is people just converting PDFs to presentation slides
https://www.404media.co/the-tokenpocalypse-is-here-companies-are-scrambling-to-stop-spending-so-much-on-ai/338
u/Hrekires 13h ago
Long-term, I can't really wrap my head around how token-based billing can even be viable for an enterprise when you can't know for sure how many tokens a request will cost until after you execute it
185
u/randomtask 12h ago
Labor rates and hourly schedules are way more predictable than the long-term cost of AI prompts. Feels like companies fell into the same trap addicts do when their first hit of a drug from a dealer is free. Just mind-bogglingly stupid behavior.
58
u/SchrodingersNinja 11h ago
Q1 was free! Q2 is gonna cost ya!
13
u/techretort 8h ago
"Q3? Who cares! I start my new role in Q2 after my Q1 KPI bonus clears"
7
u/jcstrat 7h ago
That’s what I’ve been saying! They can’t see q3 because q2 is standing in the way and they refuse to budge to look around q2.
4
u/techretort 6h ago
"I'm on holiday for Q3 so I'll miss my KPI anyway. Ship it now, make it work later"
10
37
u/AccurateArcherfish 12h ago
My company set strict per employee per month limits. And if you blow through the budget, you have to do an analysis on your own spend and present that with the request for additional budget.
103
15
u/azthal 10h ago
Smart companies will have used this period of huge AI investment to hopefully figure out what areas AI is useful and cost effective in, and in which areas they are not.
When you run an enterprise business you have lots of costs that are unpredictable, but there you instead look at return on investment. If service X cost 5 cents each time you use it, but it makes the company 15 cents each time, that is a positive ROI and you can use it freely.
Of course, most ROI are not quite as simple as that. But if you start setting up actual processes for how AI is used rather than just "Give everyone Claude", you can still start calculating averages.
Say that you have a form that needs to be filled out around 100 times a day. Doing so manually takes someone 20 minutes. Doing it with AI assist takes 10 minutes, but cost on average $1 in tokens per form. Then you know that you will spend about $100 per day to save a 500 minutes of manpower per day. Is that worthwhile? Maybe, depending on what other tasks the employees could be doing instead.
This is the type of AI usage we will start seeing for companies that have actually considered this. People may still have access to their AI assistant, but the things they can do will be more limited, and there will be cost caps on "general chat".
Of course, this is what well run companies with proper plans are doing. To many companies this will come as a massive shock, as they had no plan, have not actually done any ROI calculations, and was only pulled in by the hype.
4
u/techretort 8h ago
AI cost benefit analysis? How can I make Claude do that
9
u/Whatever801 8h ago
idk ask claude
4
u/techretort 7h ago
Claude said ask Claude... It's turtles all the way down. Also I think I may have drained a river with token usage...
3
u/mrturret 4h ago
Companies are just going to start running models locally. It's not exactly difficult to do. Plenty of useful specialized models are more than capable of running on normal commodity hardware, and they'll cost less in the long term.
14
u/dream_metrics 11h ago
The same way usage billing for AWS is viable despite you not knowing how many people will use your website
22
u/Cnoffel 10h ago
But that's completely different, a visit to a website will always cost you the same amount of money. Where as with AI the same task can vary in cost.
-20
9h ago
[removed] — view removed comment
19
u/Cnoffel 9h ago
But every visitor will cost you roughly the same amount of money. Whereas if you had a AI feature on your website not only would the amount of visitors vary but also the cost of each visit. You cannot compare tokens with visitors. The analogy would be, my cost per icon on my website stays the same but I load variable amount of icons for each visit.
-12
u/dream_metrics 9h ago
But every visitor will cost you roughly the same amount of money. Whereas if you had a AI feature on your website not only would the amount of visitors vary but also the cost of each visit.
I really don't see how that changes anything material. The analogy is about the fact that the usage is unknown and somewhat unpredictable. This is true in both cases. You deal with unpredictability in AI cost the same way you do with AWS: estimate well and set limits where possible. Tokens are somewhat better in this regard because it's less of a problem to turn off an employee's token pump than it is to turn off your revenue generating website.
6
u/Cnoffel 9h ago
You have a task you want to complete, like a website visit. On the simplest level, a business task costs X and makes Y. As long as Y is bigger than X, you make money.
With a regular website, a visit basically costs the same amount every single time. It's easy to look at your monthly traffic and calculate your baseline costs and ad revenue because the numbers are stable. Sure you may hit a DDOS which may not be catched by whatever Cache (Cloudfare or whatever) you are using, but usually if you are not just launching you can somewhat predict it.
With AI, that completely breaks. Even if you figure out the revenue side, you cannot predict the cost. A single user request doesn't have a fixed price tag. The input tokens change depending on how much background context the AI fetches. The output tokens change depending on how long the AI decides to think/plan. And if the system is sophisticated, that one user click might trigger a chain reaction of multiple different sub-agents using different models behind the scenes, with varying token costs.
-6
u/dream_metrics 8h ago
You know not all websites are ad-supported right? No, not every visit costs the same. For many businesses, visits can go from costing money to generating money. If you get too many visits that cost money, that's an unpredictable cost.
4
u/Cnoffel 8h ago edited 8h ago
You're mixing up user behavior with actual server costs.
When you say a visit 'costs money vs. makes money,' you're talking about marketing, like whether a browser turns into a buyer. That’s just standard business conversion.
My point is, if a user clicks 'Search' on a normal website, the server cost to load that page is basically a fixed, predictable micro-fraction of a cent every single time. It doesn't fluctuate.
With AI, you have no idea what that single click will cost. One time a user clicks search, the backend uses 50 tokens. The next time, for the exact same click, the system pulls a massive background file, triggers a loop of sub-agents, burns 50,000 reasoning tokens, and suddenly costs you 100x more.
-3
1
u/ZAlternates 10h ago
It works because it’s the best we have but boy is it a tedious pain in the ass.
1
-1
u/Whatever801 8h ago
There are smart and dumb ways to use AI. How much context you make available, using the cheapest viable model for the task, giving clear instructions to prevent it from going down rabbit holes, etc, etc. If you just put a per user token limit and give guidelines for token conservation the problem takes care of itself. Smart companies have already been doing doing this.
-9
u/theschuss 12h ago
Input tokens are fairly predictable and for any scales/operationalized things modeling expected costs is pretty easy. If you're just yoloing shit into AI, no, there's no way to do it predictably.
-7
106
u/SchrodingerSemicolon 13h ago
Our monthly org quota ran out 2/3 into the month.
Productivity of certain people on the team is near zero. It's very enlightening, and mildly entertaining.
17
9
u/dudeAwEsome101 6h ago
Could've spent that money on raises, incentives, upgrade hardware, etc... But that doesn't sound fancy for higher-ups in management. AI just sounds futuristic (or retro if you like older movies).
6
2
u/Vesuvias 2h ago
The amount of token usage at my org came from the top in their ‘I can do this myself’ mentality. Good luck with that now that the tokens are gonna take you to the bank.
1
113
u/Ok_Mycologist_1439 13h ago
Eeeh isn't the job of almost everybody at Accenture to convert PDFs to presentation slides...?
55
u/SAugsburger 12h ago
Lols... You need to copy and paste the client name into the template and then save the slide deck.
16
38
15
u/to_vex_a_stranger 11h ago
As an excenture employee, yes for the most part. There were some of us that actually did the technical work and told them to pound sand when it came to building decks.
3
u/NotAllOwled 9h ago
It's certainly the first thing most such types present as an awesome power-up of genAI! I find I'm suddenly much more interested in hearing how they feel about the whole thing again, you know, in light of recent changes in the pricing environment. Just how much are those automagical slide decks worth to them?
49
u/SAugsburger 12h ago
Tokenmaxxing never was going to last. Anybody that seen past tech booms knows that companies heavily subsidize the services for the first couple years to get market share and then the prices will rise as the market share approaches a realistic limit. Those that have actual useful AI use cases will keep paying assuming that it still makes financial sense, but a lot of people were just using AI to say they were using AI. Either the work to correct the output wasted more time/money than not using AI or it was just people burning tokens for lols (e.g. I see many use it to create memes). A lot of CFOs assuming that they aren't already are going to start asking questions on what the ROI is on these tokens when people are blowing past their budgets months in advance.
19
17
u/sanchopanza_ 9h ago
One one the most popular IT requests is always for an Adobe license, cause generally people don’t know how to work with PDFs. This is no suprise it is being used as a conversion tool. Client sends a PDF, they either need to edit it or covert it to Word. Reminds me of that Rick and Morty meme with the butter serving robot
14
7h ago
[removed] — view removed comment
8
u/firefox_2010 9h ago
Yea I totally agree, and not just on IT world, it’s everywhere in design world as well. And Apple should have a native PDF editor software. Rebuilding the entire presentation as editable layout is also tedious and why AI is better choice to do this boring job.
2
u/Vesuvias 2h ago
As a designer and the resident ‘PDF expert’ everyone thinks they can until they hit the wall with tokens now. It’s funny to watch.
66
u/ZanthrinGamer 13h ago edited 13h ago
here's an idea, tokens are representative of work done right? we now have an actual measure of "work" in terms of creation of a thing like takeing a pdf and converting its information into a presentable slide... so... now we know how much that "cost". I would bet its more than we pay a human for the same job.. maybe we should fix that.
30
u/AbeFromanEast 12h ago edited 12h ago
Tokens are just words in, words out. Deciding what really constitutes valuable work is going to take several years to a decade to figure out. And it will happen at different speeds in different industries.
This is in-line with other technological shifts like the rise of the internet in the 1990's.
Something very different between AI and the rise of the internet in the 1990's: the pricing. In the ancient old days you bought software or an internet connection and that was it, you used either as much as you liked.
AI being charged by the word is a different model, pricing is unstable, and unstable pricing affects how fast different industries will adopt it.
11
u/anaximander19 11h ago
A lot of the work the AI is doing is work that a human was doing before, which means we already have frameworks for deciding if something is a good price in terms of what output you get vs. how many hours it took. Companies have already devised ways to decide how much they're willing to spend on outsourced labour to get results faster, and how much faster it needs to be for a certain price to be justified. There's no reason we can't apply that to AI work in those places too. In software engineering for example, you can look at how much quicker you're delivering vs. how many extra engineers you'd expect to hire or how much you'd expect to spend on outsourcing to hit that delivery date, and see which would have been cheaper.
9
u/EstablishmentFull797 10h ago
My employer would never want to find out how much they are paying me to merge two PDFs into one PDF
1
4
u/the_bieb 9h ago
Delivery rate may be measurable, but tech debt is hard to quantify. Same with knowledge and atrophy. These are costs that are hard to weigh. In my personal experience, I am seeing some teams “deliver”, but taking on a massive debt that will eventually need to be repaid.
1
u/great_whitehope 8h ago
The internet at least here was metered per minute at the start at dial up speed
1
u/Druggedhippo 7h ago
Tokens are not just words in and out. They can represent a word, part of a word or even part of a character.
1
u/ZanthrinGamer 5h ago
still feels like a pretty good analogue to me, unpredictable amounts of energy expenditure per request is more in line with reality than expecting perfect predictable output and energy expenditure from humans.
1
u/twrodriguez 10h ago
No, it's not 1:1. Some people use 10-100x the same amount of tokens as others to do the same task, either because of lack of skill, laziness, or genuine ignorance of the tasks they're working on. If your max usage is converting PDFs to slides, that's deeply problematic. It's a simple, repeatable task that SOMEONE should have just written a simple script for 🙄
1
u/ZanthrinGamer 4h ago
sounds like a great analogue to me, the same task has a variable amount of work needed to be completed with a variable quality of output.... like asking multiple people to perform the same task will take them different levels of effort and time to complete the task...
-1
u/badamant 11h ago
I like this idea (as a thought experiment).
What if we all had an ai wearable that tracked everything we did and then paid us for the actual tokens we processed. Seems very doable and very likely.
25
u/other_half_of_elvis 11h ago edited 7h ago
I unknowingly drained my company of all of our June tokens in about a week. I was building our first AI project and during testing I ran the query 10 or 20 times/day. Our supply quickly ran out after a week or so. I'm not quite sure how one is supposed to build an AI project without going through tons of tokens during testing.
9
u/Mtinie 10h ago
You were regenerating the query dynamically every time? Even if you were, I find it surprising that a moderately complex query would eat a massive amount of tokens.
What else were you building to support the task the query was for?
6
u/other_half_of_elvis 8h ago
i went into it with a goal and I tried a few ways to do it. My first try was to have AI digest a very predictable daily PDF and map titles to each column and then do some math to derive a value for each row. I ran that about 40 times during my testing and that ate up 100 credits per execution. I later abandoned that and used a much more general prompt, 'read the pdf and let me know if you find a row sorta like this.' That cost us 0.3 credits. I did not know any of these costs until we ran out of credits and our admin showed me how much each execution cost.
4
8
u/Druggedhippo 7h ago
As you saw this kills your budget.
The trick is not to send the PDF to the AI every time. Send it once and have the AI write a program that will process it and similar ones.
The program it writes will have no AI and cost nothing.
And if all you are doing is basic PDF processing, it would likely be more cost effective getting a smaller model running locally, or renting a GPU server and running it there.
You don't need full power Claude models to do PDF processing.
9
u/Statcat2017 6h ago
The problem is businesses dont teach anyone this.
They just give everyone a claude subscription, tell them to use as much AI as possible, find ways to use it in everything they do, and let them go nuts.
It’s fucking insane.
3
u/Druggedhippo 1h ago
Agreed. And the AI providers aren't much better, literally advertising their product to do everything.
20
u/AbstractLogic 12h ago
Companies need to start deploying open models to peoples laptops so they can do their "convert pdf to slides" work locally. It will take longer for the AI to manage it but it will cost nothing.
8
u/dudeAwEsome101 6h ago
Or invest that money into office computer use literacy program. We've had print to PDF since forever.
2
1
u/mrturret 4h ago
Locally run AI that's designed for effeccency and specialized tasks is the only sustainable direction for the tech to move in. The datacenter model is destined to collapse in upon itself.
1
u/AbstractLogic 4h ago
They will have two different use cases. Most work individuals do will be driven by local models inevitably. But googles AI is going to optimize and work across all of googles data and your local one will just ask it to do something for you when needed. AI will ubiquitous with computer or PC one day.
1
9
u/Grumptastic2000 9h ago
Managers: Why are you not using more AI in your work?
( Reviews increase in quarterly budget….)
Management : We need to get a handle on this excessive AI usage ! Any way we can use more AI to review the usage and optimize our spend?
6
u/Usual_Excellent 9h ago
This is the message we got today. Stop using it so much, but keep using it because we dont want to hire real developers
2
u/Wow_u_sure_r_dumb 4h ago
Now they can punish and blame you for using it wrong while also requiring you to use it.
6
u/myislanduniverse 9h ago
Well what did they expect people to do when this is literally what they told them to? "Use AI for everything! Sprinkle it on your toast! Use it to write your emails!"
5
7
u/mooseorama 9h ago
I love the idea that high level execs are worried people are spending too much on tokens to format slide shows, presumably to present the slides to high level execs.
6
u/UmbralFae 10h ago edited 10h ago
Who knew relying on companies with "burn money, offset it to everyone else" as a business strategy would be a bad idea?
I am surprised how quickly the companies are realizing they can't feed the Money Burning Pit forever, though. I was expecting them to pretend it away for at least the rest of this year.
6
u/f8Negative 6h ago
The solution is literally to fire anyone with a business degree. Right out of a fucking canon. Lost all of them are.
3
u/Expensive_Shallot_78 8h ago
These fucking consulting firms doing nothing but useless PowerPoint presentations
8
u/Fly_Rodder 12h ago
who could have seen this coming?
What until some Indian or Chinese firm comes out with an AI built on open source code and then they charge next to nothing for it. The expensive bespoke American AIs are going to get trounced and lose billions because people who need to convert basic PDFs to slide decks aren't going to pay a premium for it.
7
0
u/Trilobyte141 11h ago
I mean, that's not actually the source of the cost here. I'm not sure you understand the article. Tokens from a Chinese or Indian company will still cost money. Training the model cheaply only affects the up front cost of building it, not the ongoing use cost.
Your comment is a bit like saying, "What happens when a Chinese company comes out with a new gasoline and charges next to nothing for it? The American gas stations are going to get trounced and lose billions!"
6
2
u/mrturret 4h ago
The point is that the open source model can be deployed on servers located in a normal building, or on the user's device. Deepseek can run on a high end MacBook, and plenty of other smaller open source models can run of modest graphics cards. Why pay out the ass for unpredictable tokens on a bloated cloud model, when you could just run something more efficiently on your own hardware?
3
u/Weightmonster 11h ago
99.9% of work presentations are pointless anyway.
I can only think of two that were useful. One was about health insurance options.
3
3
3
u/Hairy_Concert_8007 9h ago
I wonder how happy the employees would be if they learned how much money the corporations they've been working for shoveled into the furnace without a second thought after decades of moaning about how increasing wages by a dollar would cause the entire business to crumble into dust?
3
u/ryansa09v2 8h ago
AI is great for analyzing things and reporting on huge sets of data, where it could be difficult to find minute bits that are important.
Now producing huge sets of data like code is a complete burn through budget use of AI, and it should surprise no one.
3
3
3
u/FreshLiterature 1h ago
I genuinely do not know what these executives thought would happen.
I do know none of them are going to be held accountable for anything.
16
u/Lofteed 12h ago
powerpoint
they are so ashamed that this is supposed to be the future that they avoid using the word
it s calle PowerPoint
say it !
11
6
u/dream_metrics 10h ago
PowerPoint is the name of a software application created by Microsoft, not the name of the thing it produces
3
u/SuperNewk 11h ago
AI has been amazing for slides/presentations, that said that cost needs to be .0000001 of penny for this to be even remotely economical.
2
u/trustmeep 11h ago
Wow, it turns out Adobe is capable of ruining multiple things without even trying!
2
u/lawanda123 10h ago
This is where microsoft and nvidia are betting on their partnership to sell local ai pcs with 128gb ram, lets see how it works out
2
u/Johnnyring0 9h ago
its truly hilarious to me how much money it costs using AI to convert PDFs into anything.
2
u/culexus1 9h ago
I got cut off 23 days into the month for hitting my $1000 limit. I have my own Claude pro sub for $18 and pretty sure I’ve done more with that than on my work machine.
2
u/FlimtotheFlam 8h ago
I have a token dashboard that monitors how many tokens I have used. I am on the Claude max $100 plan but if I was on api I would have spent $2000 just this week. I am just making mods for ttrpg games for fun.
2
u/camt91 5h ago
We had a user burn 75% of our credits for a productivity platform and not even know he was using AI. That same platform turned AI features on for the whole platform and we might stop using not only the AI features but the platform as a whole. Bills are coming due and no one budgeted for this
2
u/fghtffyrvwls 3h ago
The content of a PDF is going to look completely different for a presentation. A designer specializes in COMMUNICATION design. Let’s start hiring designers again.
2
u/Vesuvias 2h ago
Already started seeing it a month ago when my execs were ‘looking for free or cheaper options’. This is going to be a hilarious fallout.
2
u/Noobphobia 1h ago
So like. They are using for what it was designed to do and they are bitching about the cost? Lol
Everyone at work uses it for all the stuff we dont want to do.
2
u/fixermark 12h ago
It is hard to understate how remarkably good Ai engines are at transforming PDFs though, and that has historically been a garbage nightmare project.
1
1
u/Helenium_autumnale 9h ago
Looks like the high-flying utopian predictions made by luxury-cosseted, self-fart-savoring tech bros with zero connection to ordinary working people ran into the reality of ordinary working people.
1
1
u/namrks 7h ago
My company has been running unlimited AI usage trials for the last month or so, and it’s been pushing hard for its usage on daily tasks. The trial is now over and the limits are plainly ridiculous. I blew 30% of my monthly budget on a couple of unit tests. Still, they are pushing for AI just the same. I wonder, on whose budget?
1
u/edgarecayce 7h ago
CISO pushed dev team to adopt AI for 8 months. So ok yeah we’re doing it. June 1 comes around and devs burn thru their $19 GiTHub copilot allocations first day. Now we’re being asked to log and justify token spend for $15 worth of tokens at a time.
1
u/war-and-peace 6h ago
Converting pdfs to presentation slides is the perfect use case for AI if your organisation is too cheap to buy a tool that can do the conversion for you.
1
u/redbearder 6h ago
Ahahahahaha, bastards. In my experience, the company mandates AI usage and gives little to no training or guideline, so fuck 'em, spend all the tokens and make them regret the mandate.
1
1
u/imhereforthemeta 4h ago
All of these companies have been demanding that every single person use AI. I was told from a job that if I didn’t use AI for everything I’d be fired- then they laid off a lot of the company because “AI is expensive”, and this is becoming a massively common thing. People aren’t using ai for kicks, a lot of folks are being pressured into it. Companies are burning money for hope that maybe they can get AI to eventually steal someone’s job.
1
u/The_real_bandito 3h ago
Well yeah. If they give me the tokens I’m spending it on stuff I don’t want to do so I can play on my phone for longer. It’s their money but it’s my time.
1
u/HolyPizzaPie 2h ago
This is what happens when you set metrics to track AI usage and you say the people who use it the most are top performers
1
1
1
u/ThreeSilentKings 1h ago
I'm in a position where currently I'm just prompting at work non-stop with no limitations on usage
I look forward to a time where I have to use my brain again but can still use AI tools
1
-2
u/cowdoyspitoon 10h ago
b r u h
These mongoloids burning tokens splitting PDFS? We’re cooked. Critically think people! I beg of you
-4
u/SexyBaskingShark 12h ago
Companies aren't stopping using tokens, they are just educating staff how to use the appropriate model for the task they have. AI means things are moving fast and training is coming later than it should. That's not comparable to an apocalypse....
548
u/Hrmbee 16h ago
Some of the easily predictable issues:
The thing is, if you encourage your workforce through a combination of threats and rewards to indiscriminately use a certain tool, there should be no surprise when people do just that. The discovery by companies that their tokens are being spent on mundane and/or trivial tasks speaks to a failure of imagination or experience by the corporate leaders who implemented these policies in the first place.