Jensen Huang has a test for whether an engineer is worth keeping, and it comes with a token budget attached. Speaking on the All-In Podcast at the close of GTC 2026, the Nvidia chief executive said that if a $500,000 engineer’s annual AI token consumption came in under half their salary, “I am going to be deeply alarmed.” Nvidia, he confirmed, is working toward a $2 billion yearly token bill for its engineering force.
He was describing a trade-off most companies have already made with less fanfare: money that once paid people increasingly pays for tokens. The four largest hyperscalers have guided roughly $700 billion in combined 2026 capital expenditure, nearly double last year, while data from outplacement firm Challenger, Gray & Christmas shows AI as the most-cited reason for US job cuts for a record fourth consecutive month.
An internal Meta memo obtained by Reuters described May’s cuts of 8,000 roles as offsetting the company’s substantial investments, in a quarter when revenue grew 33%. The layoffs at companies like these aren’t survival measures. They’re financing.
The trouble is that the financing hasn’t bought what it promised. Gartner surveyed 350 executives at companies with over $1 billion in revenue, all deploying AI agents or automation, and found roughly 80% had cut headcount with no correlation to improved returns. Analyst Helen Poitevin’s verdict was blunt: “Workforce reductions may create budget room, but they do not create return.”
Uber learned the token side of that lesson the expensive way, giving 5,000 engineers AI coding tools in December and exhausting its entire 2026 AI budget by April. Chief Operating Officer Andrew Macdonald conceded that despite 70% of committed code being AI-generated, the connection to anything customers notice is missing: “That link is not there yet.”
Put those two failures side-by-side and the actual problem comes into focus. Companies treated the token bill as fixed and the workforce as flexible, when the opposite is true. Payroll cuts happen once and take institutional knowledge with them. A token budget, it turns out, bends in half a dozen places if anyone bothers to engineer it.
Where the token budget bends
The cheapest fix is also the least glamorous: stop paying to process the same text repeatedly. Prompt caching, now standard across the major API providers, cuts the cost of repeated input by up to 90% under Anthropic’s and OpenAI’s published pricing, because static content like system instructions and reference documents gets processed once and reread at a fraction of the rate.
Security firm ProjectDiscovery documented raising its cache hit rate from 7% to 84% by restructuring prompts, cutting its total LLM spend by 59 to 70% while serving 9.8 billion tokens from cache. That single engineering exercise recovered more budget than most AI-attributed layoff rounds save.
The next lever is routing work to the right-sized model. Providers’ own price lists show flagship…
Source link
Disclaimer
We strive to uphold the highest ethical standards in all of our reporting and coverage. We blogs.grocliq.com want to be transparent with our readers about any potential conflicts of interest that may arise in our work. It’s possible that some of the investors we feature may have connections to other businesses, including competitors or companies we write about. However, we want to assure our readers that this will not have any impact on the integrity or impartiality of our reporting. We are committed to delivering accurate, unbiased news and information to our audience, and we will continue to uphold our ethics and principles in all of our work. Thank you for your trust and support.
Website Upgradation is going on for any glitch kindly connect at [email protected]
