What’s The ROI Of Software Engineers’ Spending On AI?

TL;DR

Buying for a business?Offer from Amazon

Get business pricing on networking and server gear

• Business-only prices and quantity discounts
• Tax-exempt purchasing
• Multiple users, one account, clear invoices

Create a free Business account →Free to register

As an affiliate, we earn on qualifying purchases.Hide

The Pragmatic Engineer surveyed engineering leaders and senior engineers at 27 workplaces about AI-tool spending and pull-request (PR) output. CodeRabbit platform data showed PR counts more than doubled over a year, but the report says few companies can yet establish whether that increase is caused by AI or delivers a measurable financial return.

A survey of 27 workplaces found that companies are spending widely varying amounts on AI tools for software development, while many still cannot say whether that spending produces a measurable return. In data shared with The Pragmatic Engineer, CodeRabbit recorded a 113% rise in weekly pull requests among developers using its platform over a year, but the figures do not establish that AI caused the increase or that more PRs mean higher-value work.

The report, published by The Pragmatic Engineer, draws on conversations with engineering leaders and senior engineers at 27 workplaces, as well as aggregate usage data from CodeRabbit and Gitar. Its central finding is a measurement gap: companies can track AI bills and software output, but few interviewed organizations could connect the two to a clear return on investment. The report uses pull-request volume as a readily available measure of engineering output, not as a direct measure of quality, revenue or productivity.

CodeRabbit’s anonymized platform data showed a 113% increase in weekly PRs and a 127% rise in monthly PRs. In a tracked group of 20,000 developers, monthly PRs rose 128% over a year. Gitar, an AI code-verification company, reported that some teams categorized as AI-native shipped more than 60 PRs per developer per week. These are observations from platform data; the report does not show a controlled comparison that isolates AI’s effect from other changes.

Spending varied substantially, with no apparent relationship between company size and per-engineer costs in this sample. The report describes a two-person .NET business spending $3,000 per developer monthly, while a publicly traded company with 1,500 engineers capped monthly AI spending at $150 per developer. At a trading firm, total AI spending reportedly reached $8 million a month in September, up from $500,000 in January; average spending was about $1,600 per developer monthly, although traders were described as the largest users.

Why PR Counts Fall Short of ROI

Rising PR counts can indicate that engineers are producing and submitting code more often, but they do not show whether that code is useful, safe, maintainable or tied to business results. A company that judges AI tools solely by volume could mistake more activity for better outcomes. The report’s findings matter to engineering teams facing growing AI bills because they highlight the need to compare costs with outcomes that matter, including code quality, review burden, delivery time and the work required to maintain software.

Cost controls are also becoming a practical concern. The report says spending caps are common among medium- and high-spending companies, while some organizations expect subsidized subscriptions to end. Those changes could push bills up by five to 10 times, according to the companies cited. The range of spending in the sample suggests that budgeting decisions depend on tool access and usage patterns, rather than company size alone.

How Companies Track AI Use

The report describes pull-request throughput as a metric available at many software companies and potentially useful for comparing activity before and after AI adoption. But it also notes that workplace norms shape how frequently engineers submit PRs: teams with different expectations can have very different output rates. PR counts therefore need to be interpreted alongside each team’s baseline and practices.

Companies in the sample reported using tools and models including Claude, Cursor, Codex and GitHub Copilot. The report says API-based spending peaked for many respondents in June and July and had roughly halved by the time of the report, attributing the decline to a mix of model changes and a move to open models. These are reported patterns among the companies contacted, not a market-wide spending measure.

“The change in PR throughput is a very simple metric, and one of those “readily-available” measurements at nearly all companies.”

— The Pragmatic Engineer report, describing the measure

What the Data Cannot Establish

The report does not provide a controlled test showing that AI tools caused the observed growth in PRs. The platform figures cover users of particular services and may reflect changes in team size, work practices, tool adoption or other factors. Nor does the report establish whether the additional PRs improved software quality or business performance. The figures should not be treated as a complete estimate of engineering productivity or an industry-wide average.

The 27-workplace sample is not presented as representative of all companies, and many respondents’ names were withheld. The report also does not provide a common accounting method for comparing subscription fees, API use and other costs across organizations. It says few companies could answer the ROI question, but does not quantify how many or set a shared definition of return.

Companies Face Cost and Measurement Decisions

Companies whose subsidized AI subscriptions are ending will need to decide whether to limit usage, move to less costly tools or models, or absorb higher bills. The report also describes some organizations using spending caps and rate limits, while the trading firm said it was building internal GPU and CPU capacity and developing models. Those plans are reported intentions; the report does not provide results from them.

For employers evaluating AI investments, the next useful step is to compare spending with more than PR volume, using a consistent baseline and measures of quality, delivery and maintenance. Whether companies can produce reliable, comparable ROI figures remains unresolved in the report.

Key Questions

Did the report find that AI tools increased software engineers’ productivity?

It found higher PR counts in CodeRabbit platform data and high PR output among some Gitar customer teams. The report does not establish that AI caused these results or that they represent a measured productivity gain.

How much are companies spending on AI per developer?

Spending varied widely in the 27-workplace sample. The report cites examples from $150 per developer monthly at one large company to $3,000 at a two-person .NET business; these are examples, not an industry average.

What does PR throughput measure?

It counts pull requests submitted over a period. It can track software-development activity, but does not by itself measure code quality, business value or the time needed to review and maintain code.

Can companies currently calculate AI’s return on investment?

The report says few companies appear able to answer that question. It does not provide a standardized ROI calculation or enough evidence to draw a general conclusion across the industry.

Source: rss