HOUSTON — For decades, elected leaders and corporate executives have chased a dream of independence from unstable or unfriendly foreign oil producers. Mission accomplished: Oil companies are producing record amounts of crude oil and natural gas in the United States and have become major exporters.

Yet the companies themselves are finding little to love about this seeming bonanza. With a global glut driving down prices, many are losing money and are staying afloat by selling assets and taking on debt.

The value of oil and gas stocks as a proportion of the S&P 500 over the past six years has dropped to about 4.6%, from 8.7%.

“It’s really a psychological punch in the gut,” said Matt Gallagher, chief executive of Parsley Energy, which has productive shale fields in the Permian Basin of Texas and New Mexico and has tripled output over the past three years. His company’s shares have tumbled to about $19 a share, from $38 in late 2016. “There’s a lot of risk in this industry, people are working very hard, and we feel we have made the right moves and it doesn’t show up in the share price.”

Domestic oil production has increased more than 60% since 2013, to over 12 million barrels a day, making the United States the biggest producer of oil and natural gas in the world and slashing imports. That growth has also reduced the clout and profits of the Organization of Petroleum Exporting Countries and Russia, enabling President Donald Trump to impose sanctions on Iran and Venezuela without risking higher gasoline prices or shortages.

Rising tensions with Iran after attacks on two oil tankers and a U.S. surveillance drone have lifted oil prices, but there has been little impact on the supply outlook.


Oil executives say the United States is set to become an even bigger factor because a further 5 million or so barrels of daily crude oil production are on the way in the next few years. Russia would have to drill deep into the Arctic to keep up, a prohibitively expensive proposition, and experts don’t think Saudi Arabia can increase production significantly.

But the share price of Exxon Mobil, the largest American oil company, is barely above where it was a decade ago. In years past, investors might have celebrated Occidental Petroleum’s proposed acquisition of Anadarko Petroleum, which has some of the most lucrative oil fields in the country. Instead, Occidental’s shares have fallen about 10% since that deal in early May.

In the past four years, roughly 175 oil and gas companies in the United States and Canada with debts totaling about $100 billion have filed for bankruptcy protection. Many borrowed heavily when oil and gas prices were far higher, only to collectively overproduce and undercut their commodity prices. At least six companies have gone bankrupt this year, and Weatherford International, the fourth-leading oil services company, which owes investors $7.7 billion, is expected to file for bankruptcy protection Monday.

In a February call with analysts, Weatherford’s chief executive, Mark A. McCollum, seemed exasperated. “I don’t waste a lot of time thinking or planning how to fail,” he said. “The elephant in the room for the entire sector is we’re not generating returns that our investors expect.”

One concern is that the industry will be forced to leave oil and gas in the ground as climate change prompts environmental restrictions on drilling or a shift to alternative fuels.

“The psychology has turned,” said David Katz, president of Matrix Asset Advisors, a New York investment firm that owns Occidental shares. “When you talk to investors they are concerned about oil companies spending money on something that will be in decline. There are more concerns that electric cars and hybrid cars are going to get more and more popular.”


That is a tectonic shift from just a few years ago.

In the early 2000s, experts like T. Boone Pickens warned that world demand for oil would outstrip production and that American output was in a long decline. Exxon Mobil, Chevron, Royal Dutch Shell and BP drilled in deeper waters in the Gulf of Mexico, invested in Canadian oil sands and sent teams to explore in places as far-flung as the Arctic Circle, Azerbaijan and Equatorial Guinea.

Investors assumed that fossil-fuel reserves could only become more valuable over time. With energy demand rising in China, India and other developing countries, oil prices rose to over $140 a barrel, and oil stocks soared.

That all changed when companies like Mitchell Energy, EOG Resources and Chesapeake Energy began experimenting with drilling for oil and gas in shale fields. Not only did that reverse the production decline in the United States, but it also encouraged Wall Street investors to shower money on the industry. In 2014 alone, investors lent North American exploration and production companies more than $17 billion, according to IHS Markit, a consultant.

Shale production upended the global oil market once dominated by OPEC, and oil prices fell sharply.

Energy experts say annual global oil demand is growing about 1.2%, which is not enough for American oil companies to produce more oil profitably. Demand was growing roughly three times as fast in the early 2000s, but greater fuel efficiency and an economic slowdown in China have reduced the world’s thirst for oil.


That slowdown has been particularly hard for smaller companies not lucky or smart enough to invest in the most productive fields. Because shale wells decline rapidly, executives also liken this business to running on a treadmill that never stops — they need to drill new wells constantly to maintain production and continue to generate revenue.

“It’s a race,” said Raoul LeBlanc, director for energy at IHS Markit. “But there’s not enough money to grow aggressively and give money back to shareholders at the same time.”

The race is all the more arduous with oil prices hovering around $60 a barrel after rising above $75 last year.

Rystad Energy, a consulting firm, found that 36 of the 40 shale oil companies it looked at in the first quarter of this year could not generate enough revenue to sustain their businesses, reduce debt and reward their investors with dividends or share buybacks.

Energy experts say oil companies could make a comeback in the next few years, even if it is only a temporary one. Advanced technologies could bring down costs of production while increasing output. Production in countries like Venezuela and Mexico could continue declining, allowing American companies to gain market share.

But the industry faces a long-term challenge from climate change.

Chevron and other large American oil companies are investing in carbon capture and sequestration to bury greenhouse gases or produce new fuels, though such technologies are expensive and unproved on a large scale.


Large oil companies, which generate a lot of cash and still pay investors healthy dividends, can afford such investments. But many smaller businesses, including those that pioneered shale drilling, don’t have the money or technical expertise to make similar changes.

Since last year, investors have called on companies to cut expenses, reduce debt and pay higher dividends or buy back more shares. In response, the industry has dropped the number of rigs it commissions to drill new wells in recent weeks.

Scott Sheffield, chief executive of Pioneer Natural Resources, a major producer in the Permian Basin, says a major industry consolidation may be coming.

“Most of the companies,” he said, “can’t deliver what the investors are asking for.”