Modern large language models (LLMs) such as ChatGPT and its variants crunch through hundreds of billions, if not trillions, of parameters as they search for answers.

As these models scale, the strain on the underlying infrastructure becomes impossible to ignore.

“With the rise of AI, demands on IT infrastructure, including storage, power, and cooling, are rapidly intensifying,” Rich Gadomski, director of Channel Sales and New Business Development at FUJIFILM North America Corp., told TechRepublic. “These demands create pressure to control costs, reduce energy consumption, and minimize carbon footprints.”

This has raised serious concerns about energy and water usage. OpenAI CEO Sam Altman recently tried to allay such fears by saying the average query uses 10 times less energy than previously estimated: 0.34 watt-hours and 0.000085 gallons of water.

But with 2.5 billion queries per day, that’s almost a billion watt-hours, enough to power a million homes for one hour. That daily volume also uses just shy of a quarter of a million gallons of water. This is a lot of power and water at a time when utilities and nations can ill afford it.

The question now is whether smarter data strategies can keep AI’s resource demands from spiraling out of control.

Tiering to lower energy usage

One effective strategy to lower energy and water usage is to establish tiers of data.

Take the case of an organization setting up an internal LLM. These models are typically much smaller than those like ChatGPT. But a structure can be established in which specific data is always accessible via queries, whereas other data might be on a lower, slightly slower data tier.

Perhaps all data from the last two years is immediately available, as well as other key data related to the organization. Everything else is relegated to a lower tier. If a specific query targets data in the lower tier or archive, the answer might be delayed by only a few minutes.

“Organizations must shift from short-term, reactive data projects to strategic, sustainable data architectures,” said Gadomski, who is also co-chairperson of the Active Archive Alliance, which just published a report on preparing for the AI storage challenge.

He advocates active archiving as a solution for intelligent data management in the AI era. The latest architectures for archives are transforming them from dusty repositories of old and rarely accessed data into more dynamic resources that can add value to the organization beyond compliance.

What was once considered obsolete or purely historical data can have high value if AI models can access it to uncover insights that had previously been overlooked.

Modern archives are fast

According to Gadomski, at least 80% of digital data can be classified as either low-activity or completely inactive.

If an AI engine query has to wade through all of that for every query, it means 80% wastage of power and water. By holding that data in an active archive until it is needed for a specific query, a significant dent can be made in the AI resource usage problem.

For some organizations, a sophisticated tiering system would span in-memory storage, SSDs, disk, the cloud, tape, and optical storage. As you descend the data hierarchy, access times diminish. But even data stored in online, automated tape systems can be accessed in minutes with the latest active archive software.

“An active archive addresses the limitations of traditional physical archives, providing fast access times to secondary (archival) storage systems,” said Gadomski. “This is an advantage for AI-related workflows.”

Reducing the AI energy and water bill

Organizations wishing to align their AI goals with ongoing sustainability programs are advised to pay careful attention to the number of parameters their LLMs need to access at all times and which parameters can be held close at hand for times when that data is essential to query resolution.

“Data in an active archive is always online and easily accessible, allowing for quick retrieval and analysis when needed, conserving expensive, high-performance, energy-intensive primary storage,” concluded Gadomski.

Elsewhere, Salesforce’s new three-pillar water plan for data centers shows how hyperscalers are tying AI growth to stricter efficiency and conservation goals.