Energy use of AI

In short: AI needs electricity: when training the models and with every request. A local AI computer costs about as much to run as a gaming PC.

In more detail: A 350 W graphics card running one hour at full load uses 0.35 kWh, about 11.5 cents at 33 ct. A single chat answer on average needs a few watt hours in the cloud, depending on model and length.

Our context: With the energy calculator you can work out what continuous operation costs per year.

In Depth

• Training: Large models need data centres with thousands of cards over weeks, in the range of gigawatt hours.
• Efficiency: Smaller, quantised models and MoE save energy per answer.
• Idle: A card at idle draws 10 to 30 W.
• Example: 150 W on average, 4 hours daily, 33 ct per kWh: 150 × 4 × 365 ÷ 1,000 × 0.33 ≈ €72 a year.

See also: Kilowatt hour (kWh), Standby consumption