The hidden cost of AI is growing as the AI models get larger. The most overlooked cost item is the environmental cost. The carbon footprint from training an AI model and running inference once AI after the deployment are significant items contributing to the environmental cost of AI.
The environmental cost of AI
Researchers have been working on the environmental cost of AI for a while. One study from 2019 shows that training a Transformer model with 213M parameters emits more than 626,000 pounds of carbon dioxide, equivalent to the emission of five cars, including their manufacturing process. Yet, today, we’re talking about billions of parameters, not millions. Google’s PALM (Pathways Language Model) has 540 Billion parameters. OpenAI’s GPT-3 uses 175 billion parameters, requiring 800GB to store. They stopped disclosing specific information about models with GPT-4.
Training is only a part of it. NVIDIA estimates training is only 10 to 20 percent of the consumption while the inference is the rest. Yet, it’s difficult to comment without knowing the volume of the data processed. As the technology gains mainstream traction, the inference volume increases, and the ratio decreases. Considering tens of millions of ChatGPT users, its inference cost is much greater than the training cost.
Like the environmental cost, the financial cost of large models is significant. Enterprises need cloud computing resources or hundreds of millions of dollars to train and run large models. That’s why almost all large AI models are generated by Big Tech or companies backed by Big Tech. The competition among Big Tech pushes everyone to create bigger models and bear all the costs to dominate the market. However, even for them, this approach is not sustainable. Microsoft has just started talking about building smaller AI models. While prominent economists discuss the potential threats of AI oligopoly, even they omit its environmental cost.
What’s Green AI?
AI has various benefits: user experience improvements, productivity, and efficiency gains... There is no value in stopping its advancement and usage, but we must be mindful of its costs. Green AI focuses on minimizing the environmental cost of AI by sustainably developing, deploying, and using artificial intelligence. Some AI vendors, like Picovoice, work on AI models that are environmentally sustainable to build, deploy, and use by replacing power-hungry approaches with efficient models.
ESG Best Practices for Green AI
ESG stands for Environmental, Social, and Governance (ESG). It measures the sustainability efforts of enterprises from a holistic view, encouraging enterprises to screen investments based on corporate policies and act responsibly. Considering the rapid adoption of AI, developing strategies addressing topics like ethical AI and Green AI becomes crucial for enterprises. We gathered three tips for all enterprises to make AI greener:
1. Fine-tune Existing AI Models. Don’t Train from Scratch:
Training AI models is complex, requiring machine learning expertise and significant investment in R&D - from data collection to several trials and errors. Creating and training models requires enormous amounts of energy. If there is no innovative angle or ambitions to sell proprietary technology, training models from scratch will only be a cost item, both environmentally and financially.
Most enterprises need an AI model that responds to their use case and is aware of their business. Fine-tuning existing models on their own data would be sufficient. Fine-tuning AI models can help enterprises achieve ESG goals and provide more value than generic models.
2. Choose Efficient Models and Methods:
Using less computationally expensive approaches to train, deploy, and run AI models with no quality or accuracy compromise is possible. Vendors should apply compression and distillation techniques. Buyers should research and compare the alternative AI models to find the most efficient model to achieve the desired outcome. Picovoice has been applying deep neural network compression techniques since 2018, cutting the cloud-dependency, i.e., high computational power requirements, achieving higher accuracy than large models provided by Big Tech, ensuring user privacy by design as user data doesn’t need to be sent to a 3rd party server.
3. Include AI Activity in Carbon Monitoring:
Developers generally run AI models via API calls without having any information about the environmental impact of their actions. Hence, it becomes invisible and gets ignored. To increase awareness, vendors should communicate the computational requirements of running their AI models or allow developers to measure and benchmark them. However, vendors may not do it without a demand. As mentioned above, OpenAI stopped disclosing details about their models. It’s not just about the size. The hardware type and location that the model runs on and the duration it takes to run inference also affect the cost. Transmitting data across regions requires more power and gets costlier. Thus, enterprises should demand transparency from their vendors.
What’s next?
Most enterprises evaluate AI models based on accuracy, performance, and cost - which are relevant and needed. However, ethical and environmental considerations should also be a part of evaluation processes to make the right and sustainable decisions. If you need help with the environmental cost of AI models and adding AI to your ESG strategy, talk to Picovoice experts. We need a habitable planet to benefit from AI and other technological advances.
Consult an Expert
