Eco-Friendly AI: How to Reduce the Carbon and Water Footprints of Your ML Models

Author:Murphy | View: 27748 | Time: 2025-03-22 20:59:28

AI and Sustainability: It's time to get serious.

As we push the boundaries of AI, especially with generative models, we are confronted with a pressing question that is forecasted to only become more urgent: What is the environmental cost of our progress? Training, hosting, and running these models aren't just compute-intensive – they require substantial natural resources, leading to significant carbon and water footprints that often fly under the radar. This discussion has become even more timely with Google's recent report on July 2, 2024, highlighting the challenges in meeting their ambitious climate goals. The report revealed a 13% increase in emissions in 2023 compared to the previous year and a 48% rise compared to their baseline year of 2019. The demand for AI has significantly strained data centers, a trend reflected in Microsoft's environmental sustainability report from May, which noted a 29% increase in emissions above their 2020 baseline due to data center usage. Additionally, the International Energy Agency predicts that global data center and AI electricity demand could double by 2026, underscoring the urgent need for sustainable practices. For everyone involved, from developers and technical project managers to CTOs and stakeholders, considering the environmental impact at every level – from individual cloud setups to broad governance policies – is essential. This article explores how each of us can start today to contribute to addressing these vital issues and adopting sustainable practices to reduce AI's environmental footprint.

The Carbon and Water Demand of AI

Carbon Footprint

Training large AI models is an energy-intensive process, consuming vast amounts of electricity which, depending on the energy source, can result in high carbon emissions. According to The Shift Project, Cloud Computing now accounts for 2.5–3.7% of global greenhouse gas (GHG) emissions – surpassing the commercial airline industry(!) This footprint is set to grow as both 1) the demand for AI capabilities and 2) the computational power for a given AI-based task increase. This carbon footprint encompasses two primary aspects:

Hardware Production: The manufacturing and supply chain of hardware contribute to "embodied carbon" emissions. The entire lifecycle of hardware, from raw material extraction to production, transport, and disposal, involves significant carbon output. As AI demands grow, so does the need for more (and more of) advanced hardware, exacerbating this footprint.
Compute Energy: This includes the energy required for training and inference of models. Different stages of model development, from pretraining to fine-tuning and real-time inference, have varying energy needs. Training a state-of-the-art natural language processing model can emit as much carbon dioxide as five cars over their lifetimes, according to a study by Strubell et al.

Water Footprint

Water is an often-overlooked component in the environmental impact of AI. Data centers consume substantial water for cooling servers to prevent overheating. According to a study by Li et al., training GPT-3 in Microsoft's U.S.-based data centers can directly consume up to 700,000 liters of clean freshwater (via evaporation). Furthermore, the global AI demand for water is projected to be between 4.2 and 6.6 billion cubic meters by 2027 – equivalent to the total annual water withdrawal of Denmark or half of the United Kingdom. This alarming trend underscores the urgent need for AI models to address their water footprint and lead by example in sustainable practices. The water footprint includes:

Direct Use: Water used in the cooling systems of data centers. Efficient cooling is critical to maintaining server performance and longevity, but traditional methods can be water-intensive.
Indirect Use: Water used in the production of electricity that powers these data centers. Power generation, especially from thermoelectric sources, requires significant quantities of water for cooling and steam generation. This indirect usage often actually surpasses the direct consumption of water by data centers themselves.

Strategies to Reduce Carbon and Water Footprints

1. Carbon-Aware Computing

Carbon-aware computing involves optimizing when and where to run ML jobs based on the carbon intensity of the electricity grid. By leveraging data on carbon intensity, developers can schedule tasks in regions and times where renewable energy sources are more available.

For example, there are open-source tools and APIs (or both in the case of Electricity Maps) to identify regions with the lowest carbon intensity and schedule training jobs accordingly. This approach, known as "carbon-aware development," ensures that the energy used for model training has a lower carbon footprint. By dynamically adjusting the workload distribution based on real-time carbon intensity data, significant reductions in carbon emissions can be achieved. Implementing such strategies requires integrating carbon intensity data into the job scheduling systems of cloud services, enabling automatic adjustments based on predefined thresholds.

A free, short course called "Carbon Aware Computing for GenAI Developers" is offered by DeepLearning.AI which I highly recommend. It's Google Cloud-focused, but many of the cloud platform-specific tools are available in similar forms for other providers.

Implementing Carbon-Aware Strategies

Dynamic Scheduling: Develop scripts and automated workflows that use APIs to fetch real-time carbon intensity data. When best written, these scripts should be able to schedule or migrate tasks to data centers with lower carbon intensity at specific times.
Location-Based Deployment: Identify – and prefer deploying workloads in – geographic regions powered predominantly by renewable energy sources. Regions with high ratios of wind, solar, or hydroelectric power supporting their grids should be prioritized.
Use of Spot Instances: Utilize cloud provider spot instances, which are typically available at lower costs and during off-peak hours…a happy collision of price and environmental consciousness.

2. Efficient Training Strategies

Efficiency in model training is another target that can simultaneously reduce both costs and carbon/water footprints. Techniques and strategies to enhance efficiency can be found far and wide, but some key ones are below.

Model Architecture and Pre-Training Strategies

Model Distillation: Compressing larger models into smaller, more efficient ones without sacrificing performance. This process reduces the computational requirements for both training and inference. By creating a more compact model, the energy consumed during these phases can be minimized.
Transfer Learning: Taking advantage of pre-trained models and fine-tuning them for specific tasks rather than training from scratch. Transfer learning significantly cuts down on the computational resources needed, as the base model has already undergone extensive training. This method also speeds up the development process.
Hyperparameter Optimization: Efficiently tuning hyperparameters to reduce the number of training runs required. Automated hyperparameter tuning methods, such as Bayesian optimization, can find optimal settings faster and with fewer resources. This approach reduces the trial-and-error phase, saving both time and energy.

Training-Specific Strategies

Adaptive Learning Rates: Use adaptive learning rate algorithms that adjust based on the training progress. This method assists efficient use of computational resources and faster convergence.
Early Stopping: Implement early stopping criteria to halt training once the model performance stops improving. This technique prevents unnecessary training epochs, saving energy and computational resources.
Parameter Sharing: Share parameters across similar tasks or models to reduce redundant computations. Techniques like multi-task learning can leverage shared representations, reducing the overall training burden.

3. Sustainable Hardware Choices

Selecting hardware with lower embodied carbon and better energy efficiency can reduce the overall carbon footprint. Consider:

Using AI-Specific Chips: AI accelerators, such as TPUs (Tensor Processing Units) and GPUs (Graphics Processing Units), are designed for efficient AI computation. These chips perform AI tasks more efficiently than general-purpose CPUs, reducing energy consumption.
Energy-Efficient Servers: Opting for servers that employ advanced cooling technologies to minimize water usage and enhance energy efficiency. Innovations in liquid cooling and immersion cooling can significantly reduce the energy required for cooling data centers.

Implementing Sustainable Hardware Choices

Lifecycle Assessment: Conduct a thorough lifecycle assessment of hardware to understand and mitigate embodied carbon emissions. Choose suppliers and products with transparent and lower carbon footprints.
Upgrading and Recycling: Regularly upgrade to more energy-efficient hardware if possible, and responsibly recycle outdated equipment. Bonus points if you can partner with recycling programs that ensure proper disposal and recovery of valuable materials.

Integrating Stakeholders in Sustainability Efforts

Stakeholders, clients, and others indirectly involved in ML work also play a critical role in driving Sustainability efforts. Setting clear expectations regarding sustainability benchmarks – in the same conversations that performance, speed, and cost are discussed – is essential for achieving environmental goals and ensures these considerations don't fall through the cracks.

Strategies for Stakeholder Engagement

Establish Sustainability Benchmarks: Define and communicate clear sustainability benchmarks for projects. These should be integrated into project planning and evaluation criteria.
Client Education: Educate clients and stakeholders on the environmental impacts of AI/ML projects and the importance of sustainability. Provide resources and examples of sustainable practices.
Collaborative Goal Setting: Involve stakeholders in setting and reviewing sustainability goals. This collaborative approach helps with buy-in and accountability from all involved parties.

Tools to Optimize Your Workload's Footprint

Selecting the best region for running cloud-based AI workloads can significantly impact the environmental footprint. Tools and APIs like Electricity Maps provide valuable insights into the carbon intensity of consumed electricity in specific regions, enabling informed decisions for both cloud and on-prem deployments.

Using Electricity Maps API

Real-Time Carbon Intensity Data: The Electricity Maps API offers real-time data on the carbon intensity of electricity in various regions. Developers can use this data to optimize job scheduling and location selection.
Integration with CI/CD Pipelines: Integrate carbon intensity data into continuous integration/continuous deployment (CI/CD) pipelines to automate the selection of low-carbon regions for deployments.

Here is an example implementation of this, using the Electricity Maps API. Let's look at which of our favorite cities have the lowest carbon intensity. This script fetches real-time carbon intensity data for a list of cities (and their corresponding lat/long coordinates), allowing us to identify the city with the lowest carbon footprint. By running computational tasks in these low-carbon regions, we can make more environmentally conscious decisions and reduce the overall carbon footprint of our operations.

import requests
import json
import logging

# Configure logging (I always do this for API calls to see what exactly might be buggy if there's an error. Omit the logging and try/except steps if you want.)
logging.basicConfig(level=logging.ERROR)

def carbon_intensity(lat, lon, API_KEY):
    url = f"https://api.electricitymap.org/v3/carbon-intensity/latest?lat={lat}&lon={lon}"
    try:
        request = requests.get(url, headers={"auth-token": API_KEY})
        request.raise_for_status()  # Check for HTTP request errors
        response = json.loads(request.content)
        if "carbonIntensity" not in response:
            raise LookupError(f"Carbon intensity data not available from Electricity Maps for coordinates (lat: {lat}, lon: {lon}).")
        carbon_intensity_value = int(response["carbonIntensity"])
        return carbon_intensity_value
    except requests.exceptions.RequestException as e:
        logging.error(f"Error fetching data for coordinates (lat: {lat}, lon: {lon}): {e}")
        raise
    except json.JSONDecodeError:
        logging.error(f"Error decoding JSON for coordinates (lat: {lat}, lon: {lon}). Response content: {request.content}")
        raise

def cleanest(coordinates, API_KEY):
    carbon_intensity_for_regions = []

    for name, coords in coordinates.items():
        try:
            carbon_intensity_value = carbon_intensity(coords['lat'], coords['lon'], API_KEY)
            carbon_intensity_for_regions.append({
                "name": name,
                "carbon_intensity": carbon_intensity_value
            })
        except LookupError:  # Skip over errors for individual regions.
            logging.warning(f"Could not get carbon intensity data for region {name}, so it was skipped.")
        except requests.exceptions.RequestException:
            logging.error(f"Failed to fetch carbon intensity data for region {name}, skipping.")

    if not carbon_intensity_for_regions:
        raise ValueError("No valid data retrieved for any region. Please check your API key and coordinates.")

    return min(carbon_intensity_for_regions, key=lambda x: x["carbon_intensity"])

#Sample cities of interest to compare real-time carbon intensity
coordinates = {
    'Madrid, Spain': {'lat': 40.4168, 'lon': -3.7038},
    'Tokyo, Japan': {'lat': 35.682839, 'lon': 139.759455},
    'Sydney, Australia': {'lat': -33.8688, 'lon': 151.2093},
    'São Paulo, Brazil': {'lat': -23.5505, 'lon': -46.6333},
    'Toronto, Canada': {'lat': 43.651070, 'lon': -79.347015}
}
#Get your (free) API key from https://api-portal.electricitymaps.com/
api_key = 'your_electricity_maps_api_key_here'

try:
    low_carbon_region = cleanest(coordinates, api_key)
    print(f'The region with the current lowest carbon intensity is {low_carbon_region["name"]} with a carbon intensity of {low_carbon_region["carbon_intensity"]} gCO2eq/kWh')
except ValueError as e:
    print(e)

After plugging in your API key, you should get an output of the form:

"The region with the current lowest carbon intensity is [city] with a carbon intensity of [value] gCO2eq/kWh."

For example, at 10:45 AM PST on July 2, 2024, the output was the following:

We can confirm this carbon intensity reading by checking the map directly:

The website confirms our script's output that the most recent carbon intensity data for Toronto's regional grid is 69 gCO2eq/kWh. Check this for the other regions to verify that it's also comparing across cities correctly. (Tip: comment out the region reported to have the lowest carbon intensity in the "coordinates" dictionary, and repeat this step city-by-city to get the next lowest value and confirm it's higher than the previous. Then check that city's region in the interactive map!)

Use this to help decide which region to select in your cloud environment of choice to train and host your ML models with the lowest carbon footprint.

Integrating Sustainability into AI/ML Governance

The best place for organizations is to codify their commitment to eco-friendly AI/ML workloads is by making it part-and-parcel of their governance frameworks and best practices. Here are some tips to get your org started:

Establish Carbon and Water Metrics

Monitor and Report: Regularly measure and report the carbon and water footprints of AI ops. Use tools and frameworks to monitor these metrics in real-time and visualize them through dashboards to provide actionable insights.
Act on Insights: Develop mechanisms to interpret the data collected and translate it into actionable steps for reducing environmental impact. This might ultimately lead to adjusting operational practices or switching cloud providers.

Set Sustainability Goals

Define Targets: Set specific, measurable targets for reducing the environmental impact of AI operations. These goals should align with the broader sustainability initiatives of the organization.
Review and Adjust: Regularly review the progress toward these goals and adjust strategies as needed. This could involve periodic audits and updates to sustainability plans based on the latest data and technological advancements.

Commit to Training and Awareness

Educational Programs: Conduct training programs for developers, engineers, and stakeholders on sustainable AI practices. Focus on the importance of reducing carbon and water footprints and provide practical steps for achieving this.
Awareness Campaigns: Launch awareness campaigns to highlight the significance of sustainability in AI. Use case studies and success stories to demonstrate the positive impact of sustainable practices.

Incorporate Environmental Impact into Procurement

Sustainable Procurement Policies: Develop procurement policies that prioritize sustainability. Require hardware and cloud service providers to meet stringent environmental standards. Make sure these cloud provider standards are reflected in real commitments, like Service Level Agreements or case studies.
Lifecycle Considerations: Consider the entire lifecycle of the hardware and services procured, from production and usage to disposal, ensuring minimal environmental impact throughout.

Naturally, these steps require research, and then alignment of those findings with both your geographical* and company culture toward environmental stewardship. It's certainly additional homework, but once the standards have been set and the monitoring steps are in place (or ideally, automated), your efforts will have a significant and sustainable impact.

*Geographical considerations come into play if you're running workloads on-prem and are at the mercy of your local power grid's energy sources. If cloud computing isn't an option, focus on 1) using energy-efficient servers/hardware accelerators, 2) investing in on-site renewable energy sources like solar panels to offset grid reliance, and 3) scheduling workloads outside of peak hours.

Conclusion

The environmental impact of AI is a critical consideration for the future of sustainable technology development. By adopting carbon-aware computing practices, optimizing training strategies, choosing sustainable hardware, selecting eco-friendly cloud providers, and involving stakeholders in sustainability efforts, developers and organizations alike can significantly reduce the carbon and water footprints of their AI operations. Integrating these practices into AI/ML governance ensures a holistic approach to sustainability, paving the way for a greener and more responsible AI future.

Parting Thoughts

Congratulations on making it to the end! We've dissected the looming issue of AI's environmental footprint and explored actionable strategies to balance innovation with sustainability. By integrating these practices into our workflows, we can take steps toward ensuring that our advancements in AI are not at the expense of our planet.

For those who enjoy diving into the intricacies of AI problem-solving as much as I do, follow me on Medium and LinkedIn. Together, let's navigate the AI landscape with a commitment to both innovation and environmental responsibility.

Until our next journey, keep exploring, keep learning, and keep advocating for a greener, more sustainable future in AI! And in your data science and ML endeavors, may your carbon and water footprints be ever decreasing.

Tags: Artificial Intelligence Cloud Computing Editors Pick Mlops Sustainability