Chapter 07 — Sources, cleaning, and reproducibility
The atlas is built from joined country, facility, and grid records.
The analysis combines the Global Data Centers and AI Water/Electricity Usage data, Ember global and U.S. electricity releases, and supplemental power-system records.
Notebook setup
Imports and data loading
The analysis begins with a small set of Python tools and explicit local paths to the project’s source files.
Import analysis tools
Load the tabular, numerical, filesystem, and reverse-geocoding tools used to prepare the analytical datasets.
import pandas as pd
import numpy as np
from pathlib import Path
import reverse_geocoder as rg
print("pandas:", pd.__version__)Define paths and load source data
Keep source locations together, then load the cleaned facility data and the global and U.S. electricity releases into DataFrames.
power_plants_url = "./data/raw/Global_Power_Plant.csv"
us_electricity_url = (
"./data/raw/us_electricity_yearly_full_release_long_format.csv"
)
data_centers_url = "./data/raw/data_center_hybrid.csv"
global_electricity_url = (
"./data/raw/electricity_yearly_full_release_long_format.csv"
)
ember_country_panel_url = "./data/ember_country_panel.csv"
dcai_clean_path = "./data/data_center_clean.csv"
dcai_clean = pd.read_csv(dcai_clean_path)
global_electricity = pd.read_csv(
global_electricity_url, low_memory=False
)
us_electricity_data = pd.read_csv(us_electricity_url)
dc = pd.read_csv(data_centers_url)Validation / Before analysis
Data quality and limitations
The notebook checks structural consistency, geographic coverage, and whether estimated facility values are physically plausible. These diagnostics support cautious interpretation: electricity use is estimated, country averages hide local constraints, and correlation does not establish causation.
Check duplicates and country identifiers
Confirm that the electricity release has no duplicate rows and that ISO codes map consistently to country names.
duplicate_rows = global_electricity.duplicated().sum()
reference_map = (
global_electricity[["ISO 3 code", "Area"]]
.dropna()
.drop_duplicates()
)
iso_conflicts = (
reference_map.groupby("ISO 3 code")["Area"].nunique().max()
)
country_conflicts = (
reference_map.groupby("Area")["ISO 3 code"].nunique().max()
)
identifiers_are_consistent = (
iso_conflicts == 1 and country_conflicts == 1
)
print("Duplicate rows:", duplicate_rows)
print("Consistent identifiers:", identifiers_are_consistent)Audit coordinates and implied utilization
Missing coordinates affect state assignment. Implied utilization above 100% flags facility estimates that cannot be treated as directly measured operations.
print("Rows with null coordinates:", dc["Latitude"].isna().sum())
print(
dc.loc[dc["Latitude"].isna(), "Country"].value_counts()
)
dc["implied_max_mwh_day"] = (
dc["Estimated_Capacity_MW"] * dc["PUE"] * 24
)
dc["utilization"] = (
dc["Daily_Electricity_Usage_MWh"]
/ dc["implied_max_mwh_day"]
)
print(dc["utilization"].describe())
print(
"Rows above 100% utilization:",
(dc["utilization"] > 1).sum(),
)Transformation / Global analysis
Building the global country-year panel
Ember stores demand, generation, and generation shares as separate long-format observations. The notebook selects comparable country-level measures, pivots them into annual records, and joins them at a shared country-year grain.
Select comparable country-level measures
Separate fuel generation, aggregate generation shares, and electricity demand while retaining only country or economy records.
COUNTRY_LEVEL = "Country or economy"
countries = ember.loc[ember["Area type"] == COUNTRY_LEVEL].copy()
gen_fuel = countries[
(countries["Category"] == "Electricity generation")
& (countries["Subcategory"] == "Fuel")
& (countries["Unit"] == "TWh")
]
share_agg = countries[
(countries["Category"] == "Electricity generation")
& (countries["Subcategory"] == "Aggregate fuel")
& (countries["Unit"] == "%")
]
demand = countries[
(countries["Category"] == "Electricity demand")
& (countries["Variable"] == "Electricity demand")
& (countries["Unit"] == "TWh")
]Pivot and merge the annual panel
Convert long-format measures to columns, then combine demand, renewable share, clean share, fossil share, and regional identifiers by country and year.
gen_wide = gen_fuel.pivot_table(
index=["Area", "ISO 3 code", "Continent", "Ember region", "Year"],
columns="Variable",
values="Value",
).reset_index()
share_wide = share_agg.pivot_table(
index=["Area", "ISO 3 code", "Year"],
columns="Variable",
values="Value",
).reset_index()
demand_wide = demand.rename(
columns={"Value": "Demand_TWh"}
).drop(columns="Variable")
panel = share_wide.merge(
demand_wide[["Area", "Year", "Demand_TWh"]],
on=["Area", "Year"],
how="left",
)
panel = panel.merge(
gen_wide[["Area", "Year", "Continent", "Ember region"]],
on=["Area", "Year"],
how="left",
)Table 1 — Data-center state-year panel
Facility counts, estimated capacity, and daily electricity demand after coordinate-based state assignment.
Build the state-year data-center panel
Aggregate unique facilities, estimated capacity, and daily electricity use at the state-year grain.
dc_state_year = (
dc_us.groupby(["State", "Year"])
.agg(
n_facilities=("Facility_ID", "nunique"),
total_capacity_MW=("Estimated_Capacity_MW", "sum"),
total_daily_MWh=("Daily_Electricity_Usage_MWh", "sum"),
)
.reset_index()
)Table 2 — Merged state-year analytical panel
The data-center panel joined to renewable share, fossil share, total generation, and grid CO₂ intensity.
Join grid outcomes to data-center demand
Filter comparable U.S. state measures, merge on state and year, and retain the shared 2019–2025 window.
gen_merged = (
renew.merge(fossil, on=["State", "Year"])
.merge(intensity, on=["State", "Year"], how="left")
.merge(total_gen, on=["State", "Year"], how="left")
)
gen_merged = gen_merged[gen_merged["Year"].between(2019, 2025)]
full = dc_state_year.merge(
gen_merged, on=["State", "Year"], how="inner"
)Table 3 — National trend, 2019–2025
The compact annual series behind the national comparison of data-center demand and renewable share.
Build the national trend table
Aggregate daily data-center electricity use nationally and join it to the U.S. renewable-generation share by year.
national_dc = (
dc_state_year.groupby("Year")["total_daily_MWh"]
.sum()
.reset_index()
)
nat_renew = us_total[
(us_total["Category"] == "Electricity generation")
& (us_total["Subcategory"] == "Aggregate fuel")
& (us_total["Variable"] == "Renewables")
& (us_total["Unit"] == "%")
& (us_total["Year"].between(2019, 2025))
][["Year", "Value"]].rename(columns={"Value": "Renewable_pct"})
national_trend = national_dc.merge(nat_renew, on="Year")
national_trend["Data Center Demand (GWh/day)"] = (
national_trend["total_daily_MWh"] / 1000
)Table 4 — State change summary
State-level starting values, ending values, and changes used in the correlation analysis.
Calculate state changes from 2019 to 2025
Place start and end years side by side, then calculate demand growth, renewable-share change, and CO₂-intensity change.
start = full[full["Year"] == 2019].set_index("State")
end = full[full["Year"] == 2025].set_index("State")
summary = pd.DataFrame({
"DC_MWh_2019": start["total_daily_MWh"],
"DC_MWh_2025": end["total_daily_MWh"],
"Renewable_pct_2019": start["Renewable_pct"],
"Renewable_pct_2025": end["Renewable_pct"],
"CO2_intensity_2019": start["CO2_intensity"],
"CO2_intensity_2025": end["CO2_intensity"],
})
summary["DC_MWh_growth_pct"] = (
(summary["DC_MWh_2025"] - summary["DC_MWh_2019"])
/ summary["DC_MWh_2019"] * 100
)
summary["Renewable_pct_change"] = (
summary["Renewable_pct_2025"] - summary["Renewable_pct_2019"]
)
summary["CO2_intensity_change"] = (
summary["CO2_intensity_2025"] - summary["CO2_intensity_2019"]
)