The heart, the first organ to develop in the embryo, undergoes complex morphogenesis that when defective results in congenital heart disease (CHD). With current therapies, more than 90% of patients with CHD survive into adulthood, but many suffer premature death from heart failure and non-cardiac causes. Here, to gain insight into this disease progression, we performed single-nucleus RNA sequencing on 157,273 nuclei from control hearts and hearts from patients with CHD, including those with hypoplastic left heart syndrome (HLHS) and tetralogy of Fallot, two common forms of cyanotic CHD lesions, as well as dilated and hypertrophic cardiomyopathies. We observed CHD-specific cell states in cardiomyocytes, which showed evidence of insulin resistance and increased expression of genes associated with FOXO signalling and CRIM1. Cardiac fibroblasts in HLHS were enriched in a low-Hippo and high-YAP cell state characteristic of activated cardiac fibroblasts. Imaging mass cytometry uncovered a spatially resolved perivascular microenvironment consistent with an immunodeficient state in CHD. Peripheral immune cell profiling suggested deficient monocytic immunity in CHD, in agreement with the predilection in CHD to infection and cancer. Our comprehensive phenotyping of CHD provides a roadmap towards future personalized treatments for CHD.