Objectives: This study designed and evaluated an end-to-end deep learning solution for cardiac segmentation and quantification. Background: Segmentation of cardiac structures from coronary computed tomography angiography (CCTA) images is laborious. We designed an end-to-end deep-learning solution. Methods: Scans were obtained from multicenter registries of 166 patients who underwent clinically indicated CCTA. Left ventricular volume (LVV) and right ventricular volume (RVV), left atrial volume (LAV) and right atrial volume (RAV), and left ventricular myocardial mass (LVM) were manually annotated as ground truth. A U-Net−inspired, deep-learning model was trained, validated, and tested in a 70:20:10 split. Results: Mean age was 61.1 ± 8.4 years, and 49% were women. A combined overall median Dice score of 0.9246 (interquartile range: 0.8870 to 0.9475) was achieved. The median Dice scores for LVV, RVV, LAV, RAV, and LVM were 0.938 (interquartile range: 0.887 to 0.958), 0.927 (interquartile range: 0.916 to 0.946), 0.934 (interquartile range: 0.899 to 0.950), 0.915 (interquartile range: 0.890 to 0.920), and 0.920 (interquartile range: 0.811 to 0.944), respectively. Model prediction correlated and agreed well with manual annotation for LVV (r = 0.98), RVV (r = 0.97), LAV (r = 0.78), RAV (r = 0.97), and LVM (r = 0.94) (p < 0.05 for all). Mean difference and limits of agreement for LVV, RVV, LAV, RAV, and LVM were 1.20 ml (95% CI: −7.12 to 9.51), −0.78 ml (95% CI: −10.08 to 8.52), −3.75 ml (95% CI: −21.53 to 14.03), 0.97 ml (95% CI: −6.14 to 8.09), and 6.41 g (95% CI: −8.71 to 21.52), respectively. Conclusions: A deep-learning model rapidly segmented and quantified cardiac structures. This was done with high accuracy on a pixel level, with good agreement with manual annotation, facilitating its expansion into areas of research and clinical import.