Background and Aims: The endoscopic evaluation of narrow-band imaging (NBI) zoom imagery in Barrett's esophagus (BE) is associated with suboptimal diagnostic accuracy and poor interobserver agreement. Computer-aided diagnosis (CAD) systems may assist endoscopists in the characterization of Barrett's mucosa. Our aim was to demonstrate the feasibility of a deep-learning CAD system for tissue characterization of NBI zoom imagery in BE. Methods: The CAD system was first trained using 494,364 endoscopic images of general endoscopic imagery. Next, 690 neoplastic BE and 557 nondysplastic BE (NDBE) white-light endoscopy overview images were used for refinement training. Subsequently, a third dataset of 112 neoplastic and 71 NDBE NBI zoom images with histologic correlation was used for training and internal validation. Finally, the CAD system was further trained and validated with a fourth, histologically confirmed dataset of 59 neoplastic and 98 NDBE NBI zoom videos. Performance was evaluated using fourfold cross-validation. The primary outcome was the diagnostic performance of the CAD system for classification of neoplasia in NBI zoom videos. Results: The CAD system demonstrated accuracy, sensitivity, and specificity for detection of BE neoplasia using NBI zoom images of 84%, 88%, and 78%, respectively. In total, 30,021 individual video frames were analyzed by the CAD system. Accuracy, sensitivity, and specificity of the video-based CAD system were 83% (95% confidence interval [CI], 78%-89%), 85% (95% CI, 76%-94%), and 83% (95% CI, 76%-90%), respectively. The mean assessment speed was 38 frames per second. Conclusion: We have demonstrated promising diagnostic accuracy of predicting the presence/absence of Barrett's neoplasia on histologically confirmed unaltered NBI zoom videos with fast corresponding assessment time.