Purpose: The purpose of this paper was to determine the architecture of the collagen fibers of the peripapillary sclera, the retinal nerve fiber layer (RNFL), and Henle’s fiber layer in vivo in 3D using polarization-sensitive optical coherence tomography (PS-OCT). Methods: Seven healthy volunteers were imaged with our in-house built PS-OCT system. PS-OCT imaging included intensity, local phase retardation, relative optic axis, and optic axis uniformity (OAxU). Differential Mueller matrix calculus was used for the first time in ocular tissues to visualize local orientations that varied with depth, incorpo-rating a correction method for the fiber orientation in preceding layers. Results: Scleral collagen fiber orientation images clearly showed an inner layer with an orientation parallel to the RNFL orientation, and a deeper layer where the collagen was circularly oriented. RNFL orientation images visualized the nerve fibers leaving the optic nerve head (ONH) in a radial pattern. The phase retardation and orientation of Henle’s fiber layer were visualized locally for the first time. Conclusions: PS-OCT successfully showed the orientation of the retinal nerve fibers, sclera, and Henle’s fiber layer, and is to the extent of our knowledge the only technique able to do so in 3D in vivo. Translational Relevance: In vivo 3D imaging of scleral collagen architecture and the retinal neural fibrous structures can improve our understanding of retinal biomechanics and structural alterations in different disease stages of myopia and glaucoma.