Background: Converging evidence suggests that the lateral and medial orbitofrontal cortices (lOFC and mOFC) may contribute distinct neural mechanisms in depression. This study investigated the relations of their functional and structural organizations with postnatal maternal depressive symptoms in young children. Methods: Resting-state functional magnetic resonance imaging and structural magnetic resonance imaging were acquired in children at age 4 (n = 199) and 6 years (n = 234). Child's withdrawal behavior problems were assessed using Child's Behavior Checklist. Results: In 4-year-old girls, postnatal maternal depressive symptoms were positively associated with the lOFC functional connectivity with the visual network but negatively with the cognitive control network. The lOFC functional connectivity with the visual network and cerebellum, which was influenced by postnatal maternal depressive symptoms, was also associated with child's withdrawal behavior problems in 6-year-old girls. Moreover, postnatal maternal depressive symptoms were also negatively associated with the mOFC functional connectivity with the cognitive control and motor networks in 4-year-old girls. Furthermore, postnatal maternal depressive symptoms influenced the structural connectivity of left mOFC with the right middle frontal cortex and left inferior temporal cortex in 4-year-old girls. Unlike girls, boys showed that postnatal maternal depressive symptoms selectively impacted the mOFC functional connectivity with the memory system at age 6 years. Conclusion: Our study provided novel evidence on the distinct neural mechanisms of the lOFC and mOFC structural and functional organizations for intergenerational transmission of maternal depression to the offspring. Boys and girls may potentially employ different neural mechanisms to adapt to maternal environment at different timings of early life.