Seed development in flowering plants is a paradigm for the coordination of different tissues during organ growth. It requires a tight interplay between the two typically sexually produced structures: the embryo, developing from the fertilized egg cell, and the endosperm, originating from a fertilized central cell, along with the surrounding maternal tissues. Little is known about the presumptive signal transduction pathways administering and coordinating these different tissues during seed growth and development. Recently, a new signal has been identified emanating from the fertilization of the egg cell that triggers central cell proliferation without prior fertilization. Here, we demonstrate that there exists a large natural genetic variation with respect to the outcome of this signaling process in the model plant Arabidopsis thaliana. By using a recombinant inbred line population between the two Arabidopsis accessions Bayreuth-0 and Shahdara, we have identified two genetic components that influence the development of unfertilized endosperm. Exploiting this natural variation, we could further dissect the interdependence of embryo and endosperm growth during early seed development. Our data show an unexpectedly large degree of independence in embryo growth, but also reveal the embryo's developmental restrictions with respect to endosperm size. This work provides a genetic framework for dissection of the interplay between embryo and endosperm during seed growth in plants.