Due to neutral or adaptive genetic divergence of populations within a species, plant populations can be differentiated in their phenotypes. Divergent environmental conditions and GxE interactions can further alter the phenotypic expression of plant characteristics, including fitness. In each scenario of crop-wild gene flow, different wild populations and crop lines hybridize and the resulting hybrids may be found in different environmental conditions. These factors affect fitness and could shape the relative fitness of hybrid and wild individuals, resulting in variable amounts of crop gene introgression across hybrid zones. In our research in sunflower, the use of nine wild populations, three crop lines, and four environments helped to elucidate environmental conditions where crop gene introgression may be more likely for certain genotypes. In particular, both interspecific competition and herbicide application increased the fitness of crop-wild hybrid plants relative to wilds. These stressful conditions also increased the range of relative fitness across genetic backgrounds. These results indicate that stress should increase the likelihood of crop gene introgression and the variability in those introgression levels. Biosafety research for effects of crop-wild gene flow would benefit from incorporating diverse germplasm and diverse testing conditions. Only by doing so will we understand the conditions under which crop gene introgression is most likely.