Each sapling and crop in the game can only grow in its own natural biomes (driven by config file).

So by default, wheat can grow in plains but not in the nether. Chorus can only grow in end biomes. Cactus and sugar cane do not just grow anywhere.

Plants that cant grow in the current biome sit stationary for a while, and eventually drop as an item.

You can craft the Growth Detector [1.14] to use on biomes and blocks to read information in-game of how the growth settings have been defined.

Modded blocks should be no problem as long as they connect to forge in a similar way to common saplings and crops.

1.14.4 config example file

#General settings
[growthcontrols]
#Map growable block to CSV list of biomes no spaces, -> in between. It SHOULD be fine to add modded blocks. An empty list means the crop can gro nowhere. Delete the key-entry for a crop to let it grow everywhere.
CropBlockToBiome = ["minecraft:wheat->minecraft:plains,minecraft:swamp,minecraft:beach", "minecraft:carrots->minecraft:taiga,minecraft:savanna,minecraft:savanna*,minecraft:*savanna,minecraft:shattered_savanna_plateau,minecraft:sunflower_plains,minecraft:giant_tree_taiga_hills", "minecraft:potatoes->minecraft:taiga,minecraft:snowy*,minecraft:taiga*,minecraft:*forest,minecraft:dark_forest_hills,minecraft:*taiga,minecraft:*mountains,minecraft:giant_tree_taiga_hills,minecraft:mountain_edge", "minecraft:beetroots->minecraft:taiga,minecraft:forest,minecraft:swamp,minecraft:flower_forest,minecraft:birch_forest,minecraft:birch*,minecraft:tall_birch_forest,minecraft:tall_birch_hills,minecraft:dark_forest,minecraft:dark_forest_hills", "minecraft:cactus->minecraft:desert,minecraft:desert_hills,minecraft:desert_lakes,minecraft:badlands,minecraft:badlands*,minecraft:*badlands", "minecraft:sugar_cane->minecraft:desert_lakes,minecraft:stone_shore,minecraft:ocean,minecraft:river,minecraft:beach,minecraft:deep_ocean,minecraft:warm_ocean,minecraft:lukewarm_ocean,minecraft:deep_warm_ocean,minecraft:deep_lukewarm_ocean", "minecraft:chorus_flower->minecraft:small_end_islands,minecraft:end*,minecraft:the_end,minecraft:the_void", "minecraft:chorus_plant->minecraft:small_end_islands,minecraft:end*,minecraft:the_end,minecraft:the_void", "minecraft:brown_mushroom->minecraft:mushroom*,minecraft:nether,minecraft:roofed_forest*", "minecraft:red_mushroom->minecraft:mushroom*,minecraft:nether,minecraft:roofed_forest*", "minecraft:cocoa->minecraft:jungle_edge,minecraft:jungle,minecraft:jungle_hills,minecraft:modified_jungle,minecraft:bamboo_jungle,minecraft:bamboo_jungle_hills,minecraft:modified_jungle_edge,minecraft:modified_jungle", "minecraft:melon_stem->minecraft:ocean,minecraft:river,minecraft:beach,minecraft:deep_ocean,minecraft:warm_ocean,minecraft:lukewarm_ocean,minecraft:deep_warm_ocean,minecraft:deep_lukewarm_ocean,minecraft:jungle,minecraft:jungle*,minecraft:*jungle,minecraft:bamboo_jungle,minecraft:bamboo_jungle_hills,minecraft:modified_jungle_edge,minecraft:modified_jungle", "minecraft:pumpkin_stem->minecraft:extreme*,minecraft:taiga,minecraft:snowy*,minecraft:taiga*,minecraft:dark_forest_hills,minecraft:*taiga,minecraft:giant_tree_taiga_hills"]
#Drop sapling item on failed growth
dropOnFailedGrow = true
#Map growable block to CSV list of biomes no spaces, -> in between. It SHOULD be fine to add modded saplings. An empty list means the sapling can gro nowhere. Delete the key-entry for a sapling to let it grow everywhere.
SaplingBlockToBiome = ["minecraft:acacia_sapling->minecraft:savanna,minecraft:shattered_savanna,minecraft:shattered_savanna_plateau,minecraft:savanna_plateau,minecraft:modified_wooded_badlands_plateau,minecraft:wooded_badlands_plateau", "minecraft:birch_sapling->minecraft:birch_forest,minecraft:forest,minecraft:birch_forest_hills,minecraft:tall_birch_forest,minecraft:tall_birch_hills", "minecraft:spruce_sapling->minecraft:taiga,minecraft:giant_tree_taiga,minecraft:snowy_tundra,minecraft:taiga_hills,minecraft:snowy_taiga,minecraft:snowy_taiga_hills,minecraft:giant_tree_taiga_hills", "minecraft:oak_sapling->minecraft:forest,minecraft:dark_forest,minecraft:wooded_mountains,minecraft:wooded_hills,minecraft:swamp,minecraft:swamp_hills,minecraft:flower_forest", "minecraft:dark_oak_sapling->minecraft:dark_forest,minecraft:dark_forest_hills,minecraft:flower_forest", "minecraft:jungle_sapling->minecraft:jungle_edge,minecraft:jungle,minecraft:jungle_hills,minecraft:modified_jungle,minecraft:bamboo_jungle,minecraft:bamboo_jungle_hills,minecraft:modified_jungle_edge,minecraft:modified_jungle"]

Example 1.12 biomegrowthcontrol.cfg

# Configuration file

biomegrowthcontrol {
    # entry must be 'biome#list,of,sapling,item,ids'.  An empty entry for a biome means all saplings disabled in this biome.  No entry for a biome means no changes for that biome, this mod ignores it.  Biome IDs must be unique, if the same one is listed twice it might probably take the second.  Sapling meta example: 0=oak,1=spruce,2=birch,3=jungle,4=acacia,5=darkoak [default: [minecraft:hell#], [minecraft:sky#minecraft:chorus_flower,minecraft:chorus_plant,endercrop:ender_crop], [minecraft:forest#minecraft:sapling:0,minecraft:pumkpin_stem], [minecraft:forest_hills#minecraft:sapling:0,minecraft:pumkpin_stem], [minecraft:swampland#minecraft:sapling:0], [minecraft:smaller_extreme_hills#minecraft:sapling:1,minecraft:potatoes], [minecraft:extreme_hills_with_trees#minecraft:sapling:1,minecraft:potatoes], [minecraft:extreme_hills#minecraft:sapling:1,minecraft:potatoes], [minecraft:taiga#minecraft:sapling:1], [minecraft:taiga_hills#minecraft:sapling:1], [minecraft:redwood_taiga#minecraft:sapling:1,minecraft:sapling:5], [minecraft:redwood_taiga_hills#minecraft:sapling:1,minecraft:sapling:5], [minecraft:jungle#minecraft:sapling:3,minecraft:melon_stem,minecraft:cocoa], [minecraft:jungle_hills#minecraft:sapling:3,minecraft:melon_stem,minecraft:cocoa], [minecraft:jungle_edge#minecraft:sapling:3,minecraft:melon_stem,minecraft:cocoa], [minecraft:birch_forest#minecraft:sapling:2], [minecraft:birch_forest_hills#minecraft:sapling:2], [minecraft:roofed_forest#minecraft:sapling:5], [minecraft:savanna#minecraft:sapling:4], [minecraft:savanna_rock#minecraft:sapling:4], [minecraft:mesa#minecraft:sapling:4], [minecraft:mesa_rock#minecraft:sapling:4], [minecraft:mesa_clear_rock#minecraft:sapling:4], [minecraft:plains#minecraft:wheat,minecraft:carrots], [minecraft:desert#minecraft:cactus], [minecraft:desert_hills#minecraft:cactus], [minecraft:river#minecraft:reeds], [minecraft:stone_beach#minecraft:reeds], [minecraft:beaches#minecraft:reeds], [minecraft:cold_beach#], [minecraft:frozen_river#], [minecraft:ice_flats#], [minecraft:ice_mountains#], [minecraft:mushroom_island#], [minecraft:mushroom_island_shore#], [minecraft:taiga_cold#minecraft:beetroots], [minecraft:taiga_cold_hills#minecraft:beetroots], [minecraft:ocean#], [minecraft:frozen_ocean#], [minecraft:deep_ocean#]]
    S:"biome sapling map" <
        minecraft:hell#
        minecraft:sky#minecraft:chorus_flower,minecraft:chorus_plant,endercrop:ender_crop
        minecraft:forest#minecraft:sapling:0,minecraft:pumkpin_stem
        minecraft:forest_hills#minecraft:sapling:0,minecraft:pumkpin_stem
        minecraft:swampland#minecraft:sapling:0
        minecraft:smaller_extreme_hills#minecraft:sapling:1,minecraft:potatoes
        minecraft:extreme_hills_with_trees#minecraft:sapling:1,minecraft:potatoes
        minecraft:extreme_hills#minecraft:sapling:1,minecraft:potatoes
        minecraft:taiga#minecraft:sapling:1
        minecraft:taiga_hills#minecraft:sapling:1
        minecraft:redwood_taiga#minecraft:sapling:1,minecraft:sapling:5
        minecraft:redwood_taiga_hills#minecraft:sapling:1,minecraft:sapling:5
        minecraft:jungle#minecraft:sapling:3,minecraft:melon_stem,minecraft:cocoa
        minecraft:jungle_hills#minecraft:sapling:3,minecraft:melon_stem,minecraft:cocoa
        minecraft:jungle_edge#minecraft:sapling:3,minecraft:melon_stem,minecraft:cocoa
        minecraft:birch_forest#minecraft:sapling:2
        minecraft:birch_forest_hills#minecraft:sapling:2
        minecraft:roofed_forest#minecraft:sapling:5
        minecraft:savanna#minecraft:sapling:4
        minecraft:savanna_rock#minecraft:sapling:4
        minecraft:mesa#minecraft:sapling:4
        minecraft:mesa_rock#minecraft:sapling:4
        minecraft:mesa_clear_rock#minecraft:sapling:4
        minecraft:plains#minecraft:wheat,minecraft:carrots
        minecraft:desert#minecraft:cactus
        minecraft:desert_hills#minecraft:cactus
        minecraft:river#minecraft:reeds
        minecraft:stone_beach#minecraft:reeds
        minecraft:beaches#minecraft:reeds
        minecraft:cold_beach#
        minecraft:frozen_river#
        minecraft:ice_flats#
        minecraft:ice_mountains#
        minecraft:mushroom_island#
        minecraft:mushroom_island_shore#
        minecraft:taiga_cold#minecraft:beetroots
        minecraft:taiga_cold_hills#minecraft:beetroots
        minecraft:ocean#
        minecraft:frozen_ocean#
        minecraft:deep_ocean#
     >

    # If true, then whenever sapling growth is denied it tries to drop the plant as an item.  Does not work on crops or every single block. [default: true]
    B:dropBlockOnDeny=true

    # If true, lots of events and data are logged.  Useful for debugging config values and building modpacks.   [default: false]
    B:logSpamEverything=false
}

LEGACY CONFIG PRE 1.12

Below I list each tree and where they are allowed to grow. Also included are variants (for example, if I list Jungle, then of course Jungle Edge is included)

Oak Saplings
Forest

Mesa

Jungle

Swamp

Extreme Hills

Ice Plains

Roofed Forest

Dark Oak Saplings

Roofed Forest

Acacia Saplings

Savanna

Mesa

Spruce Saplings

Taiga

Extreme Hills

Jungle

Ice Plains

Birch Saplings

Forest

Birch Forest

Flower Forest

Roofed Forest

Jungle Saplings

Jungle

In the newest versions, all of the above biomes are listed in the config file, so you can customize this any way you wish, and also use the Nether/ The End / and biomes from mods. See the config file for details.

Bonus Feature

This mod also tries to spawn sapling items that are about to despawn, if they are on grass/dirt. This end up mimic-ing natural forest growth, if you chop down trees but the leaves despawn naturally, and then trees just grow. This is also controllable in the config file

Biome Growth Controls