Use F3 + A to manually refresh the display lists, and the GPU memory should drop. This mod will also automatically refresh the display lists by default. (You can disable it by editing redisplaylist.cfg)
Tested on my computer (RX 580 + i5-8400)
Main function source code is copied from @TechoMan from AMD Community https://community.amd.com/t5/opengl-vulkan/high-vram-consumption-in-minecraft/m-p/607299
I added the auto refresh feature. It can detect your GPU free memory and trigger the main function.