It’s been a while for me, but in my experience the shaders will only compile again if you make a change somewhere that requires a recompile.
One thing to note about UE4 shaders is the difference between a “Master” material and a material “Instance.”
When you create a new shader you’re creating a “Master” material, meaning they’re base level shaders. Making almost any node changes beyond scalar or vector parameter changes here will require a recompile.
However, if you create a material instance from a master material you can adjust “most” of the parameters without forcing a re-compile. That’s why, as @tharlevfx mentioned it’s always good to build Scalar and Vector parameters into your master materials so that you can make many instances from that one base shader and have control over their parameters without re-compiling.
Some things will always require a re-compile though such as Changing Blend Modes or Lit Options.
There are also some options in the menus that control re-compiling. Your issue could be as simple as a check box, but I’m not up to speed enough with the latest UE4 to be certain.