eclipse removes external jars from the build path when gradle dependencies are refreshed

Yes, that is the expected behavior: Refresh Gradle Project resets the dependencies to the ones specified in the build.gradle file. So it overwrites your manually made changes in the Java Build Path.

In a Gradle project, instead of changing the Java Build Path settings directly, add all dependencies via the build.gradle file.