how can i implement string conversion from base64 to hex in java encoding with given js implementation?

Try below – You need to decode, not encode. Consider only value from decoded, ignore sign part. i.e take only absolute value, using Math.abs().

           private static String base64ToHex(String input) {
            byte[] raw = Base64.getDecoder().decode(input.getBytes());
            String result = "";
             for (int i = 0; i < raw.length; i++) {
                  String hex = Integer.toString(Math.abs(raw[i]), 16);
                  result += (hex.length() == 2 ? hex : '0' + hex);
              }
            return result.toUpperCase();
          }