Why not just use XDR?

XDR is a specific format. The encdec functions are ideal for picking apart and building arbitrary encodings like binary file formats and network messages. Like XDR, it can be used very effectively to implement custom RPCs but that is incedental. XDR is also not available in certain environments like Java ironically. In this case encdec's open ended encoding/decoding capabilities could be used with whatever the peer encoder/decoder has available (e.g. DataInputStream and DataOutputStream).

How do encdec's string functions differ from iconv?

The encdec mbs functions are just a different interface into Bruno Haible's libiconv. They are more convenient for in-situ string manipulation.

For example, if you are given a string that you need to convert to the locale encoding (e.g. UTF-8) but you do not know how much memory the resulting string will occupy, something like the following code example could be used. It passes a NULL destination buffer to indicate to the dec_mbsncpy function that the exact number of bytes required to encode the resulting string should be returned. This value is used to allocate the correct amount of memory and the string is converted again with the dec_mbsncpy function but this time with the allocate memory instead of NULL:

  if ((n = dec_mbsncpy(&src, sn, NULL, -1, -1, "KOI8-R")) == (size_t)-1) {
      perror("Failed to determine required buf size");
      return -1;
  }
  
  buf = malloc(n);
  
  if (!buf || (n = dec_mbsncpy(&src, sn, buf, n, -1, "KIO8-R")) == (size_t)-1) {
      perror("Failed to convert string");
      return -1;
  }
  // buf is now the decoded string