what could be an equivalent approach to a casex statement from systemverilog in c?

Ignoring some bits can be done by masking them with bitwise AND operation.

if ((op & 0xFC) == 0x8C) {
    /* 8'b100011xx */
} else if ((op & 0xFF) == 0xAC) {
    /* 8'b10101100 */
} else if ((op & 0xFF) == 0x70) {
    /* 8'b01110000 */
} else if ((op & 0xF0) == 0x90) {
    /* 8'b1001xxxx */
}

If you will stick to use switch statement, you can use multiple levels of ones.

switch (op & 0xF0) {
    case 0x80: /* 8'b1000xxxx */
        switch (op & 0x0C) {
            case 0x0C: /* 8'bxxxx11xx (8'b100011xx) */
                break;
        }
        break;
    case 0xA0: /* 8'b1010xxxx */
        switch (op & 0x0F) {
            case 0x0C: /* 8'bxxxx1100 (8'b10101100) */
                break;
        }
        break;
    case 0x70: /* 8'b0111xxxx */
        switch (op & 0x0F) {
            case 0x00: /* 8'bxxxx0000 (8'b01110000) */
                break;
        }
        break;
    case 0x90: /* 8'b1001xxxx */
        break;
}