how do i read a matrix into r using tidyverse tools?

In R, you need to use backslash to denote the special and escape characters.

Try

as.matrix(read_delim("matrix.txt", delim = "\t"))

The read_delim function does not have a row.names option, so you will have to transform the data to get your rownames as rownames (and not the first column). You should do this before you convert to matrix.

library(tidyverse)
matrix <- read_delim("matrix.txt", delim = "\t") %>%
  column_to_rownames(first_column) %>%
  as.matrix()