how do you write the final dense layer to accept xy tuples?

Try this:

import tensorflow as tf
from tensorflow.keras.layers import Input, Dense, Reshape
from tensorflow.keras import Sequential

model = Sequential([
  Reshape((128, -1), input_shape=(128, 60, 41, 2)),
  Dense(3)
])

inp = tf.random.uniform([10, 128, 60, 41, 2], dtype=tf.float32)
labels = tf.random.uniform([10, 128], 0, 3, dtype=tf.int32)
pred = model(inp)
model.compile(optimizer='adam', loss='sparse_categorical_crossentropy')
model.fit(inp, labels)