I don’t think you will be able to overcome these times using the existing SPI HW and SW. 48 MHz is the fastest the SPI hardware can run because that is the clock signal it gets before any clock dividing. Some things I have read say not to use it at faster than /3 (16 Mhz). If you really need to run faster, program a PRU to bitbang SPI: I think you could write fixed sequences out at 100 Mbits/second, if you can manage to get 100 MHz signals off the board without too much noise and ringing. (I would put 25 ohm resistors in series with the cs, clock, and mosi signals to terminate them.) Then you can have zero time between bytes, use whatever cs hold time you need, etc.