and simply build my own USB to serial adapter that uses pogo pins instead of pin headers. This actually wouldn’t be that much work — two hours to design the circuit, maybe an hour assembling ...

There was a time when if you bought a USB mouse, in the box was also an ... it may well have come with an adapter for a 9-pin RS232 serial port. Those mice from a decade or more ago would have ...