USB/IP has been pretty useful to me, though locking it down is a bit of a chore, as it does not natively support any type of authentication or authorization (a not unreasonable design decision).
Maybe tunnel it over a secure protocol? If possible?
Do you use it on a lan I’m guessing?