Setup another local network with a router/AP/with DNS that is air gapped and test between big box and target only. Many issues can be going on if this is on a network with many appliances attached. Highly doubt its the BBB or your Host.
Problem is not speed, problem is that im working on the beaglebone over bbb and sometimes freeze for some seconds, this dont happen if i work over it directly (non over ssh), so seems to me that is something related to the ethernet connection.
I will check puting host and bbb on a separate router