Copying module folder to target

I compiled all the modules as following command:
$ make ARCH=arm CROSS_COMPILE=arm-linux-gnueabihf- modules -j$(nproc)
I also installed all the compiled modules to the host machine directory by:
sudo make ARCH=arm modules_install

But when I send the module directory to the target with the scp command, it actually sends my entire linux source directory as well(size = 5.7G)
scp -r /lib/modules/5.4.106/ debian@…

I only want the necessary files for the module to be copied, not my entire Linux source code.

-r’ Recursively copy entire directories. Note that scp follows symbolic links encountered in the tree traversal.

This can’t be disabled with scp…