$ tar cp -C /path/to/dir . | ssh server2 'tar x -C /path/to/target'
With a large number of files, scp or rsync can take very very long. It's much faster to tar up on one side and extract on the other. Without the
You may (or may not) gain an extra speed boost by compression, either with the z flag for tar, or with the -C flag for ssh, or with gzip pipes in the middle, like this:
Depending on your system and version of tar, you may need to hyphenate the flags, for example