[JSch-users] Transferring large number of small files

[Luís Lobo]

Hi all.

I'm trying to transfer a large number of small files to a remote server (with reasonable latency) using JSch.

The code I'm using creates a Session and a ChannelSftp and then iterates over the local files (some times thousands) to transfer them to the server, with something like (I omitted exception handling for simplicity):

Session session=...;
ChannelSftp sftp=(ChannelSftp)session.openChannel("sftp");
for(Path path:...){
sftp.put(path.toString(), String.format("/destination/%s",path.getFileName().toString()));
}
sftp.disconnect();
session.disconnect();

The problem is this is very slow (sometimes it takes hours) compared with the something like:

rsync -avu /local/<some glob> server:/destination/ # takes seconds

Is there any faster way to transfer large number of small files with JSch?

Regards,
LL

[Luís Lobo]

(now with correct encoding... sorry)
Hi all.

I'm trying to transfer a large number of small files to a remote server (with reasonable latency) using JSch.

The code I'm using creates a Session and a ChannelSftp and then iterates over the local files (some times thousands) to transfer them to the server, with something like (I omitted exception handling for simplicity):

Session session = ...;
ChannelSftp sftp = (ChannelSftp) session.openChannel("sftp");
for(Path path : ...){
sftp.put(path.toString(), String.format("/destination/%s",path.getFileName().toString()));
}
sftp.disconnect();
session.disconnect();

The problem is this is very slow (sometimes it takes hours) compared with the something like:

rsync -av /local/<some glob> server:/destination/ # takes seconds

Is there any faster way to transfer large number of small files with JSch?

Regards,
LL