[JSch-users] SFTP Performance

[Elise Atkins]

I have been using jsch's sftp functions to get directory listings on a
server in the same subnet. I have noticed that when the number of files
is large the directory listing can take longer then expected. As an
example if the number of files is 15,000 it can take 8-10 seconds to
return the listing. In addition just doing a listing with a filename (no
wildcards) takes almost the same amount of time.

Have you seen this behavior before and is there a solution to improve
the speed when a directory contains a large number of files?

Elise

[Keith Richardson]

Hi Elise,

Can you try doing the same listings using openssh or another implementation of SFTP? Does it still take a long time?

This will help identify if the problem is in JSch or somewhere else

Thanks,
Keith Richardson


Sent from my iPhone

Post by Elise Atkins
I have been using jsch's sftp functions to get directory listings on a
server in the same subnet. I have noticed that when the number of files
is large the directory listing can take longer then expected. As an
example if the number of files is 15,000 it can take 8-10 seconds to
return the listing. In addition just doing a listing with a filename (no
wildcards) takes almost the same amount of time.
Have you seen this behavior before and is there a solution to improve
the speed when a directory contains a large number of files?
Elise
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
JSch-users mailing list
https://lists.sourceforge.net/lists/listinfo/jsch-users

[Elise Atkins]

Hi Keith,

Thanks for the quick response.

I used the linux sftp client. I have a directory with 15000 files on it.
The time to do a directory listing (ls -l) are fairly close using sftp
client and jsch, about 8 seconds.

The real difference is when a single file name is specified in the ls.
In sftp client: ls -l file_40
In jsch: ChannelSftp.ls(file_40);

Using linux sftp client, the listing of the one file returns
immediately. Using jsch's sftp, the listing takes 7.5 seconds.

I am using jsch-0.1.51. When I looked at the ls(String path,
LsEntrySelector selector) method ChannelSftp.java, it looks like the
whole directory is read. There are some comments indicating that if the
longname is known then opendir call can be skipped but the code is
commented out and may be incomplete.

Elise

Post by Keith Richardson
Hi Elise,
Can you try doing the same listings using openssh or another implementation of SFTP? Does it still take a long time?
This will help identify if the problem is in JSch or somewhere else
Thanks,
Keith Richardson
Sent from my iPhone

Post by Elise Atkins
I have been using jsch's sftp functions to get directory listings on a
server in the same subnet. I have noticed that when the number of files
is large the directory listing can take longer then expected. As an
example if the number of files is 15,000 it can take 8-10 seconds to
return the listing. In addition just doing a listing with a filename (no
wildcards) takes almost the same amount of time.
Have you seen this behavior before and is there a solution to improve
the speed when a directory contains a large number of files?
Elise
------------------------------------------------------------------------------
"Accelerate Dev Cycles with Automated Cross-Browser Testing - For FREE
Instantly run your Selenium tests across 300+ browser/OS combos.
Get unparalleled scalability from the best Selenium testing platform available
Simple to use. Nothing to install. Get started now for free."
http://p.sf.net/sfu/SauceLabs
_______________________________________________
JSch-users mailing list
https://lists.sourceforge.net/lists/listinfo/jsch-users