Note: This answer only applies to versions of gFTP <= 2.0.17
What probably is the problem, sftp-server isn't in your path on the remote server. When you try to connect to the remote server, you'll see the command gFTP is trying to run, and it'll look something like:
ssh -e none -l username hostname "echo -n xsftp ; sftp-server"
Try running this command. If you get a prompt asking:
The authenticity of host 'hostname (###.###.###.###)' can't be established.
RSA key fingerprint is xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx:xx.
Are you sure you want to continue connecting (yes/no)?
Tell it yes. gFTP won't even try to answer this question for you at the moment. Then, if you get the following output:
Enter passphrase for key '/home/brian/.ssh/id_dsa':
xsftp
bash: sftp-server: command not found
You'll have to set the path to the sftp-server program. Alternatively you could go under FTP->Options->SSH and enable Use sftp subsys. If you enable this, then you don't have to know the remote path to your sftp executable. But if you enable this, gFTP won't be able to send your password over to the ssh client. So if you're not using ssh-agent, then ssh will spawn the ssh-askpass program to grab your password.