Difference between revisions of "Seon Enterprise - Fetch files from (S)FTP server"

Revision as of 10:14, 27 May 2014

Under construction!!!

Seon offers an easy way to create Seon Enterprise receive jobs from FTP server content. This solution is based on two mechanisms:

• Mount remote server directory as a local directory
• Configure the Seon Directory Scanner to that mounted directory

This documentation covers all technical aspects to implement a functionality to automatically fetch new files from an (S)FTP server.

Assumption

All details explained here are based on the freely available pre-installed Seon VMware virtualized image, which is also available for other virtualization solutions via OVA. In general, all solutions explained here can be used in any modern Linux environment. All steps explained here must be executed as user "root" unless any other documentation states to switch user context.

In this example, the following attributes are used:

• User running Seon: "www-data", group "www-data".
• Target mount point for FTP directory: /mnt/ftp/server1
• Target FTP server: 192.168.20.71, username "seon", password "seon"

Install required packages

You need to install the following packages for (S)FTP mounting:

apt-get -y install sshfs curlftpfs

Change user group membership

The user running Seon (configured in "Configuration" -> "Daemon" -> "Run Seon programs as user") must be added in the user group "fuse":

adduser www-data fuse

Change permissions of /dev/fuse

By default, the required device file "/dev/fuse" is only writable by user "root". We need to extend the permissions:

chgrp fuse /dev/fuse
chmod g+rw /dev/fuse

Create target mountpoint

The FTP target directory must be mounted somewhere into the local filesystem to be readable by Seon. You may use any (in best case empty) directory. We need this directory name later for the configuration of the Seon directory scanner. The owner of the target directory must be the configured user running Seon):

mkdir -p /mnt/ftp/server1
chown www-data.www-data /mnt/ftp/server1

Save FTP credentials securely

In order to automatically connect to the (S)FTP server, save the credentials in a single line in the following file:

/root/.netrc

The syntax of the file is simple: per line, one server can be given by its name (hostname or IP), followed by keywords for username and password, with their values. Example:

machine 192.168.20.71 login seon password seon

This file must have permissions to be readable only by root, so you might change the permissions after creating / modifying the file:

chmod 600 /root/.netrc

Add server mount for bootup

Many situations for different FTP servers may occur. Some common situations are documented here. To let the (S)FTP server be mounted at bootup (which is the most common way), you have to add a line to the filesystem table file:

/etc/fstab

You have to change the IP address and mount point accordingly to your needs.

Add simple FTP server

The line to be added has the following syntax:

curlftpfs#192.168.20.71 /mnt/ftp/server1 fuse auto,allow_other,disable_eprt,_netdev 0 0

Add FTPS server

The line to be added has the following syntax:

curlftpfs#192.168.20.71 /mnt/ftp/server1 fuse auto,allow_other,disable_eprt,_netdev,ssl,no_verify_peer 0 0

Add FTP over explicit TLS server

The line to be added has the following syntax:

curlftpfs#192.168.20.71 /mnt/ftp/server1 fuse auto,allow_other,disable_eprt,_netdev,ssl_control 0 0

Add SFTP (FTP over SSH)

You have to know the absolute path fro the remote server to be mounted for that task. For an automatic mount, you need to save your own SSH public key in the remote system's file "~/.ssh/authorized_keys". This enabled an automatic login without password prompt (if allowed by the remote SSH server).

sshfs#seon@192.168.20.71:/home/seon/ /mnt/ftp/server1 fuse auto 0 0

Using a proxy server

When using a proxy server, you have to add an option to the entry line in "/etc/fstab":

proxy=http://username:password@proxy-server:3128

A complete line in "/etc/fstab" would be:

curlftpfs#192.168.20.71 /mnt/ftp/server1 fuse auto,allow_other,disable_eprt,_netdev,proxy=http://proxyuser:proxypwd@proxy-server:3128 0 0

Beware that the credentials are saved in a system-wide readable format, so use a pre-defined proxy user only for that task (i.e. with limited permissions). You may also want to set up the proxy environment variable as described in Seon HTTP Proxy support.