BHLe_Content_Provider_Uploading_Config
BHL Content Provider File Uploading
Requirement (from TMB Amsterdam meeting)
- User names and passwords to create accounts – ( there must be Content Provider & BHL negotiation before uploading data…policy issue for accepting uploads into system).
- Creation of accounts - account per institution i.e. 1 ftps account per CP (23 of them)
Implementation strategy – define each institution account as a virtual user rather than host account to minimise potential for exploit.
- Storage – 1TB per institution or 500GB?
- Isolated buckets per institution to maintain data integrity.
apply chroot jailing on FTPS accounts to be used to enforce isolated upload spaces
- 1 aggregated directory of everyone’s content to show a complete list i.e a web interface of what everyone else has – purely for viewing.
- Each inst. has read/write access for their inst. Upper level for read access only.
read/write only via FTPS; given the chroot lockdown, enable a read only view only via aggregated web interface to ensure jailing is effective.
- 1 sandbox directory for CPs to test system if unsure of usage to test uploading.
No issues, add this as a 24th account. (virtual user account = test)
- NFS access to files to 23 accounts.
Using virtual users will map all actual file permissions to one controlled account (current choice is the ftp service account, can be changed), which will minimise potential issues with file permission conflicts later. Quota enforcement for virtual users is the main area of pain; can report via scheduled jobs on space usage, but the backend account applied to data areas is applied as the ftps service account.
Solution
Use vsftpd with SSL and virtual users only for FTPS content upload. Virtual users mean that any exploit to the account should not provide a vector for privilege escalation to other services. FTPS provides only data transfer and no shell options, with a reasonable level of configuration lockdown. Automation not as readily achievable as using SSH/rsync but transfers can be set up/passed on without a dedicated sysadmin at each end.
HTTP to provide a read only view of the data.
Virtual users managed via a standalone htpasswd table in the first instance, can move to openldap or other source once identity provider up and in service.
Configuration Steps
Configure secure VSFTPD
apt-get install vsftpd
/etc/vsftpd.conf
#VSFTPD standard config lockdown
listen=YES
anonymous_enable=NO
local_enable=YES
write_enable=YES
connect_from_port_20=YES
secure_chroot_dir=/var/run/vsftpd
pam_service_name=vsftpd
chroot_local_user=YES
hide_ids=YES
Configure SSL
/etc/vsftpd.conf
#SSL config
#This option specifies the location of the RSA certificate to use for SSL
#encrypted connections.
rsa_cert_file=/etc/ssl/certs/vsftpd.pem
ssl_enable=YES
force_local_data_ssl=YES
force_local_logins_ssl=YES
ssl_tlsv1=YES
#Permit SSL v2 protocol connections. TLS v1 connections are preferred
ssl_sslv2=NO
#permit SSL v3 protocol connections. TLS v1 connections are preferred
ssl_sslv3=NO
now generate a self signed certificate
cd /etc/ssl/certs
openssl req -x509 -nodes -days
nnn -newkey rsa:1024 -keyout \
/etc/vsftpd/vsftpd.pem -out /etc/vsftpd/vsftpd.pem
days arbitrarily set as 1095 (3 year certificate)
Configure File Upload Space
Upload will need a good chunk of storage; provisioned via NFS from the main filespace.
on nfs server side (cld-demeter):
mkdir /gpfs/bhlfsa/nfs/upload
mkdir /gpfs/bhlfsa/nfs/upload/providers
[FTP working space under mountpoint]
chown 103 /gpfs/bhlfsa/nfs/upload/providers
[103 = ftp service account]
chmod 775 /gpfs/bhlfsa/nfs/upload/providers
/etc/exports add:
/gpfs/bhlfsa/nfs/upload 157.140.72.0/24(rw,sync,fsid=105)
then
exportfs -a
on client host side (vsftpd server):
/etc/fstab
dfs-ctdb:/gpfs/bhlfsa/nfs/upload /mnt/upload nfs rw,hard,intr,rsize=32768,wsize=32768 0 0
then
mkdir /mnt/upload
mountall
Configure Virtual Users
Initial usage will be to provision FTPS users as virtual users via a htpasswd file; the configuration below will chroot each virtual user connection to its own space under /mnt/upload/providers and will map the actual file owner back to ‘ftp’.
By using a virtual user table with controlled pam configuration no host shell accounts will be able to logon via vsftpd, precluding brute force attacks on those.
Option to later shift pam config to use ldap authentication source.
apt-get install libpam-pwdfile
add to /etc/vsftpd.conf
#virtual user configuration
guest_username=ftp
guest_enable=yes
user_sub_token=$USER
virtual_use_local_privs=YES
local_root=/mnt/upload/providers/$USER
copfigure /etc/pam.d/vsftpd
#Standard behaviour for ftpd(8).
auth required pam_listfile.so item=user sense=deny file=/etc/ftpusers
onerr=succeed
#BHLe Customized login using htpasswd file for virtual users
auth required pam_pwdfile.so pwdfile /etc/vsftpd/passwd
account required pam_permit.so
mkdir /etc/vsftpd
touch /etc/vsftpd/passwd
User creation/modification
currently using htpasswd:
htpasswd /etc/vsftpd/passwd testuser
As each user is created, a corresponding upload directory must be created to match under /mnt/upload/providers
/username
Put a link to this into /mnt/upload/webroot in the agreed short name, so that web presentation uses the agreed name and does not publish the actual virtual user names
eg.
cd /mnt/upload/webroot
ln –s /mnt/upload/providers
/username NHM
Read only web service
Provision a web service - apache available by default on the host so unless performance requires, use this apache, with a cut down configuration.
cd /var/www
ln –s /mnt/upload/webroot
[amend default apache configuration to remove unneeded modules and disable ServerTokens and ServerSignature]
drop in a basic index.html with an href to the upload/webroot for later revision. [the reason for doing it this way is that should the gpfs filesystem need to be unmounted or shifted, the website will present at least the entry page cleanly – I’m a believer that a website should have at least a top level page]
Client Connection
FTPS upload client usage will differ depending on the client chosen.
As an example, using filezilla, connection can be achieved by using:
Host: ftpes:
upload.bhl-europe.eu [note this name does not yet exist and may differ when it does]
Username: userid
Password: password
Appendix
bhl-celsus.nhm.ac.uk is the current virtual host, currently residing on 157.140.72.27 which is within the NHM firewall, IP address likely to change once we’ve determined the best way to proxy this via NHM DMZ or otherwise.
For full service presentation a name should be created in the bhl-europe.eu space and directed to this - either CNAME->bhl-celsus.nhm.ac.uk (preferable) or A->157.140.72.27)
WebDAV Fallback
FTPS connection issues with some clients cannot all be worked around without ability to influence client side firewalling, so an alternative webDAV option is in place.
/etc/apache/sites-available/bhl-ssl copied from default-ssl and modified as below, configured to use the same directory and user table as the vsftp d service.
<VirtualHost _default_:443>
ServerAdmin
ITSupport@nhm.ac.uk
LimitXMLRequestBody 131072
DavLockDB /var/dav/DavLock
DocumentRoot /var/www/
<Directory />
Options FollowSymLinks
AllowOverride None
</Directory>
<Directory /var/www/>
Options Indexes FollowSymLinks MultiViews
AllowOverride None
Order allow,deny
allow from all
deny from 202.130.35.117
</Directory>
<Location /uploads>
Options +Indexes
IndexOptions FancyIndexing
AddDefaultCharset UTF-8
AuthType Basic
AuthName "WebDAV"
AuthUserFile /etc/vsftpd/passwd
Require valid-user
</Location>
<Location /uploads/LANDOE>
Dav On
Require user at-landoe
</Location>
<Location/uploads/next-client-etc>
Dav On
Require user next-client-user-etc
</Location>
....... rest of config file standard except:
SSLCertificateFile /etc/ssl/certs/apache.crt
SSLCertificateKeyFile /etc/ssl/private/apache.key
With the server keys being generated via:
cd /etc/ssl/certs
openssl genrsa -des3 -out apache.key.orig 1024
openssl rsa -in apache.key.orig -out apache.key
openssl req -new -key apache.key -out apache.csr
openssl x509 -req -days 1000 -in apache.csr -signkey apache.key -out apache.crt
