.. _file_transfer:

|

:Last modified: |today|

|

#############
File transfer
#############

.. attention:: 

    Do not store personally identifiable research data (PID) on the Neuro cluster. The Committee for Protection of Human Subjects (CPHS) requires that PID be secured with encryption, strong passwords, audits, etc. If you accidentally upload PID to the Neuro cluster, remove it immediately and contact support-neuro@berkeley.edu to remove the data from online backups. For more information, see: `CPHS Data Security Requirements <http://www.oshpd.ca.gov/Boards/CPHS/DataSecurityRequirements.pdf>`_.


.. _sftp:

---------------------------------------------------------------------------------------------------------------------------------
sftp 
---------------------------------------------------------------------------------------------------------------------------------
There are many sftp clients available, including graphical clients like FileZilla and Cyberduck. 

#. If you haven't done so already, then enable :ref:`2-step authentication<mfa>`

#. If using a graphical sftp client, configure interactive logins. 


   In FileZilla, open the Site Manager (File-> Site Manager) and configure a 'New Site', e.g.::

        Name: Neuro Cluster
        Host: nx2.neuro.berkeley.edu
        Protocol: SFTP
        Login Type: Interactive
        User: <your Neuro Cluster username>


   In Cyberduck, ::

        select Edit > Preferences > Transfers > General
        Under "Transfers", use the "Transfer Files" drop-down to select Use browser connection


#. You will be prompted for your login password and 2-step 'verification code' at login.

.. _scp:

---------------------------------------------------------------------------------------------------------------------------------
secure copy (scp)
---------------------------------------------------------------------------------------------------------------------------------
secure copy (scp) is commnd-line program available for Mac and Linux.

#. If you haven't done so already, then enable :ref:`2-step authentication<mfa>`

#. Open a terminal window

#. Run the 'scp' command, e.g. to recursively copy the directory (DataDir) on my local desktop to my home directory on the Neuro cluster::

    scp -r DataDir joe@nx2.neuro.berkeley.edu:~

#. You will be prompted for your login password and 2-step 'verification code' at login.

.. _rsync:

---------------------------------------------------------------------------------------------------------------------------------
rsync
---------------------------------------------------------------------------------------------------------------------------------

The rsync program is used to synchronize files over a network. It's distributed with Mac and Linux. Rsync is a great tool for backups, because rsync transfers only new or modified files, potentially saving you time and bandwidth.

#. If you haven't done so already, then enable :ref:`2-step authentication<mfa>`

#. Open a terminal window

#. Run the 'rsync' command, e.g. the following rsync command will recursively copy files in /home/despo/joe/data on the Neuro cluster to /backup/data on the machine where the command is run. The '-z' option compresses files during the ftransfer::

    rsync -aHxz --stats --delete joe@nx2.neuro.berkeley.edu:/home/despo/joe/data /backup

.. Note::

    the --delete option deletes files from the destination directory that do not exist in the source directory. This option can be dangerous, so verify that the destination directory is correct.

See also:

`Rsync documentation <http://www.samba.org/ftp/rsync/rsync.html>`_

`Rsync FAQ <http://www.samba.org/rsync/FAQ.html>`_