Zato Blog: SFTP connections

Zato has had support for FTP/FTPS since its inception. In Zato 3.1+, SFTP is also an option to consider for file transfer and this post offers an introduction to the functionality.

Web-admin

For build and deployment automation, zato enmasse is the command line tool most convenient to use, but during initial development SFTP connections can be constructed in web-admin.

The form lets one provide all the default options that apply to each SFTP connection - remote host, what protocol to use, whether file metadata should be preserved during transfer, logging level, bandwidth limit for each connection, SSH identity and config files as well as additional SSH options - the last one means that any SSH option that man sftp lists can also be used in Zato connections.

Pinging

The first thing that one can do right after the creation of a new connection is to ping it, to check if the server is responding.

Pinging opens a new SFTP connection and runs the ping command - in the screenshot above it was ls . - a practically no-op command whose sole purpose is to let the connection confirm that commands in fact can be executed, which proves the correctness of the configuration.

This will either returns details of why a connection could not be established or the response time if it was successful.

Rapid development

Having validated the configuration by pinging it, we can now execute SFTP commands straight in web-admin from a command shell:

Any SFTP command, or even a series of commands, can be sent and responses retrieved immediately. It is also possible to increase the logging level for additional SFTP protocol-level details.

This makes it possible to rapidly prototype file transfer functionality as a series of scripts that can be next moved as they are to Python-based services.

Python API

For Python services, an extensive API is available. The API can execute transfer commands individually or in batches but alternatively it may make use of SFTP scripts previously created in web-admin. Here is how it can be used in practice:

# -*- coding: utf-8 -*-

from __future__ import absolute_import, division, print_function, unicode_literals

# Zato
from zato.server.service import Service

class MySFTPService(Service):
    def handle(self):

        # Connection to use
        conn_name = 'My SFTP Connection'

        # Get a handle to the connection object
        conn = self.out.sftp[conn_name].conn

        # Execute an arbitrary script with one or more SFTP commands, like in web-admin
        my_script = 'ls -la /remote/path'
        conn.execute(my_script)

        # Ping a remote server to check if it responds
        conn.ping()

        # Download an entry, possibly recursively
        conn.download('/remote/path', '/local/path')

        # Like .download but remote path must point to a file (exception otherwise)
        conn.download_file('/remote/path', '/local/path')

        # Makes the contents of a remote file available on output
        out = conn.read('/remote/path')

        # Uploads a local file or directory to remote path
        conn.upload('/local/path', '/remote/path')

        # Writes input data out to a remote file
        data = 'My data'
        conn.write(data, '/remote/path')

        # Create a new directory
        conn.create_directory('/path/to/new/directory')

        # Create a new symlink
        conn.create_symlink('/path/to/new/symlink')

        # Create a new hard-link
        conn.create_hardlink('/path/to/new/hardlink')

        # Delete an entry, possibly recursively, no matter what kind it is
        conn.delete('/path/to/delete')

        # Like .delete but path must be a directory
        conn.delete_directory('/path/to/delete')

        # Like .delete but path must be a file
        conn.delete_file('/path/to/delete')

        # Like .delete but path must be a symlink
        conn.delete_symlink('/path/to/delete')

        # Get information about an entry, e.g. modification time, owner, size and more
        info = conn.get_info('/remote/path')

        self.logger.info(info.last_modified)
        self.logger.info(info.owner)
        self.logger.info(info.size)
        self.logger.info(info.size_human)
        self.logger.info(info.permissions_oct)

        # A boolean flag indicating if path is a directory
        result = conn.is_directory('/remote/path')

        # A boolean flag indicating if path is a file
        result = conn.is_file('/remote/path')

        # A boolean flag indicating if path is a symlink
        result = conn.is_symlink('/remote/path')

        # List contents of a directory - items are in the same format that .get_info uses
        items = conn.list('/remote/path')

        # Move (rename) remote files or directories
        conn.move('/from/path', '/to/path')

        # An alias to .move
        conn.rename('/from/path', '/to/path')

        # Change mode of entry at path
        conn.chmod('600', '/path/to/entry')

        # Change owner of entry at path
        conn.chown('myuser', '/path/to/entry')

        # Change group of entry at path
        conn.chgrp('mygroup', '/path/to/entry')

Summary

SFTP are a new file transfer option added in Zato 3.1. Users may quickly prototype SFTP scripts in web-admin and employ them in Zato services. Alternatively, a full Python API is available for programmatic access to remote file servers. Combined, the features make it possible to create scalable and reusable file transfer services in a quick and efficient manner.

from Planet Python
via read more