Blog

Using FTP to transfer files from one environment to another is a long-standing process. But when the destination for a file transfer is a server residing in a private subnet, one must get around the inability to connect directly to it over the Internet. This article describes how to easily accommodate doing this, but first, it is necessary to define some terminology and declare some assumptions that will be made throughout.

Terms

• SFTP: Secure File Transfer Protocol. A technology for transferring files between an origin computer and a destination computer. SFTP supplants FTP given that early generation FTP made no attempt to transfer files securely and required multiple channels to be open before it could be used. SFTP requires only port 22. SFTP should not be confused with FTPS which is nothing more but traditional FTP with encryption added on. SFTP on the other hand was developed from the ground with encryption being an inherent property of the protocol.
• EC2: Elastic Compute Cloud. This is the service within Amazon Web Services (AWS) to provision servers. An individual server is referred to as an EC2 instance and may use a Windows, Linux or MacOS operating system.
• VPC: Virtual Private Cloud. Customers of AWS may create logically isolated segments of the AWS cloud for their own infrastructure and resources.
• Subnet: Each VPC may be divided into logically isolated segments and assigned a range of IP addresses specific to it.
• Private subnet: A subnet within a VPC that does not permit traffic from the Internet to reach any of the resources inside the subnet.
• Public subnet: A subnet within a VPC that allows traffic from the Internet to access its resources. For example, an EC2 inside a public subnet may have a public IP address attached and be reachable across the Internet.

Assumptions

• This article is specific to the Amazon Web Services (AWS) cloud computing platform though the high level concepts are applicable to any architecture.
• The article assumes that any EC2 instances have the Systems Manager agent installed and running on it as is the case by default.
• You have the AWS Command Line Interface (CLI) installed on your local machine.
• An operable SFTP server is installed on your EC2 instance and ready to accept connections. Setting up an SFTP server is outside the scope of this article. Refer to SFTP Server for more. One special note for Windows users, that page mentions that OpenSSH is "not so great for Windows." While that may have been true at one time, OpenSSH is now built into the Windows Server operating system (though must be enabled as an optional feature) and could not be any simpler to set up and use.

Scenario

You have created a VPC with both private and public subnets. There is an EC2 hosting a website in the private subnet. The simple act of placing servers into private, rather than public, subnets means they are not exposed as attack vectors over the Internet and instantly goes a very long way toward enhancing their security posture.

Of course, being unreachable across the Internet also means that if they host a public-facing website, they will need some help publishing their content to the web. That could be achieved with a content delivery network, but for this example, we'll assume there is an Application Load Balancer (ALB) in the public subnet. When a user requests a page from this website, that request is first routed to the ALB, and then to an EC2 instance in the private subnet before being returned to the end user's browser.

So far, so good. The ALB can handle HTTP and/or HTTPS traffic from client browsers. But now suppose that the administrator of this website wishes to put some new and/or updated files from a development environment onto the production server sitting tucked away in that private subnet. The ALB will not help. Perhaps you've heard something about using a bastion host in the public subnet that you could connect to and then jump from there over to the EC2 in the private subnet. In fact, AWS even recommends that for some things. But there are costs associated with bastion hosts; they need to be hardened; and they are going to require creating security group rules. All doable things, but there is a better way. Systems Manager.

Systems Manager to the Rescue

By using port forwarding in systems manager we may create an encrypted tunnel that runs from a local machine to the EC2 instance. We then connect our client software, and in this example we'll use FileZilla since it is an enormously popular SFTP client, to connect to a local port which then tunnels all the way through to the EC2.

Systems Manager essentially takes the place of a bastion host but has the added benefit of not having to create any security group rules to permit access; there is no server hardening to perform; and there is no cost associated with running a bastion.

As stated in the assumptions earlier, the Systems Manager agent must be installed and running and the EC2 instance must be registered in Systems Manager for this to work. The agent comes preinstalled on all EC2 instances and is configured to start after reboots so you should be ready to go with that unless some customization of an EC2 has removed it. A full discussion of the agent is outside the scope of this article, but you may refer to the documentation at Working with SSM Agent for more and see What is AWS Systems Manager for complete documentation about Systems Manager.

Also stated in the assumptions section earlier is that you have the AWS CLI installed and running on your local machine. A full discussion of the CLI is outside the scope of this article, but documentation is available at What is the AWS Command Line Interface?

Create the tunnel

Once the CLI and Systems Manager are ready to go, you simply open a command prompt and enter:

aws ssm --profile default start-session --target i-xxxxxxxxxxxxxxxxx --document-name AWS-StartPortForwardingSession --parameters "portNumber=22,localPortNumber=12345"

• The credentials profile used is the default. If you've configured your CLI differently and are using a profile other than the default to perform this action, you will of course need to reference the specific profile in the –profile argument.
• For the –target argument, provide the ID of the instance to which you wish to connect. The example above uses xxxxxxxxxxxxxxxxx so as to not expose any real instance IDs, but you may get the real instance ID of the instance to which you wish to connect in the EC2 management console of your AWS account.
• For the portNumber in the –parameters argument, 22 is specified because that is the port used by the SFTP server on the EC2 instance. The localPortNumber is set to 12345, but could be any available port number on your local machine.

After entering the command above, you will know you have successfully created a tunnel when it returns the message, Waiting for connections...

Create the SFTP connection

Leave open the command prompt showing Waiting for connections… and now open Filezilla. You will need to first configure your site so go File > Site Manager. Click the New site button and provide values as follows:

• Protocol: SFTP - SSH File Transfer Protocol
• Host: 127.0.0.1
• Port: 12345
• Logon Type: Normal
• User: Your SFTP user
• Password: Your SFTP user's password

The following screen image demonstrates what this looks like in FileZilla.

Click the Connect button and your connection is made. (Though on the first connection you will be prompted to accept a certificate from the SFTP server.) That's it! You may now transfer files back and forth between origin and destination. What's happening here is that FileZilla thinks it's connecting to a local machine, but thanks to the magic of the wormhole created by Systems Manager, the localhost (127.0.0.1 in FileZilla's site setup) is really just a proxy for the actual SFTP server running on the EC2 in the private subnet.

Bonus Usage

This article was written using SFTP as an example, but the port forwarding feature of systems manager can be used for multiple purposes. Suppose the destination EC2 instance is a Windows machine and you wish to connect to it using RDP. Just change the portNumber parameter in the CLI statement from 22 to 3389 (the default port for RDP) and then point your RDP client software to localhost:12345 exactly as was done above using FileZilla. Voilá, you have an RDP connection to the EC2.