Lately I have been neglecting my blogging in sacrifice towards getting more work done. However, I am still reading my books and writing notes as I go. I will continue my review series soon. However, today I am going to discuss the solution to a problem I struggled with for several hours. That is, getting an AWS Lambda function running inside a VPN but with access to the internet.
A Note About Using This To Access A Database
Having used this for some time now, I have come to the realization that using this setup to talk to a database gets messy quickly. Instead of giving each lambda function access to your database directly, you should put a microservice in front of it that handles requests. The benefits:
- This allows you to have a single connection pool to your database.
- This allows you to put a cache in front of that microservice.
- This limits how many things need your database credentials.
- This adds another layer of security between the world and your data.
- It’s actually much easier to build and maintain.
Having said all that, the configuration below still gives a lambda function access to outside world resources, as well as a way to access items secured within your VPC (for example a microservice?). Read on with those things in mind.
What and Why
Amazon Web Services (AWS) offers a service called Lambda which allows for some arbitrary code to be run based on custom triggers (For example, on a time schedule). This is a great way to accomplish Cron like functionality within your AWS infrastructure. To keep your infrastructure secure you probably have most of your stuff (databases, servers, etc) running inside a Virtual Private Cloud (VPC). This is great because everything can inter-communicate over a network without being open to the outside world. Since all your stuff is running inside your VPC, you also probably want your Lambda code to run there. However, if your Lambda code needs to access the internet (to hit an external API for example) you are going to have some problems.
Atypical of AWS, the documentation on how to solve this problem is sparse to nonexistent. When you are creating your Lambda service, once you select your VPC, you will see some help text that looks like:
When you enable VPC, your Lambda function will lose default internet access. If you require external internet access for your function, ensure that your security group allows outbound connections and that your VPC has a NAT gateway.
Trying to find any more help, beyond this message was not successful for me. So I spend a lot of time tinkering with AWS networking stuff and figured out how to get this all to work. I am going to share that solution here for posterity and hopefully it helps someone else down the road.
This image sums up what we are going to create, hopefully adding some clarity to what you are about to read.
Initially you should have a VPC in which you want to put your Lambda code. If you do not, there is plenty of documentation on creating a VPC inside your AWS account. The option to create a VPC is under the Networking and Content Delivery group of services inside your console. Your VPC will have a CIDR Block that looks something like: 192.168.0.0/16, you will need to know this number.
For your VPC to have access to the internet it requires an Internet Gateway. You can create one of these by going to Networking and Content Delivery > VPC > Internet Gateway. The only customization to do here is to give it a name.
You will need 2 subnets for this solution. The first is where your Lambda code will be run (you select the subnet to run in under the Lambda configuration, right after you select your VPC). The second subnet will be used as a gateway through which you can direct traffic to the internet. To create the two subnets, select the subnets field under Networking and Content Delivery > VPC > Subnets. Create two subnets, naming them whatever you want (For Example: LambdaSubnet and PublicAccessSubnet). When you create the subnet you will need to assign a CIDR Block for each. This should be a sub block of your VPC CIDR block. For example, if your VPC CIDR clock is 192.168.0.0/16 then for your Subnets you might pick 192.168.100.0/24 and 192.168.110.0/24. For now, no further customization of these subnets is needed.
We are also going to need an Elastic IP. This will be the IP address that the world sees whenever your Lambda function talks to the internet. To create one go to Networking and Content Delivery > VPC > Elastic IPs and allocate new. Write down your new IP address for use later.
This is the thing they were talking about in their help text on the Lambda configuration page. We want to build a NAT Gateway inside our Public Subnet (Not the one where the Lambda code will run). To build one of these you go to Networking and Content Delivery > VPC > NAT Gateways and create new. Here you will need to select the Public Subnet and the Elastic IP that you created.
Now we only need to tell each subnet where to send its traffic, and we do this using Route Tables. The first Route Table we are going to build goes from our Lambda Subnet to the NAT gateway. To build this go to Networking and Content Delivery > VPC > Route Tables and create new, naming it whatever you want. Once the Table is created select it and switch to the Routes tab. Inside that tab edit the table and add a new route from destination 0.0.0.0/0 (Meaning all traffic) to target <Your NAT gateway>. Under the Subnet Associations tab click edit and select your Private subnet.
Great, now any traffic leaving your Lambda subnet will be routed to your NAT gateway (using your elastic IP) inside your Public subnet. Next we will use another route table to tell your public subnet to route the traffic to the Internet Gateway. Create a new Route Table and this time edit its routes to send destination 0.0.0.0/0 to your Internet Gateway. On this route associate the public subnet.
The last step, when configuring your Lambda code, is to add it to a security group that allows outgoing connections. To create this security group go to Compute > EC2 > Network & Security > Security Groups. Here, create a new security group and add the outbound rule All Traffic, All Protocols, All ports, Destination 0.0.0.0/0.
This will allow your Lambda code to send out requests (and receive their responses) but not to let inbound connections.
With this all setup your Lambda function should be able to talk to other services inside your VPC while also sending requests out to the internet. As you can see the help text provided by AWS was a clear guide on how to make this work (/s). I hope this helps some people out and saves others the hours I spent messing around with AWS networking and Lambda.
If there is an error here, I missed something, or this no longer works for you, leave me a message and I will look into it. If this does work for you, also leave me a message so I feel better about my time.AWS, Lambda, VPC