Ari Palo

As exciting the re:Invent season with all the product and feature launches is, there are some small under-the-radar features I'd really love to see AWS to improve on. Based on various bits of information what I've been able to gather around the internet (such as the public roadmaps and Github/StackOverflow issues etc), I don't expect to see these features to be addressed during the re:Invent 2020: Which is why I am writing this blog post, in the hopes it will increase the visibility of these issues.

These are things that might not seem so big-of-a-deal at first and they're definitely not show stoppers, but they can be quite annoying if you happen to bump into them; And once you start to work around them, you soon realize they're something that could almost be described as undifferentiated heavy lifting in 2020.

“Customers want to focus on their unique application logic and business needs – not on the undifferentiated heavy lifting of provisioning and scaling servers, keeping software stacks patched and up to date, handling fleet-wide deployments, or dealing with routine monitoring, logging, and web service front ends.”

– Dr. Werner Vogels, CTO, Amazon.com

There's yet another blog post about a Lambda-based solution on updating security groups with CloudFront IP ranges. I've been saying for years that in my opinion this is unnecessary, undifferentiated heavy lifting.

Origin protection should be something that (in a “perfect world”) AWS would provide out-of-the-box. Yes, one can always set custom headers etc in CloudFront that the app servers are validating against, but still, there is a clear need for protecting your origin load balancer in network level with security groups.

In a perfect world, security groups could utilize the Origin Access Identity construct that already exists in CloudFront (and currently can be used with S3 origins): So that one could set a specific OAI as allowed inbound source in a security group. This would solve two things regarding VPC-based origins:

1. Origin could only be accessed via (any) CloudFront - would remove the need for those Lambda-based solutions updating CloudFront IP ranges to Security Groups

2. Origin could only be accessed via specific CloudFront distribution - would remove the need to do some header-value checking in the origin application

In a not-so-perfect world, if the above would be impossible (point 2, allow access only via specific distribution), then to solve the point 1 (allow access via any CloudFront) atleast, maybe there could be an AWS-managed Prefix list since - a construct like that already exists for similar purposes - for CloudFront IP ranges?

Resources:
  MyOAI:
    Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: Some OAI later assigned to CF distribution
  MySG:
    Type: AWS::EC2::SecurityGroup
    Properties:
        GroupDescription: Allow access via CF distribution
        VpcId:
          Ref: myVPC
        SecurityGroupIngress:
        - IpProtocol: tcp
          ToPort: 80 # Or 443 if origin can handle HTTPS
          SourceOriginAccessIdentity: !Ref MyOAI

Workaround ¶

Fellow AWS Community Builder, Ben Bridts‏, pointed out a workaround using WAF & Secrets Manager (also see his blog post), which solves the issue of protecting the origin by allowing access only via specific CloudFront distribution, BUT it still requires quite a lot of setup and I think it still counts as that undifferentiated heavy lifting I'd like to avoid.

One of the most exciting launches of re:Invent 2020 for me has been Aurora Serverless v2 (still in preview). It solves a lot of the pain points in Aurora Serverless (v1). I can't wait to get my hands dirty and start playing with the v2! If you haven't heard about the Aurora Serverless v2 yet, do read the excellent post by AWS Serverless Hero Jeremy Daly.

There is a strong indication that ASv2 will support all the major features of AWS Aurora once it goes to GA, but one feature I really, really, hope it will support is the RDS Proxy! I think the lack of RDS Proxy support in ASv1 has been the biggest feature miss in that product.

Yes, there is the Data API which is cool, but it's not the same thing as using real SQL-connection which allows using all the possible SQL-libraries and tools out there! Even ORMs if you so wish! Also, it makes local development somewhat more cumbersome, as you need to mock the AWS Data API responses. Instead, if you'd be able to use SQL-connections, then for local development you could just spin up a SQL-server of your choice inside a Docker container and hack away!

I guess there are valid technical reasons why AWS has not yet enabled pairing these two, but I really hope they will once ASv2 goes GA since Aurora Serverless and RDS Proxy would be a match made in heaven!

Limitations for RDS Proxy
• You can't use RDS Proxy with Aurora Serverless clusters.

– AWS documentation on RDS Proxy

Have to add: I really wish ASv2 will support RDS Proxy for both MySQL and PostgreSQL! 🙏 (I personally prefer PostgreSQL 🙃)

The AWS CLI v2 and the Python SDK (boto3) do now have nice support for MFA, assuming named profiles from ~/.aws/config and caching temporary session credentials - so you don't have to insert the MFA token for each operation. The challenge is that not all AWS SDKs and CLI tools have the same level of support: For example AWS CDK added MFA support a while back now, but it lacks the support to cache the temporary session credentials; Same goes for several SDKs. Which makes sense, since they are all separate implementations.

The other elephant in the room is AWS SSO: On the surface, it is a great product and it also plays really well together with AWS CLI v2, but again tools such as CDK is missing support for SSO named profiles and the situation seems to be the same for the other SDKs.

There are community-driven projects such as the awesome aws-sso-util built by Ben Kehoe that address these issues with AWS SSO; But even though Ben is a well-known figure in the community (which means I am keen to trust his implemenation), generally speaking I think that having to resort to 3rd party code for authentication in the enterprise just feels a bit wrong! And even if you accept the fact of using 3rd party code for the process in general, there's still the issue of having to teach hundreds (or in some larger enterprises even thousands) of developers how to setup this custom tool that is not maintained or documented by AWS!

I really hope 2021 is the year AWS manages to unify the authentication experience across the board and also fix the rough edges related to AWS SSO.

“Not all AWS SDKs have support for AWS SSO (which will change eventually). However, they all have support for credential_process, which allows an external process to provide credentials. aws-sso-util credential-process uses this to allow these SDKs to get credentials from AWS SSO. It's added automatically (by default) by the aws-sso-util configure commands.”

– benkehoe/aws-sso-util documentation

Summary ¶

While I really have been enjoying all the awesome product/feature launches (that have been made so far) of re:Invent 2020 such as the Lambda Container Image Support, 1ms billing granularity for Lambda, Aurora Serverless v2, Aurora PostgreSQL Lambda triggers, S3 Strong Read-After-Write Consistency and many others - I have to say that fixing these four issues I've described in this blog post would make the day-to-day development for me and many others a lot more fun! 🤩

Generally speaking 2020 has been a shit challenging year due to the pandemic as everybody knows, so here's to an awesome 2021 and who knows, maybe we'll even see each other at re:Invent in Las Vegas! 🥂

… and dear AWS, please put a cherry on top by implementing that wishlist! 😉