19 replies

1. cthos 🐱 5 min later

   @zachleat you’ll have a lot more control over this via Cloudfront. I am very pleased watching this thread.

   Is that a weird thing to say? Am I being weird? Oh well.

   1. Zach Leatherman 14 min later

      @cthos to be honest I feel like it’s weird that more folks aren’t complaining about this? seems like a very expensive difference in service offerings

      1. cthos 🐱 25 min later

         @zachleat it’s the managed service trap with a free tier. It’s easy to set up, and they’re hoping to keep you there with inertia and sunk cost fallacy.

         That said AWS also has some major footguns.

         1. Zach Leatherman 28 min later

            @cthos Yeah, there is plenty of technical wiggle room to explain the difference away but 10× metrics has me scratching my head

            1. cthos 🐱 29 min later

               @zachleat oh no I think Vercel is charging a pretty solid margin. I don’t have any hard numbers though.

               1. Zach Leatherman 32 min later

                  @cthos I guess my hunch is that it isn’t “overhead” in the classical sense of we’re charging you a reseller fee for convenience. I’d be fine with that. But what if the *usage* measurements are inflated?

                  1. cthos 🐱 39 min later

                     @zachleat ohhh I see. Yeah that’d be, not good.

                     1. cthos 🐱 46 min later

                        @zachleat I started to walk through (in addition to what you said above) and I have a couple of wild speculations. They could also be doing a "keep warm" tactic where they invoke the function to keep it awake periodically to eliminate cold starts.

                        1. Zach Leatherman 57 min later

                           @cthos oh huh! How warm should the start be 😅

                           1. cthos 🐱 1 hr later

                              @zachleat Depends on how "critical path" the function is. AWS will keep a lambda function "warm" for a period of time until it stops getting requests and then it'll put it to sleep. It'll incur a performance penalty to wake it back up. How long that penalty is depends a lot on the function and the language the function is written in. It can be as long as a full second though. One common tactic is to just have a cron job go ping the service to keep it alive every couple of minutes.

                              1. cthos 🐱 1 hr later

                                 @zachleat Related fun, there are like... at least 2 other AWS addons for solving this problem in various ways (managed provisioning and SnapStart for Java).

2. Zach Leatherman 12 min later

   Worth noting that Netlify’s On-demand Builders provide global persistence across all edge nodes: https://www.netlify.com/blog/2021/10/25/faster-more-reliable-page-loads-with-update-to-on-demand-builders/

   Alternatively, CloudFront has 13 Regional Edge Caches that (in my tests with a VPN) seem to be segmented too: https://aws.amazon.com/cloudfront/features/?whats-new-cloudfront.sort-by=item.additionalFields.postDateTime&whats-new-cloudfront.sort-order=desc

   CloudFront also offers an Origin Shield product that appears to provide a similar feature but I have not enabled it: https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cache-hit-ratio.html#cache-hit-ratio-use-origin-shield

   1. cthos 🐱 28 min later

      @zachleat right so you can opt into the regional caches if you want to, and there is cost attached to that (iirc there’s also a buried cross region data transit cost). You don’t *have* to do it (choice!)

      You can also look at lambda@edge

      1. Zach Leatherman 30 min later

         @cthos I’m very happy with the current setup I think. I’ll enable origin shield once I hit the inflection point where it costs less than my lambda usage (assuming that exists)

         1. cthos 🐱 38 min later

            @zachleat def set up a billing alarm now while you’re thinking about it if you don’t have one already.

   2. Zack Tollman 1 hr later

      @zachleat at some point in this journey, I hope you look into Fastly. Some major improvements over Cloudfront

      1. Zach Leatherman 1 day later

         @tollmanz that is a great callout. I need to dive in there.

3. benschwarz 5 hr later

   @zachleat this is interesting because I keep getting emails about hitting the function invocation limit. From poking around I could see lots of responses that weren’t cached at the edge, but should’ve been.

   1. Zach Leatherman 1 day later

      @benschwarz another distinction here is that I don’t think cached responses are excluded from invocation counts—but I’m not super worried about that since they charge $.60 per million there. It’s the $.18 per GB hour that is biting me (at ~42h per day usage)

      correction: cached hits are allegedly not counted in invocation counts: https://vercel.com/docs/pricing/serverless-functions#managing-function-invocations