Caddy v2.8 Released

As of 6/5/2024, we're officially running Caddy v2.8.4 across our entire network!

Caddy v2.8 is here! What's new?

There are literally hundreds of improvements that are a part of this release, with many that will improve the performance of our large-scale Caddy deployment. We're gonna focus on the things we're most excited to see in this post.

ZeroSSL API Integration

Background: Skip2 uses ZeroSSL as our primary certificate issuer.

Thus far, Caddy has used the ZeroSSL ACME endpoint for provisioning and retrieving SSL certificates. This works excellently for a few nodes, but even with shared Caddy storage in a cluster, we were running into issues with issuance. We would see gateway timeouts on the ACME endpoint, which would result in certificates being issued and provisioned, but not retrieved in time. The order would expire, and the cycle would repeat until a valid certificate could be retrieved. No more

Instead of using the ACME endpoints, moving forward, Caddy will be utilizing the ZeroSSL API to provision and retrieve certificates. In our testing so far, this has solved our duplicate certificate issue, and presented a few other benefits as well:

Noticeably faster response times
IP certificates
(For us) better reporting in the ZeroSSL dashboard

ACME Renewal Information (ARI) Support

ARI is a draft specifying how an ACME provider (in our case, ZeroSSL,) might be able to suggest renewal times to a client. Currently, ACME clients periodically evaluate their certificates for any expiring soon and attempt to renew at their convenience.

The goals of ARI are relatively straightforward:

Establish a mechanism by which ACME servers may provide a suggested renewal window to ACME clients.
Prevent load spiking for Certificate Issuers.
Help CAs more easily change certificate lifetimes.

You can read the full ARI draft at this link.

HTTP/3 QUIC to reverse_proxy backends

This is one we're excited for! For those who don't know, HTTP/3 QUIC is a major improvement to the HTTP protocol. We've covered HTTP/3 before and how it greatly improves the performance of our CDN.

Currently, the flow of traffic through our network starts at the edge. A user makes a request to cdn.skip2.net, which resolves to the nearest edge server. If the response to the request is cached, the edge server sends it and we're done. If the response isn't cached, we have to proxy from one edge of our network to another. Currently, this happens over HTTP/2 with TLS. We always encrypt traffic in-transit even though the transit occurs through an overlay VPN. We're still testing, but we anticipate that upgrading this backend connection from HTTP/2 to HTTP/3 with QUIC should yield some performance improvements while data traverses our network.

Odds & ends that are worth mentioning

Sensitive HTTP headers are now replaced with "REDACTED" instead of nothing
Health checks can now be configured with consecutive passes (or fails) to change backend status
TLS curves can be configured in preparation for Post Quantum Cryptography (PQC)

Our Appreciation

We want to extend our heartfelt thanks to the Caddy development team and community contributors. This startup could not exist without Caddy.

You can read the full release notes here

Learn more

Get Started