Skip to main content

Deploy a multi-cloud API gateway

You've developed a world-class API, and now you want to make it available online on not just one cloud provider, but two or more.

Aside from the challenges involved in any multi-cloud deployment, you're looking for a multi-cloud API gateway that allows you to consistently apply security and traffic management policy in one place, provide a single pane of glass for observability, provide high availability with automatic failover, and work identically in every cloud or environment.

In this tutorial, you'll learn how to implement ngrok as a multi-cloud API gateway with these broad steps:

  1. Set up the common pattern of cloud and internal agent endpoints, which uses a single cloud endpoint to accept all API traffic and route it to internal agent endpoints via the forward-internal Trafic Policy action.
  2. Use endpoint pooling to enable dead-simple load balancing between replicas of your API service.
  3. Apply Traffic Policy rules to your endpoints to manage traffic according to common API gateway use cases.
info

This guide uses endpoint pools, which are not yet generally available. To use them, you must request access to the developer preview.

What you'll need​

  • An ngrok account: Sign up for for free if you don't already have one.
  • Your authtoken: Create an authtoken using the ngrok dashboard.
  • A reserved domain: Reserve a domain in the ngrok dashboard or using the ngrok API. You can choose from an ngrok subdomain or bring your own custom branded domain, like https://api.example.com. We'll refer to this domain as {YOUR_NGROK_DOMAIN} throughout the guide.
  • The ngrok agent: Download the appropriate version and install it on the same machine or network as the API service you want to make available via ngrok's API gateway.
  • (optional) An API key: Create an ngrok API key using the ngrok dashboard if you'd like to use the ngrok API to manage your multi-cloud API gateway setup.

Deploy a demo API service (optional)​

If you don't yet have API services you'd like to bring online with a multi-cloud API gateway, or just want to quickly wire up a POC using ngrok, we recommend running multiple "replicas" of httpbin, which is simple HTTP request and response service.

Assuming you have Docker installed on the systems where your API services run, you can deploy an httpbin container listening on port 8080.

Loading…

Repeat this process on your other cloud providers where you want to run replicas of your API service.

Bring your multi-cloud API gateway online​

Create internal agent endpoints to add API services to an endpoint pool​

When your create two endpoints with the same URL (and binding), those endpoints automatically form a "pool" and share incoming traffic. For handling API traffic in a multi-cloud environment, you'll create two internal agent endpoints on the same URL to form the pool and load balance traffic between them automatically—no complex networking or cloud-specific tools required.

In one cloud, create an internal agent endpoint on an internal URL, like https://foo.internal. Replace 8080 with for an API service you've brought yourself.

Loading…

Run the same command on other clouds for each replica of your API service.

These internal agent endpoints are now in a single endpoint pool with traffic automatically load balanced between them.

They aren't, however, accessible on the public internet. To fix that, you need two things:

  1. A cloud endpoint for traffic routing and centralized policy management.
  2. A Traffic Policy rule with the forward-internal action, which lets you specify which internal URL—in this case, your endpoint pool at https://foo.internal—to forward traffic to.

Create a cloud endpoint for your API​

Cloud endpoints are persistent, always-on endpoints that you can manage with the ngrok dashboard or API.

You centrally control your traffic management and security policy on this cloud endpoint, then forward traffic to your endpoint pool, rather than trying to apply policy separately for each replica and cloud.

First, log into the ngrok dashboard. Click Endpoints → + New.

Leave the Binding value Public, then enter the domain name you reserved earlier. Click Create Cloud Endpoint.

With your cloud endpoint created, you'll see a default Traffic Policy in the dashboard. Paste in the YAML below to apply the rule.

Loading…

Click Save to apply your changes.

Access your API services​

At this point, you have a front door to your multi-cloud API gateway with a cloud endpoint, which in turn forwards traffic to your endpoint pool for automatic load balancing between clouds.

If you're running httpbin as a demo API service, send a query to the /get route—if you've brought your own service, change the route accordingly.

Loading…

With httpbin, you'll see a response like:

Loading…

At this point, you have a functional multi-cloud API gateway with ngrok. Requests to {YOUR_NGROK_DOMAIN} are forwarded to your pool of internal agent endpoints and load balanced between them, plus automatic failover if one of your replicas—or entire clouds—goes offline for any reason.

You can now start to take full advantage of Traffic Policy, our configuration language for managing traffic. You can attach Traffic Policy rules to both your cloud and agent endpoints, allowing you to compose some rules across all your APIs and others to individual upstream services or replicas.

See our API gateway traffic management for details.

Extend your multi-cloud API gateway with non-replicated services​

So far, you've deployed multiple replicas of a single API service and added automatic load balancing with an endpoint pool.

What if you have more than one API service you'd like to make available behind a multi-cloud API gateway? Or you've acquired a new business and want to integrate their API services into your existing API gateway?

You'll need to add additional forward-internal actions and implement a strategy for routing to different upstream services based on the qualities of incoming requests.

Let's assume you want to start a new bar service, which runs on port 9090. Create a new internal agent endpoint on a unique URL, like https://bar.internal, to separate it from your pool of replicated services on https://foo.internal.

Loading…

When you send another request to {YOUR_NGROK_DOMAIN}, ngrok forwards it to https://foo.internal because your Traffic Policy rules don't yet contain a strategy for routing traffic to multiple API services. ngrok has a few common patterns to fix that.

Route by path​

Path-based routing enables you to direct traffic to different backend services based on the value of the path.

The Traffic Policy file below:

  1. Routes requests to https://{YOUR_NGROK_DOMAIN}/foo to the https://foo.internal endpoint pool.
  2. Routes requests to https://{YOUR_NGROK_DOMAIN}/bar to the https://bar.internal agent endpoint.
  3. Uses the custom-response action to capture requests to any other path and respond with a generic 403 error.
Loading…

Apply this Traffic Policy rule to your cloud endpoint through the dashboard or API.

When you send additional requests to either the /foo or /bar path, ngrok will forward them accordingly.

Route by headers​

You can also route to various internal agent endpoints or endpoint pools based on the value of a header, and do dynamically, with CEL interpolation.

The Traffic Policy file below:

  1. Checks whether requests contain a x-api header, and if so, forward to the internal URL with the same name using CEL interpolation.
  2. Uses the custom-response action to capture requests without an appropriate header and respond with a generic 403 error.
Loading…

Apply this Traffic Policy rule to your cloud endpoint through the dashboard or API.

When you send additional requests with a x-api: foo or x-api: bar path, ngrok will forward them accordingly.

What's next?​

You've now brought your multi-cloud APIs online with ngrok's API gateway, which includes features like DDoS protection and global load balancing.

Continue your API gateway tutorial with adding and composing traffic management rules to finish offloading non-function requirements, like rate limiting and authentication, to your ngrok API gateway.

Check out your Traffic Inspector (documentation) to observe, modify, and replay requests across your API gateway.

Explore other opportunities to manage and take action on API traffic in our Traffic Policy documentation.