AWS API Gateway HTTP APIs with Keycloak JWT authentication - Part 1

AWS API Gateway is an AWS service to "front" a variety of AWS services by providing an HTTP front end.  One common use is to access logic coded into an AWS Lambda to allow services and web browsers to access the Lambda and it's services.  A Lambda is one of the main tools for serverless development in the AWS ecosystem.

If your API Gateway service is public (meaning it's not enclosed within a VPC) then anyone in the world meaning can use (and abuse) your API.  Therefore, it is imperative to have some sort of validation check on who is calling the API and to make sure the caller is authorized to interact with the API.

So this post will show you how to use one of the types of API Gateway authorization, JWT authorizers.  JWT is a standard for tokens that are passed (usually over HTTP) from a consumer to a service.  It is most commonly used in Oauth2 environments.

API Gateway has two synchronous ways of interacting with it, along with a Websocket integration:

1. HTTP API - a simple way of leveraging API Gateway.  HTTP endpoints are simple to setup and expose.  We will be using this pattern for this article.
   
2. REST API - a more expansive integration pattern.  The REST API endpoints allow for nearly complete control over the input and output of the calls.  A future post will address security with this type of integration.

One thing that makes the HTTP API simple to integrate is that it has built in JWT authentication.  This allows you to set up and API that can be pointed to any Oauth2 server to validate the token.

For this article I'll use my own Keycloak instance.  Keycloak is an open-source identity and access management platform.  It's super simple to get running and, as my Grandmother would have said, "the price is right" - free!

Note: in the code below I am showing you the "ClickOps" method of setting up your environment by doing everything though the AWS console.  This is not a best practice by any means.  Best practice is to use a toolkit such as AWS CDK or others to have an IaC (Infrastructure-as-Code) environment.  An IaC environment allows you to reproduce your overall infrastructure easily and allows much simpler auditing.  But this is a demo - it's not meant to be the be-all and end-all for AWS best practices.

For this article I'll be using my AWS Lambda API Gateway code.  This is a very simple Java Lambda that calls another service (https://icanhazdadjoke.com/) purely for demonstration purposes.  That and I like Dad jokes.

This post isn't meant to be an in depth examination of Lambda but I'll give you the basics.  Again, I'm using the AWS console for this but there are better ways long term.

Start by cloning the repository.  It requires Java 21 and Maven.  Maven 3.9.3 was used for this code but other versions are likely to work.  The instructions for building are in the source code but generally it should just be:

mvn clean package

The output will be in target/api-gw-service-1.0.jar in your local machine.  Using the AWS console, the basic deployment steps are:

Go to Lambda in the AWS console - make sure you're in the region that you want:

If you haven't created a function before, select the "Create a function" button:

Otherwise, if you have other Lambda's, select the "Create function" button:

In the next screen, decide on a name for your function - it's up to you besides the AWS restrictions.  Make sure you select a "Runtime" of Java 21 though and then "Create function":

After clicking "Create function" it will take a moment or two to show you the definition of the function.  There are multiple things to configure but let's start by uploading the code.  Under "Code source", select the "Upload from" drop down and choose ".zip or .jar file":

A dialog will popup.  Select "Upload" and then navigate in your file system to the location of the api-gw-service-1.0.jar file.  This file should be about 1.4 MB in size or so. Important: do not upload the file original-api-gw-service-1.0.jar.  It will be about 4-5kb and does not contain the contents you need.   Once you've selected it in the dialog, select "Save".  It will take a moment or two to upload your file.  Next, let's update the "Runtime settings".  Scroll down to the "Runtime settings" area and select "Edit":

A new page will display.  Update the "Handler" section to have com.hotjoe.handler.HttpClientLambdaHandler::handleRequest for the the Handler:

Don't change the "Runtime" or the Architecture (though the Java code will run on either x86_64 or arm64).  Select "Save".

One more edit in the Lambda area and we're done.  From the default screen of the Lambda, select the "Configuration" tab and then "General configuration":

Select the "Edit" button in the General configuration area and, in the next screen, update the "SnapStart" selection to "PublishedVersions":

This enables Lambda to keep a version of your Lambda in a state that allows faster restarts.  Leave the rest of the parameters the same for now.  This is a very small Lambda and doesn't use anywhere close to the default of 512MB of memory.  And no ephemeral storage is used at all so that setting doesn't matter.

Once you've selected "Save" on this screen you'll be back to the Lambda function "home page" for your function.  We still need to setup API Gateway but let's stop and test here first to make sure that Lambda is setup correctly.

Select the "Test" tab in the middle section and then select the "Test" button to the right.  If everything worked right after a few seconds you should get a green box indicating success:

If you want to see the output, select the "logs" link in the green box to see the CloudWatch log group:

Select the "Log stream" (yours will be named differently from mine) to display the logs.  You can dig in to see what the dad joke API got (it will be nothing because the code expects an API Gateway request) and what it sent back.  Note that the very first time you run the Lambda this way it will take 3 to 4 seconds to run.  The next time will be closer to 150ms.  This is the reason you enabled SnapStart as Java Lambdas can have a high startup time but, once cached, are very fast.

 

This is part 1 of the post - basically getting the Lambda setup.  In the next post I'll show you how to tie it to the Keycloak server.

Generating JWT's using the Auth0 library

I've created a small example set of code to generate and learn about JWTs.  This code allows you to create and sign your own JWT.  You can create your own public/private keys to sign and verify the tokens.

In and of itself this code would not likely be used for a production environment but knowing how a JWT works and the part of it are an important part of understanding Oauth2 in general.  I've got some future code to show you how to use a JWT in more of a production environment but this is useful to learn from.

Starting SSO with Keycloak

 I've been using Keycloak for years now and have been experimenting with the newer versions that are based on top of Quarkus.  One of the struggles I've had is spinning up a test server for development.  The new Quarkus model though is pretty simple.  A small Docker Compose script let's you spin up an environment in almost no time.

My script looks like:

version: '3.8'

services:
  postgres:
    image: postgres:latest
    environment:
        POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
    restart: unless-stopped
    healthcheck:
      test: [ "CMD-SHELL", "pg_isready -U postgres" ]
    networks:
      - pg_network
    volumes:
      - ./pgdata:/var/lib/postgresql/data
      - ./create_db.sql:/docker-entrypoint-initdb.d/create_db.sql

  keycloak:
    image: quay.io/keycloak/keycloak:latest
    depends_on:
      postgres:
        condition: service_healthy
    environment:
      KC_DB: ${KC_DB}
      KC_DB_URL: ${KC_DB_URL}
      KC_DB_USERNAME: ${KC_DB_USERNAME}
      KC_DB_PASSWORD: ${KC_DB_PASSWORD}
      KEYCLOAK_ADMIN:  ${KEYCLOAK_ADMIN}
      KEYCLOAK_ADMIN_PASSWORD: ${KEYCLOAK_ADMIN_PASSWORD}
      KC_PROXY: edge
      KC_HOSTNAME_URL: ${KC_HOSTNAME_URL}

    volumes:
      - /tmp:/opt/keycloak/data/import

    networks:
      - pg_network
    restart: unless-stopped
    ports:
      - 8080:8080
    entrypoint: /opt/keycloak/bin/kc.sh start

networks:
  pg_network:
    driver: bridge

So what does this do?   Ultimately we start a PostgreSQL database server and then Keycloak.  The PostgreSQL server is the current latest one.  A small script:

CREATE USER keycloak WITH PASSWORD 'keycloak';
CREATE DATABASE keycloak with
    owner = keycloak
    encoding = 'UTF8';

is used to create a Keycloak user and password.  This script is meant for development so I'm not too concerned about the database password.  Plus that, we never expose the PostgreSQL port to the outside world so it really doesn't matter - think of the database running in the private network of a VPN.

Next, we startup Keycloak.  Most of the configuration comes from a local .env file:

#
# postgres env vars
#
POSTGRES_PASSWORD=postgres


#
# keycloak env vars
#
KC_DB=postgres
KC_DB_URL=jdbc:postgresql://postgres:5432/keycloak
KC_DB_USERNAME=keycloak
KC_DB_PASSWORD=keycloak

KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=changeme!

KC_HOSTNAME_URL=http://localhost:8080

that allows you to separate out the code from the credentials.

Running the server

To run the server, you don't need to build a local Docker.  Just run:

docker compose -f docker-compose.yml up

This will start the PostgreSQL server along with Keycloak. As part of this it will create the local directory pgdata/ that contains the PostgreSQL database. Note that on Unix-based environments like Linux this directory may have files that are owned by root that will require superuser permissions to remove.

Now you can connect to http://localhost:8080/. You'll be able to log into Keycloak with the user admin with password changeme! as specified in the .env file.  See the Keycloak docs to get started.

Code

The code above is available at https://github.com/stdunbar/keycloak-docker