API’s Security 101 (2022 edition), Part 2: The API Gateway
Welcome to the second and last part of this short series about API Security!
The goal is to get a simple hands-on first experience on basic API security concepts practices for API developers and architects that don’t have a lot of experience in the area. If you are a security specialist or you already know about how to attack an API from several points of view, you probably shouldn’t waste your time reading this post.
Firstly, we’ll go through the deployment of a target API and an API Gateway in our local environment to do some basic security actions in that API. Then, we’ll review some types of vulnerabilities in APIs along with some basic mitigations, not modifying the implementation of the upstream API service but exclusively working at the API Gateway level.
At the end of the practice, I think you’ll understand the nature of basic API vulnerabilities, you’ll be aware of how vulnerable APIs can be if you are not careful, and how these attacks work. By doing it by yourself, you’ll understand much better the mechanism behind API attacks and how they can be mitigated in a fast and cheap way using the API Gateway (but always remember, best API protection is a defense in depth, no trusting on a single component). By applying the mitigation, you’ll check out the importance of these measures and protections in your APIs.
Now, please read carefully and try to do this at home!
Step 1: Let’s See some Basic Vulnerabilities in APIs and Quick Mitigations
The first thing we’re going to need is a vulnerable API to play. Let’s take the popular Tiredful API which is basically a Django app with some vulnerabilities we have to find. It will provide us with a proper victim to do our experiment about API Security. Let’s install this in your machine running as a Docker container. Just follow the steps described here.
As you can see in the introduction page, Tiredful API will be useful to get some basic challenges.
We have used Postman as our API client. So, any mention to calls to TiredfulAPI are referred to Postman and shown in the screenshots. You can use curl, httpie, Insomnia or any other client to send your requests to the target API.
Let’s have a look at the main vulnerabilities and its matching with the OWASP API Top 10:
INSECURE DIRECT OBJECT REFERENCE (API1:2019 Broken Object Level Authorization)
In the Tiredful API we have to obtain the access token by using the user/password of one of the provided users, batman or superman. Go to the /handle-user-token/ path in the TiredfulAPI UI app and enter the user and password. Then, use the token as the value of the Authorization header (Authorization: Bearer <token>) in the requests using your favourite API client.
What’s the issue with this? Well, we’ve obtained a valid token after our authentication. We used this token to get an authorized access to the API, which is the expected right behavior. The vulnerability consists of the lack of authorization to access data. Any authenticated to the TiredfulAPI can get all data without restriction. This is shown in TiredfulAPI in the path /exams/<entityID>. We can get all the items of a user simply by including the item ID (encoded in base 64 ) in the path ( /api/v1/exams/<entityID>). We can find some provided user IDs in TiredfulAPI. Let’s use the base64 encoded IDs NTY= and OTM= to show this issue. As you can see in below screenshots:
INFORMATION DISCLOSURE (API3:2019 Excessive Data Exposure)
This vulnerability is based on this: if the requestor sends an unexpected query resulting in an error, the API includes in the response meaningful stack trace information. To reproduce this, use the path /api/v1/books/978–93–80658–74–2 (use ) to get the record for the ISBN in the path.
Modify the value of the ISBN (for instance, remove or change a character) and you’ll find the API returns the stack trace information.
What’s the issue with this? This flaw allows attackers to sniff API traffic and analyze the responses for any sensitive information. In the TiredfulAPI case, stack trace can be used for subsequent attacks.
ACCESS CONTROL (API2:2019 Broken User Authentication)
In this case, the API allows GET to anonymous users.
The DELETE method is forbidden, you need to be an administrator.
But the authorization is based on the value of a simple header (isAdmin: True). This can be overridden simply by including the isAdmin header in the request with the value true. You can see below the successful deletion of an item by an anonymous user.
What’s the issue with this? A non authorized user can perform actions that are not allowed by tampering certain information in the request.
THROTTLING (API4:2019 Lack of Resources & Rate Limiting)
The Tiredful API includes this path /api/v1/trains/ with a generic rate limit, allowing 10 requests to the API with anonymous user (without the Authorization header) and 20 requests with authenticated request(with the Authorization header).
Poorly managed Rate Limiting could be the cause of a grave security issue. By not having a planning for Rate Limiting in your APIs you cannot face up-front what is the expected traffic and therefore, you cannot prevent DDOS attacks. Also, this is a problem of cost and maintenance. If the Rate Limit is implemented in the API service (which is not the best option as it has to be maintained) you are missing multiple key features (multiple instances of the service, centralization, response headers management, etc).
What’s the issue with this? In the sequence below, the response does not return any header with information related to throttling which doesn’t allow the API consumers to be informed about the remaining quota.
SQL INJECTION (API8:2019 Injection)
Attackers will feed the API with malicious data through whatever injection vectors are available (e.g., direct input, parameters, integrated services, etc.), expecting it to be sent to an interpreter. Injection flaws are very common and are often found in SQL, LDAP, or NoSQL queries, OS commands, XML parsers, and ORM. These flaws are easy to discover when reviewing the source code. Attackers can use scanners and fuzzers. Injection can lead to information disclosure and data loss. It may also lead to DoS, or complete host takeover.
Source: OWASP API Security
In this case, the interpreter is the relational database. This case applies to API implementations that use a relational database and send queries composed from the parameters in the request to the API. A poor implementation of the API, no evaluating, filtering and sanitizing the provided parameters can result in an injection of SQL. To test this vulnerability, Tiredful API includes the Injection vulnerability related to SQL queries to the database, with the /activities path. To see how it works, we send a POST request with a query, including one parameter (month) and its value (11).
TiredfulAPI does not filter the queries to the database, which is the real vulnerability here. In this case, the number of the month in the request body is used as a value for the SQL query. Assuming this, the attacker gets the response body and counts the number of columns. As you see above, the response body contains 6 name-value pairs which means that the number of columns used in the database is 6 or greater. The thing is, an attacker can guess the structure of the data coming from the database. Then, the attacker can replace the value of the parameter in the body for a SQL query. He‘d be able to get all the records and the table names in the response body.
What’s the issue with this? Queries to the database via API must be validated, filtered, and sanitized. Otherwise, the attacker can be able to get all the records in the queried tables in your database. Use the path /api/v1/activities to find it out.
CROSS SITE SCRIPTING (API8:2019 Injection)
This vulnerability affects to web applications running in the web browser of users. This vulnerability is similar to SQL Injection, in this case inserting non-sanitized content in the database via API, so that the content consumed by the application in the web browser can be vulnerable to Cross Site Scripting attacks.
To see this vulnerability in action using Tiredful API, let’s use the path /api/v1/advertisements/ to create a new record, POST method.
Then, let’s try to post some content with the <script> tag so that it can be executed in the browser when the API is used by our API consumer, the web application running in the web browser. You can see the regular insertion and the polluted insertion below:
What’s the issue with this? Well, the API allowed the insertion of a record in your database with the script tag with potentially executable code in the client side. When the API consumer (e.g. a web application) gets the records to be displayed using a web browser, this script will run in the web browser of your final users. It is not good for the reputation of the apps that are your API consumers!
Conclusion? We’ve got here a nice collection of vulnerabilities in a really flawed API that match with some of the OWASP API Top Ten
Step 2: Put An API Gateway In Your Life
For starters, let’s deploy an API Gateway. We’ll use Kong API Gateway running on the target machine as a docker container. You can run Kong API Gateway and its database using docker-compose and this file.
We are going to use the Kong version 2.8.1, DB Less mode (KONG_DATABASE=off). That means that we are not going to need a database container along with the API Gateway. You can see below the docker command to run the Kong container.
#!/bin/bashdocker run -d --name kong-gateway \
--network=kong-net \
-v "$(pwd):<YOUR PATH HERE>/kong" \
-e "KONG_DATABASE=off" \
-e "KONG_DECLARATIVE_CONFIG=<YOUR PATH HERE>/kong.yml" \
-e "KONG_PROXY_ACCESS_LOG=/dev/stdout" \
-e "KONG_ADMIN_ACCESS_LOG=/dev/stdout" \
-e "KONG_PROXY_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_ERROR_LOG=/dev/stderr" \
-e "KONG_ADMIN_LISTEN=0.0.0.0:8001" \
-e "KONG_ADMIN_GUI_URL=http://localhost:8002" \
-e KONG_LICENSE_DATA \
-p 8000:8000 \
-p 8443:8443 \
-p 8001:8001 \
-p 8444:8444 \
-p 8002:8002 \
-p 8445:8445 \
-p 8003:8003 \
-p 8004:8004 \
kong/kong-gateway:2.8.1.0-alpine
Note the property KONG_DECLARATIVE_CONFIG is needed, pointing out to the YAML file with the configuration of our API Gateway, including plugins. Find below a proposed configuration with the routes we are going to use. There aren’t routes added yet. We’ll include the routes in each example:
_format_version: "2.1"
_transform: trueservices:
- host: <YOUR PRIVATE IP HERE, never 'localhost'>
name: TiredfulAPI
port: 8005
protocol: http
To get your private IP, just use ifconfig (Linux, MacOS) or ipconfig (Windows). You can see some examples here.
Optionally, you could use Kong as a GUI administration tool and Docker Compose is still a good option. You can use examples like this one.
In the end, you should see two containers running in your machine:
% docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
80b6c0e2e92e kong/kong-gateway:2.8.1.0-alpine "/docker-entrypoint.…" 3 weeks ago Up 8 seconds (health: starting) 0.0.0.0:8000-8004->8000-8004/tcp, 0.0.0.0:8443-8445->8443-8445/tcp, 8446-8447/tcp kong-gateway
fa06fd1928fe tiredful "/usr/bin/python man…" 4 months ago Up 5 seconds 0.0.0.0:8005->8000/tcp tiredfulJust remember, Kong runs in DB-Less mode and the admin API will not be available to enter new configurations.
Step 3: Using the API Gateway to Fix API Vulnerabilities
Now, let’s have a look at the proposed mitigations of some of the found vulnerabilities in the TiredfulAPI using Kong API Gateway.
As you can see in the sample configuration file, the TiredfulAPI service is already added and accessible in the port 8005. Let’s iterate the list of vulnerabilities found in TiredfulAPI one more time, now including a mitigation using Kong API Gateway.
INSECURE DIRECT OBJECT REFERENCE (API1:2019 Broken Object Level Authorization)
First things first: Security in Tiredful API is terrible. It requires basic authentication to obtain an access token, I admit, but the implementation of the API doesn’t apply any authorization based on roles or groups to access data. The service doesn’t consider which user is sending the request. That means, any user can see information belonging to other users simply by adding the user ID in the path (the provided access token is validated as active and you can eventually access to everything). This is a clear defective code implementation in the server side, that must implement a role-based authorization mechanism to access data. However, we have to handle with TiredfulAPI right now! How could we solve this problem when no modifications in the code are possible? Let’s see a possible fix based on a recommended mitigation:
- Implement a proper authorization mechanism that relies on the user policies and hierarchy.
- Use an authorization mechanism to check if the logged-in user has access to perform the requested action on the record in every function that uses an input from the client to access a record in the database.
- Prefer to use random and unpredictable values as GUIDs for records’ IDs.
- Write tests to evaluate the authorization mechanism. Do not deploy vulnerable changes that break the tests.
Step 1 is covered by the previous basic authentication, providing an authorization token. Step 2 is the problem because the authorization token is not used to identify the user. We need to identify the authorized user to serve only authorized data.
The API Gateway will help you to identify the user that is accessing your API, for instance from the authentication via OpenID Connect or, in a simpler way, by providing an API Key associated to a given ACL (it stands for Access Control List). So, we can allow or block the access to certain paths that provide access to restricted data. We are going to pretend there is a segregation of data by path/route (Each user belongs to a group authorized to access to data ONLY in that path, being blocked to other paths).
First, let’s split two values as two different paths in separate routes:
/api/v1/exams/OTM=
/api/v1/exams/NTY=You can use the Kong’s RESTful API to manage resources. Modify the configuration file to include two new routes. For instance:
_format_version: "2.1"
_transform: trueservices:
- host: <YOUR IP HERE, never 'localhost'>
name: TiredfulAPI
port: 8005
protocol: http
routes:
- name: UserOTM
paths:
- /api/v1/exams/OTM=/
strip_path: false
methods:
- GET
- name: UserNTY
paths:
- /api/v1/exams/NTY=/
strip_path: false
methods:
- GET
Now, let’s restrict the access to the new routes by assigning specific API consumers. This will help you to correctly identify the user sending the request by using a combination of features:
- API Consumer (we’ll create two new consumers OTM and NYT belonging to groups otm-group and nty-group).
- ACL (Access Control List)
- Key-Auth (API Key for each consumer)
Step1, enable the ACL and Key-Auth plugins (plugins section) in each new route. Then, create the consumers (consumers section) and finally, associate the consumer groups to the ACLs (acls section).
Remember each consumer has to belong to a group. Only groups can be allowed or denied access in an ACL, not consumers.
_format_version: "2.1"
_transform: trueservices:
- host: <YOUR IP HERE, never 'localhost'>
name: TiredfulAPI
port: 8005
protocol: http
routes:
- name: UserOTM
paths:
- /api/v1/exams/OTM=/
strip_path: false
methods:
- GET
- name: UserNTY
paths:
- /api/v1/exams/NTY=/
strip_path: false
methods:
- GETplugins:
- name: acl
route: UserNTY
config:
allow:
- nty-group
hide_groups_header: true
- name: acl
route: UserOTM
config:
allow:
- otm-group
hide_groups_header: true
- name: key-auth
route: UserNTY
config:
key_names:
- apikey
key_in_body: false
key_in_header: true
key_in_query: true
hide_credentials: false
run_on_preflight: true
- name: key-auth
route: UserOTM
config:
key_names:
- apikey
key_in_body: false
key_in_header: true
key_in_query: true
hide_credentials: false
run_on_preflight: trueconsumers:
- username: otm
keyauth_credentials:
- key: otm-key-auth
- username: nty
keyauth_credentials:
- key: nty-key-authacls:
- consumer: otm
group: otm-group
- consumer: nty
group: nty-group
We’ll test now the consumer “OTM” and its route. It can access OK by including the right API key (apikey:otm-key-auth). On the other side this consumer, using the same API key, cannot consume the other route and receives a 403 Forbidden error.
It’s the same for the consumer “NYT”, unable to access the other path with its API key:
Of course this is not the root cause of the problem (the lack of a decent implementation following role based authorization) and it will be boring and not scalable when the number of users increases. Nevertheless, and just as a piece of example, the API Gateway has been useful to provide authorized access to different paths.
INFORMATION DISCLOSURE (API3:2019 Excessive Data Exposure)
To solve this problem we’ll have to apply the Kong’s Response Transformer Advanced Plugin to detect, filter and eventually block any information in the response body to be replaced by an error message according to the response status code at the API Gateway Level.
- name: response-transformer-advanced
route: InformationDisclosure
config:
replace:
body: '{"error": "internal server error"}'
if_status:
- "500"Remember the InformationDisclosure route exposed the stack trace content of the error.
By applying this plugin, when a response status code 500 is got, the response body is replaced by a pre-defined message.
ACCESS CONTROL (API2:2019 Broken User Authentication)
Kong API Gateway will be extremely useful to ensure a proper Access Control in the API is guaranteed. To be completely clear, this is about Authentication, not Authorization. So, let’s remind some basic points:
- Make sure you know all the possible flows to authenticate to the API (mobile/ web/deep links that implement one-click authentication/etc.)
- Ask your engineers what flows you missed.
- Read about your authentication mechanisms. Make sure you understand what and how they are used. OAuth is not authentication, and neither is API keys.
- Don’t reinvent the wheel in authentication, token generation, password storage. Use the standards.
Remember, you’ll find many articles about API Security including API keys and OAuth in the Authentication section. This is simply wrong, I’m sorry.
So, we’ll use Kong’s mTLS Authentication plugin to apply authentication and authorization based certificates issued by a secure CA for applications accessing the API. You can find a full explanation about mTLS here but it is not complicated. Of course, devil is in details and the complicated thing in mTLS is management and provision of certificates.
We’ll follow these steps in the scope of our exercise, using the mentioned mTLS Kong plugin and OpenSSL:
1-First step, create your self-signed CA. I strongly recommend to follow these steps to get your CA certificate with OpenSSL.
2-Then, once the CA certificate is created from the previous step, let’s create the client certificate to use for authentication of your requests, which consists of the client certificate private key and certificate signing request (CSR):
openssl genrsa -out api_client.key 2048
openssl req -new -key api_client.key -out api_client.csr3-Now, we are going to sign the created client cert by using the certificate authority you previously created:
openssl x509 -req -in api_client.csr -CA <path to your CA certificate here> -CAkey <path to your CA private key here> -set_serial 01 -out api_client.pem -days 3650 -sha2564-At this point, we’ve created a self-signed certificate and have also created the basic entities to use this certificate. Now, let’s include this CA certificate in the configuration of Kong in a new section of our kong.yml file. The value of the ID field is provided by you:
ca_certificates:
- id: 12e2b302-c987-4841-bed0-ff9d2745d142
cert: |-
-----BEGIN CERTIFICATE-----
...
-----END CERTIFICATE-----5-We have to verify the ID of the server certificate to be passed as a value in the configuration of the mTLS plugin (config.ca_certificates). We should get the list of CA certificates in Kong (find below). You can store CA certificates (ca_certificates) and client certificates (certificates) in Kong configuration, that would be kept as secrets in Kong on Kubernetes:
If you get a result like the one above, it means that the certificate was successfully registered in Kong.
6-Apply the Kong mTLS plugin to the route including the ID of the CA certificate as a value in config.ca_certificates (it’s an array):
- name: mtls-auth
route: RateLimit
config:
ca_certificates:
- 12e2b302-c987-4841-bed0-ff9d2745d142
revocation_check_mode: SKIP
skip_consumer_lookup: false
cache_ttl: 607-Finally, let’s verify Kong can proxy requests when a consumer sends the client certificate. At this point, Kong should be completely set up and it’s time to test that the confidential service only works when the certificate and key are sent. Depending on the http client you’re using (Postman, curl, httpie) the way of sending the certificate in the request will change. Anyway, this can be done by doing the following request. As you can see, we changed the URL to use TLS (HTTPS) in Kong’s port 8443:
Try to send the same request without certificate. You should receive a 401(Unauthorized) error:
If you send a non-valid certificate, you should get the same result with a different error message:
Just to conclude, Kong’s mTLS authentication plugin will identify the API consumer from its certificate and you can apply other policies to the specific consumer (e.g. Rate Limiting). You can attach a specific client certificate to each consumer. Read the Kong’s mTLS plugin documentation to know more about the possibilities.
THROTTLING / RATE LIMIT (API4:2019 Lack of Resources & Rate Limiting)
Let’s apply the Rate Limiting Kong Plugin. We have to be careful here. Rate Limiting in APIs has different aspects:
- Burst Rate Limit
- Time Limits
- Security/Resiliency Limits
- API Consumer Quotas
In this example we’ll just apply a naive rate limit per second (5 RPS) and hour (10 RPS). Add this to the Kong’s Rate Limit plugin in the kong.yml file, plugins section:
- name: rate-limiting
route: RateLimit
config:
second: 5
hour: 10
policy: local
fault_tolerant: true
hide_client_headers: false
redis_ssl: false
redis_ssl_verify: falseAfter 10 requests you’ll get a 429 error message:
The Kong Rate Limit Advanced plugin offers more capabilities and flexibility. It is the tool that will allow us to define a Rate Limit Plan to know:
- When are we expecting peaks in HTTP traffic?
- What services are going to be affected?
- What’s the volume of expected traffic peaks?
- What parameters of rate limiting are we going to need? (burst limits or RPS, quotas, security limits, etc).
It is extremely important to detect unexpected traffic, and before changing the rate limit parameters we have to ensure we are not facing a DDOS attack. So, the best advice is:
- Study the documentation of the Rate Limit plugins (basic, advanced).
- Make a plan.
- Watch the logs and set alerts and notifications.
- Stop unexpected traffic until you’re completely sure it’s not hostile.
Insufficient Logging (API10:2019 Insufficient Logging & Monitoring)
This vulnerability is too often not well understood by API producers. It means, the API service doesn’t log auditable events: failed login attempts, high value transactions, logins, etc. Also, warnings and errors are not logged well and they don’t exist or are not immediately accessible. The result is, the API service cannot detect and react to active attacks. In the post-mortem analysis, it doesn’t store enough information.
To fix this, we have to improve the logging level with sufficient information context while storing logs time long enough to allow forensic analysis while also stored in a centralized management solution. Alerts and monitoring of suspicious activities must be detected and handled immediately.
In the case our service implementation doesn’t include a decent level of logging and alerting to detect intrusion attempts and anomalies, we can again use the API Gateway to mitigate the problem. But this is a combination of factors. The core is, the Kong API Gateway proxy log:
Authentication. By using Open ID Connect plugin failed attempts. Also, the IDP will log all the activity and will be an invaluable source of information.
Rate Limit. We can detect suspicious and unexpected requests coming even when we are already returning 429 error response.
ACL. The API Gateway will detect and log any failed attempt to access to unauthorized paths with 401 response errors.
Of course, they are just a few examples. Other API features implemented via plugins will enrich the API logs with events about security that have to be used to raise and trigger alerts and notifications. It’s a task for you. We assume your Kong API Gateway is integrated with a centralized log solution like ELK.
Conclusions of This Exercise?
We have seen only a few examples about how we can mitigate vulnerabilities found in API back-end services. This is compliant with the principle of make dumb services, use a smart gateway. What are the advantages of this approach?
- It’s cheaper and fast, because you don’t have to implement those features on your own in the code of your back-end services. For instance, the cost of using OIDC Kong plugin vs the implementation of Spring Security in a Spring Boot microservice is absolutely clear: you’ll save a lot of development time and money using Kong. Focus on your business logic and let the API Gateway to do the hard non-functional work.
- Less changes and maintenance. Implementation on API services depends upon configuration and a new release can be needed too many frequently. By using Kong plugins you do not have to re-release your code and again, while you are strictly focused on the functional requirements.
- Smaller technical debt. Usually the team in charge of the API Gateway and other infrastructure is different to development teams. They will keep up to date the API Gateway, integrating the last features and versions. So, you reduce your technical debt risk because it’s not in the scope of the development team, which is focused on keeping up to date their software in a much simpler way.
Q&As
- Is the API Gateway the Silver Bullet to fix security issues in APIs? Our opinion is clear: It’s NOT! But security at the API Gateway level helps to build better APIs and it is a mandatory component in Enterprise APIs, a tool that helps to implement a solid security policy.
- Do we need additional external services, e.g. WAF? If your API is public, no doubt about that! WAF, API Protection, Bot Detection (Kong also provides this feature) etc. combined with a wise configuration of the API Gateway is a winner combination.
- Should all APIs be available in Internet? This is a big NO! Consider the exposure and visibility of your API. Restrict the network access to your API if it has to be restricted to certain consumers in the same network. For APIs exposed to the Internet, IP-Restriction would be also an excellent measure to restrict traffic to your API, combined with WAF.
- What About Zero-Trust Policies? You can apply this policy to all APIs in your API Gateway (exposed to the Internet or not) applying authentication and authorization to all requests. Remember OAuth2 handles authorization and OpenID Connect manages both as it is built on the top of OAuth2. Mutual TLS authentication is also great if you have solved the problem of the issuer CA and your API Gateway is able to identify the owner of the client certificate provided by the API consumer to apply authorization restrictions.
- Could the API implementation help to increase the level of security of APIs? Again, no doubt about that! Defensive programming, scratching the paranoid level should be mandatory and taught even in primary schools! Naive implementations are behind many exploits of vulnerabilities in APIs. Other components along the way like the API Gateway can only mitigate generic issues and never go deeper into vulnerabilities in the business logic that must be protected by the API service implementation. Think of API Security as a defense in depth, never as a single barrier to attackers relying on a single component. All components in the API architectural blueprint should be involved in security concerns.
I hope you’ve enjoyed this hands-on exercise (BTW, you can find the script to run Kong here and the final Kong’s configuration example here). Nevertheless, the list of vulnerabilities is long and we have handled just a few of them. I encourage you to try at home with your favorite flawed APIs and API Gateways to keep on working on the OWASP API Top Ten.
Please, don’t forget to send your comments and have a good and fruitful API security experience!
The views, thoughts, and opinions expressed in the text belong solely to the author, and do not represent the opinion, strategy or goals of the author’s employer, organization, committee or any other group or individual.
