Tips: Design Secure API
Designing a secure API is crucial for data protection, to prevent abuse like DDoS attack or SQL injection, and/or other malicious activities that may adversely impact the integrity of distributed architecture and to ensure API for running smoothly without interruptions from security breaches. Before getting into more details on secure api, lets understand about api endpoint and different api options supported in distributed architectures.
API Endpoint
Definition: An API or application programming interface is a set of rules and protocols that allows two applications to communicate with each other. Each api define an actions on particular resource. Resource means a particular data, like if api end point defined to get all user info, the user info is the resource, api end point serve. There are different types of APIs supported.
REST - Creating a RESTful API involves defining endpoints that clients
can interact with using standard HTTP methods (GET, POST, PUT, DELETE, etc.).
Stateless transactions that part of layered system, can be cached for
reusuability.
SOAP - SOAP (Simple Object Access Protocol) APIs use XML to encode their
messages and rely on other application layer protocols, most commonly HTTP
or SMTP, for message negotiation and transmission.
GraphQL - GraphQL is a query language for APIs and a runtime for executing
those queries by using a type system you define for your data. Unlike
REST APIs, which expose multiple endpoints for different resources, a
GraphQL API exposes a single endpoint and allows clients to request exactly
the data they interested for.
gRPC - gRPC (gRPC Remote Procedure Call) is a high-performance, open-source
framework that uses HTTP/2 for transport with Protocol Buffers (protobufs) as
the interface definition language, and provides features such as
authentication, load balancing, and more. It's designed for low latency and
high throughput communication, making it ideal for microservices
architectures.This blog is mostly about REST api. You can find more details about grpc here.
Resource representation and encoding
The resources can be represented in the form of json/xml/yaml/protobuf format both for request and response. The basic objective of these methods is how efficiently (better performance) the data can be serialized and sent it over the communication media and how to maintain human readability.
JSON (JavaScript Object Notation) is the most used and lightweight data-interchange format that is easy for humans to read and write and easy for machines to parse and generate.
{
"name": "John",
"age": 30,
"isStudent": false
}XML (Extensible Markup Language) is a markup language that defines a set of rules for encoding documents in a format that is both human-readable and machine-readable.
<person>
<name>John</name>
<age>30</age>
<address>
<street>123 Main St</street>
<city>Wonderland</city>
</address>
<phoneNumbers>
<phoneNumber type="home">123-456-7890</phoneNumber>
<phoneNumber type="work">987-654-3210</phoneNumber>
</phoneNumbers>
<email />
<isStudent>false</isStudent>
</person>YAML (YAML Ain’t Markup Language) is a human-readable data serialization standard that can be used in conjunction with all programming languages and is often used to write configuration files. YAML uses indentation to represent the structure of data.
person:
name: John
age: 30
address:
street: 123 Main St
city: Wonderland
phoneNumbers:
- type: home
number: 123-456-7890
- type: work
number: 987-654-3210
email: null
isStudent: falseProtobuf: Protocol Buffers (Protobuf) is a binary serialization format a highly efficient both to serialize and deserialize, flexible, compact and language-agnostic encoding format. It is widely used for data interchange between services and for storage.
#### Defining the protobuf schema
syntax = "proto3";
message Person {
string name = 1;
int32 age = 2;
bool is_student = 3;
}
#### Generate the python structure using protobuf compiler
protoc --python_out=. person.proto
#### code generated for python
import person_pb2 # Import the generated module
# Create an instance of the Person message
person = person_pb2.Person()
person.name = "john"
person.age = 30
person.is_student = False
# Serialize to a binary format
serialized_data = person.SerializeToString()
# Deserialize from a binary format
new_person = person_pb2.Person()
new_person.ParseFromString(serialized_data)
print(new_person)JSON is ideal for lightweight, quick, and easy-to-use data interchange, especially for web applications and APIs. XML is suited for complex data structures requiring extensibility, validation, and document-centric data representation. When it comes to performance, protobuf is the most preferred solution.
To ensure the client interpreted json correctly, set content type in the response header to “application/json”. Server side framework set the content type automatically.
Client Server Communication model
Api endpoints can be communicated over HTTP or websocket. HTTP or Hypertext transfer protocol is the synchronous communication protocol with client makes a request and the server sends back a response. HTTP2.0 is the asynchronous communication protocol with some major enhance-ment like uses a binary format instead of the text format used to be in HTTP/1.1, making it more efficient to parse and less error-prone. Multiple requests and responses can be sent in parallel over a single TCP connection, reducing latency and improving throughput. HTTP/2.0 also uses HPACK, a header compression format, to reduce the overhead caused by large or repetitive headers. The server can send resources to the client proactively, without the client explicitly requesting them, improving page load times. Clients can indicate the priority of streams, allowing more important resources to be delivered first.
// Server.go
// OBJECT: Person represents a person with a name and age.
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
}
// HANDLER: handler function to respond with a JSON-encoded Person.
func personHandler(w http.ResponseWriter, r *http.Request) {
person := Person{Name: "john", Age: 30}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(person)
}
func main() {
// API ENDPOINT AND ROUTE
http.HandleFunc("/person", personHandler) // Route for /person endpoint.
fmt.Println("Server is listening on port 8080...")
// SERVER LISTENING TO CLIENT REQUEST.
if err := http.ListenAndServe(":8080", nil); err != nil { // listen to client
log.Fatal(err)
}
}
// client.go
// Person represents a person with a name and age.
type Person struct {
Name string `json:"name"`
Age int `json:"age"`
}
func main() {
// Send a GET request to the server.
resp, err := http.Get("http://localhost:8080/person"). // GET request
// to /person endpoint
...
// CHECK THE RETURN
if resp.StatusCode != http.StatusOK {
log.Fatalf("Unexpected status code: %v", resp.StatusCode)
}
// Read the response body.
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("Failed to read response body: %v", err)
}
// Unmarshal the JSON response into a Person struct.
var person Person
// encoding or deserializing
if err := json.Unmarshal(body, &person); err != nil {
log.Fatalf("Failed to unmarshal response: %v", err)
}
}Where as webSockets provide a full-duplex communication channel over a single TCP connection. They are used for applications that require real-time communication, such as chat applications, live updates, or gaming. Unlike HTTP, WebSocket allows bidirectional communication between the client and the server, meaning either party can send messages at any time.
// Server.go
// upgrade the HTTP connection to a WebSocket connection.
var upgrader = websocket.Upgrader{
ReadBufferSize: 1024,
WriteBufferSize: 1024,
CheckOrigin: func(r *http.Request) bool {
return true
},
}
func handler(w http.ResponseWriter, r *http.Request) {
// Upgrade the HTTP connection to a WebSocket connection.
conn, err := upgrader.Upgrade(w, r, nil). // for websocket, http has to
// upgrade
if err != nil {
log.Println("Error upgrading connection:", err)
return
}
defer conn.Close()
for {
// Read message from client.
messageType, message, err := conn.ReadMessage()
if err != nil {
log.Println("Error reading message:", err)
return
}
log.Printf("Received: %s", message)
// Write message back to client.
if err := conn.WriteMessage(messageType, message); err != nil {
log.Println("Error writing message:", err)
return
}
}
}
func main() {
http.HandleFunc("/ws", handler)
fmt.Println("Server is listening on port 8080...")
log.Fatal(http.ListenAndServe(":8080", nil))
}
// client.go
func main() {
// Connect to the WebSocket server.
serverAddr := "ws://localhost:8080/ws"
conn, _, err := websocket.DefaultDialer.Dial(serverAddr, nil)
....
// Start a goroutine to read messages from the server.
go func() {
for {
_, message, err := conn.ReadMessage()
if err != nil {
log.Println("Error reading message:", err)
return
}
fmt.Printf("Received from server: %s\n", message)
}
}()
....
if err := conn.WriteMessage(websocket.TextMessage, []byte(text)); err != nil {
log.Println("Error writing message:", err)
return
}
}
....
} What to use, depends on which usecase our webapps being designed for. If it’s live stream or chat application, we can use websockets. Whereas for other regular webapp like instagram or Uber kind of design can adopt HTTP2.0 ReST api endpoint.
Design of REST APIs
The following example shows how to design our api endpoints..
1. Defining the request and response structure.
2. Defining api handler.
3. Way to route request to specific endpoint.
4. Defining response or error code.
Exa:
// Defining Structure
// User represents the structure of a user
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Email string `json:"email"`
IsActive bool `json:"is_active"`
}
// Response represents a standard API response
type Response struct {
Status string `json:"status"`
Message string `json:"message"`
Data any `json:"data,omitempty"`
}
// UserService defines the operations for user management
type UserService interface {
CreateUser(user User) User
...
}
// UserServiceImpl is the concrete implementation of UserService
type UserServiceImpl struct {
users []User
nextID int
}
// NewUserServiceImpl creates a new UserServiceImpl
func NewUserServiceImpl() *UserServiceImpl {
return &UserServiceImpl{
users: make([]User, 0),
nextID: 1,
}
}
func (us *UserServiceImpl) CreateUser(user User) User {
user.ID = us.nextID
us.nextID++
us.users = append(us.users, user)
return user
}
// Defining Handler
// createUserHandler handles the creation of a new user
func createUserHandler(us UserService) http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
var user User
if err := json.NewDecoder(r.Body).Decode(&user); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
createdUser := us.CreateUser(user)
jsonResponse(w, Response{Status: "success", Message: "User created", Data: createdUser})
}
}
// Setup Routing
func main() {
userService := NewUserServiceImpl()
http.HandleFunc("/user", createUserHandler(userService))
http.ListenAndServe(":8080", nil)
}Each request associated with a method Type. REST apis with operations supported represesented as CRUD (CREATE, READ, UPDATE, DELETE) that define the method of GET, PUT, POST, PATCH and DELETE.
GET /api/atcfw/v1/account. This operation gets replicated account.
Example
curl -X GET -H "Content-Type: application/json" \
-H "Authorization: Token $API_TOKEN" \ // Auth header discussed later.
https://$API_HOST/api/atcfw/v1/accounts/10
{
"success": {
"status": 200,
"message": "Found Account",
"code": "OK"
},
"results": {
"Id": 10,
"licenses": ["lic1", "lic2"],
"customer_id", "a19626200f27a3152407c61b3ba348ef",
"created_time": "2018–07–26T18:11:32Z", // timestamp added has been discussed later.
"updated_time": "2018–07–26T18:11:37Z"
}
}
The result of this operation will be one of the following:
HTTP 404 result, if the account the request is authorized into is not licensed
for roaming device feature
HTTP 200 followed with JSON body, if the account is authorized for the roaming
device feature
"error": {
"status": 500,
"code": "INTERNAL",
"message": "Internal server error"
},
Authorization error:
"error": {
"status": 403,
"code": "PERMISSION_DENIED",
"message": "ActiveTrust Cloud subscription is required to perform request"
},REST APIs rely on the proper use of HTTP methods to perform operations on resources.
Methods
GET retrieve a representation of a resource. This method should be safe and idempotent.
POST create a new resource or perform an action on an existing resource. This method is not idempotent, as each request may have a different effect.
PUT update an existing resource or create a new resource if it doesn’t exist. This method should be idempotent.
PATCH partially update an existing resource. This method is not idempotent.
DELETE removes a resource. This method should be idempotent. DELETE operation return status 202 means accepted if the request has been successfully queued.
[Note: In case of async operation, application return task id that can be treated for success/failure states. Each request from client to server must contain all the necessary information to understand and complete the request. ]
API Returns
Common HTTP status codes include:
200 OKfor successful requests.201 Createdfor successful resource creation.204 No contentif the request was successful, but there is no response body.400 Bad Requestfor client-side errors.401 Unauthorizedfor authentication errors.403 Forbiddenfor authorization errors.404 Not Foundfor non-existent resources.500 Internal Server Errorfor server-side errors.
Key Features of API
API Versioning
Versioning your API endpoints is crucial for maintaining backward compatibility while allowing you to introduce new features and improvements. There are multiple strategy in api versioning.
URI Versioning: Include the version number in the URI (e.g.,
/v1/users,/v2/users).Query Parameter Versioning: Use a query parameter to specify the version (e.g.,
/users?version=2).Header Versioning: Use a custom HTTP header to specify the version (e.g.,
Accept: application/vnd.company.v2+json).
Media Type Versioning like using custom media types in the Content-Type or Accept headers. Choose the versioning strategy that works best for your use case and stick with it consistently across your API.
//// Versioned Object definition
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Age int `json:"age"`
}
var usersV1 = []User{
{ID: 1, Name: "Alice", Age: 30},
{ID: 2, Name: "Bob", Age: 25},
}
var usersV2 = []User{
{ID: 1, Name: "Alice", Age: 30},
{ID: 2, Name: "Bob", Age: 25},
{ID: 3, Name: "Charlie", Age: 35},
{ID: 4, Name: "David", Age: 20},
}
//// Separate handler
func usersHandlerV1(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(usersV1)
}
func usersHandlerV2(w http.ResponseWriter, r *http.Request) {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(usersV2)
}
//// versioned end point
func main() {
http.HandleFunc("/v1/users", usersHandlerV1)
http.HandleFunc("/v2/users", usersHandlerV2)
fmt.Println("Server is running on port 8080...")
log.Fatal(http.ListenAndServe(":8080", nil))
}Timestamp:
To associate a timestamp with each API request, you can use middleware in your Go HTTP server. Middleware functions are a great way to add common functionality, like logging timestamps, to all your API endpoints without repeating code.
// LoggingMiddleware logs the timestamp of each request.
func LoggingMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Get the current timestamp
timestamp := time.Now().Format(time.RFC3339)
// Log the timestamp and the request method and URL
log.Printf("Timestamp: %s - %s %s", timestamp, r.Method, r.URL.Path)
// Call the next handler
next.ServeHTTP(w, r)
})
}Handling Errors Gracefully
Errors are inevitable in any software system, and it’s essential to handle them gracefully in your API. Provide clear and descriptive error messages, along with appropriate HTTP status codes, to help clients understand and troubleshoot issues more effectively.
A typical structured error response might include:
- HTTP Status Code: Indicates the nature of the error.
- Error Code: A specific code representing the error, useful for programmatic handling.
- Message: A human-readable description of the error.
- Details (optional): Additional information about the error.
//// Error Object
type ErrorResponse struct {
StatusCode int `json:"status_code"`
ErrorCode string `json:"error_code"`
Message string `json:"message"`
Details string `json:"details,omitempty"`
}
//// Error response
func writeErrorResponse(w http.ResponseWriter, statusCode int, errorCode, message, details string) {
w.Header().Set("Content-Type", "application/json")
w.WriteHeader(statusCode)
errorResponse := ErrorResponse{
StatusCode: statusCode,
ErrorCode: errorCode,
Message: message,
Details: details,
}
json.NewEncoder(w).Encode(errorResponse)
}
//// Define Data and handler
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Age int `json:"age"`
}
var users = []User{
{ID: 1, Name: "Alice", Age: 30},
{ID: 2, Name: "Bob", Age: 25},
}
func getUserByID(w http.ResponseWriter, r *http.Request) {
idStr := r.URL.Query().Get("id")
if idStr == "" {
writeErrorResponse(w, http.StatusBadRequest, "MISSING_ID", "The 'id' parameter is required", "")
return
}
id, err := strconv.Atoi(idStr)
if err != nil {
writeErrorResponse(w, http.StatusBadRequest, "INVALID_ID", "The 'id' parameter must be a valid integer", err.Error())
return
}
for _, user := range users {
if user.ID == id {
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(user)
return
}
}
writeErrorResponse(w, http.StatusNotFound, "USER_NOT_FOUND", fmt.Sprintf("User with ID %d not found", id), "")
}Monitor and Log API Activity
Monitoring and logging API activity is crucial for troubleshooting, performance optimization, and security auditing. Implement robust logging mechanisms that capture relevant information, such as request and response payloads, execution times, and error details. Additionally, consider integrating monitoring tools to track metrics like response times, error rates, and resource utilization. All these details can be found in a separate blog of designing of a observability platform.
API Security
Securing an API involves implementing various security measures to protect api from unauthorized access, malicious attack, data breaches, and other vulnerabilities. To make the API secure, our design of api endpoint should adhere to security best practices.
1) Use of Secure Headers.
2) Use Strong Authentication Mechanisms.
3) Implement Authorization.
4) Private session between client and server.
5) Use HTTPS/TLS to ensure encrypted data in transit.
6) Validate Input.
7) Role based Access Control.
The Request header can include the custom header, authorisation header, api key header to make the request/response to be secured.
Custom headers can be an effective way to secure REST APIs by adding an additional layer of security beyond standard authentication methods.
// Define custom header name and expected value
const (
CustomHeaderName = "X-Custom-Header"
CustomHeaderValue = "expected_value"
)
// Middleware to check custom headers
func customHeaderMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Get the custom header from the request
headerValue := r.Header.Get(CustomHeaderName)
// Check if the header is present and has the expected value
if headerValue != CustomHeaderValue {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
// Proceed to the next handler if the header is valid
next.ServeHTTP(w, r)
})
}
//// Defining the secure handler
func secureHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("This is a secure endpoint"))
}
//// Set Up the HTTP Server with Custom Header Middleware
func main() {
mux := http.NewServeMux()
// Add your handlers
mux.Handle("/secure", customHeaderMiddleware(http.HandlerFunc(secureHandler)))
// Start the server
fmt.Println("Server is running on port 8080...")
http.ListenAndServe(":8080", mux)
}# To test the custom header implementation
$ curl http://localhost:8080/secure
HTTP/1.1 403 Forbidden
$ curl -H "X-Custom-Header: expected_value" http://localhost:8080/secure
This is a secure endpointThis approach can be particularly useful for implementing additional security measures, such as API key verification or other custom authentication schemes. By using middleware, you can easily enforce the presence and validity of custom headers in your API.
Similarly authorization header can also be added in the request structure for an API to make a secure HTTP request.
Client has to add the Authorisation header to your HTTP request using req.Header.Set("Authorization", "Bearer <token>") for bearer tokens or req.Header.Set("Authorization", "Basic <base64encoded(username:password)>") for basic authentication. Server has to extract and verify the authorisation header from incoming requests to control access to protected resources.
//// client.go
func main() {
// Define the API endpoint and credentials
apiURL := "http://localhost:8080/protected"
username := "your_username"
password := "your_password"
// Create a new HTTP request
req, err := http.NewRequest("GET", apiURL, nil)
....
// Set the Authorization header with basic auth credentials
auth := username + ":" + password
encodedAuth := base64.StdEncoding.EncodeToString([]byte(auth))
req.Header.Set("Authorization", "Basic "+encodedAuth)
// Send the HTTP request
client := &http.Client{}
resp, err := client.Do(req)
.....
}
//// Server.go
func protectedHandler(w http.ResponseWriter, r *http.Request) {
authHeader := r.Header.Get("Authorization")
if authHeader == "" {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
// Check for bearer token
if strings.HasPrefix(authHeader, "Bearer ") {
token := strings.TrimPrefix(authHeader, "Bearer ")
if token != "your_bearer_token_here" {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
} else if strings.HasPrefix(authHeader, "Basic ") {
encodedAuth := strings.TrimPrefix(authHeader, "Basic ")
auth, err := base64.StdEncoding.DecodeString(encodedAuth)
if err != nil {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
authParts := strings.SplitN(string(auth), ":", 2)
if len(authParts) != 2 || authParts[0] != "your_username" || authParts[1] != "your_password" {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
} else {
http.Error(w, "Unauthorized", http.StatusUnauthorized)
return
}
// If authorized, respond with a message
message := Message{Content: "This is a protected resource"}
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(message)# To test the custom header implementation
$ curl -i http://localhost:8080/secure
HTTP/1.1 401 Unauthorized
$ curl -i -H "Authorization: Bearer my_token" http://localhost:8080/secure
HTTP/1.1 200 OK
This is a secure endpointWith API Keys, simple tokens passed via HTTP headers or query parameters to authenticate requests.
const apiKey = "your_api_key_here"
func apiKeyMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
key := r.Header.Get("X-API-Key")
if key != apiKey {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
next.ServeHTTP(w, r)
})
}
func mainHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello, authenticated user!"))
}
func main() {
http.Handle("/", apiKeyMiddleware(http.HandlerFunc(mainHandler)))
log.Println("Server is running on port 8080...")
log.Fatal(http.ListenAndServe(":8080", nil))
}OAuth2 is an authentication framework that enables applications to obtain limited access to user accounts on an HTTP service. It’s A widely used open standard for access delegation, commonly used for token-based authentication.
var (
googleOauthConfig = &oauth2.Config{
ClientID: "CLIENT_ID",
ClientSecret: "CLIENT_SECRET",
RedirectURL: "http://localhost:8080/callback",
Scopes: []string{"https://www.googleapis.com/auth/userinfo.profile"},
Endpoint: google.Endpoint,
}
oauthStateString = "random"
)
func loginHandler(w http.ResponseWriter, r *http.Request) {
url := googleOauthConfig.AuthCodeURL(oauthStateString, oauth2.AccessTypeOffline)
http.Redirect(w, r, url, http.StatusTemporaryRedirect)
}
func callbackHandler(w http.ResponseWriter, r *http.Request) {
if r.FormValue("state") != oauthStateString {
http.Error(w, "State is not valid", http.StatusBadRequest)
return
}
token, err := googleOauthConfig.Exchange(context.Background(), r.FormValue("code"))
if err != nil {
http.Error(w, "Failed to exchange token", http.StatusInternalServerError)
return
}
client := googleOauthConfig.Client(context.Background(), token)
userInfo, err := client.Get("https://www.googleapis.com/oauth2/v2/userinfo")
if err != nil {
http.Error(w, "Failed to get user info", http.StatusInternalServerError)
return
}
defer userInfo.Body.Close()
data, _ := ioutil.ReadAll(userInfo.Body)
fmt.Fprintf(w, "User Info: %s\n", data)
}
func main() {
http.HandleFunc("/login", loginHandler)
http.HandleFunc("/callback", callbackHandler)
log.Println("Server is running on port 8080...")
log.Fatal(http.ListenAndServe(":8080", nil))
}Token-based authentication Tokens that encode a payload with a signature to ensure data integrity. It Offers advantages like scalability, statelessness, and enhanced security compared to traditional session-based authentication, tokens have become the preferred choice for developers worldwide. Options like JWT (JSON Web Tokens), Paseto (Platform-Agnostic Security Tokens).
JWTs are commonly used to verify user identities and granting access to private resources. Also with JWT you can securely share information between applications. But JWT has some key management issues, like weak keys or improper storage can compromise the entire system. A leaked or compromised keys allow attackers to forge tokens or decrypt sensitive information. JWTs are stateless, meaning the server doesn’t maintain a record of issued tokens. Vulnerabilities in certain JWT libraries and implementations allow for signature verification to be bypassed.
To address these JWT flaws, we have Paseto Platform-Agnostic Security Tokens for secure stateless tokens. You can find more information in token based authentication.
Session Management
Session management ensures secure and scalable api by managing sessions based on user state. There are three methods sessions can be managed.
Cookies that store session identifiers on the client side and use them to maintain state between requests.
Tokens (JWT) use JSON Web Tokens to securely transmit session information.
Server-Side Session store session data on the server, often in memory or a database, and use session identifiers to manage state.
Example: Using JWT (JSON Web Tokens) for session management in a Go API as it’s stateless and can be easily used across different platforms.
//// Generate and validate JWT tokens.
var jwtKey = []byte("my_secret_key")
type Claims struct {
Username string `json:"username"`
jwt.StandardClaims
}
func generateToken(username string) (string, error) {
expirationTime := time.Now().Add(5 * time.Minute)
claims := &Claims{
Username: username,
StandardClaims: jwt.StandardClaims{
ExpiresAt: expirationTime.Unix(),
},
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
tokenString, err := token.SignedString(jwtKey)
if err != nil {
return "", err
}
return tokenString, nil
}
func validateToken(tokenString string) (*Claims, error) {
claims := &Claims{}
token, err := jwt.ParseWithClaims(tokenString, claims, func(token *jwt.Token) (interface{}, error) {
return jwtKey, nil
})
if err != nil {
return nil, err
}
if !token.Valid {
return nil, fmt.Errorf("invalid token")
}
return claims, nil
}
//// Create middleware to handle authentication using the JWT token.
func authMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
tokenString := r.Header.Get("Authorization")
if tokenString == "" {
http.Error(w, "Authorization header required", http.StatusUnauthorized)
return
}
claims, err := validateToken(tokenString)
if err != nil {
http.Error(w, "Invalid token", http.StatusUnauthorized)
return
}
ctx := context.WithValue(r.Context(), "username", claims.Username)
next.ServeHTTP(w, r.WithContext(ctx))
})
}
//// Define Handlers for Login and Protected Routes
func loginHandler(w http.ResponseWriter, r *http.Request) {
username := r.FormValue("username")
password := r.FormValue("password")
// This is just a simple example. In a real application, you should verify the username and password against your user store.
if username != "user" || password != "password" {
http.Error(w, "Invalid credentials", http.StatusUnauthorized)
return
}
token, err := generateToken(username)
if err != nil {
http.Error(w, "Could not generate token", http.StatusInternalServerError)
return
}
w.Header().Set("Authorization", token)
w.Write([]byte("Login successful"))
}
func protectedHandler(w http.ResponseWriter, r *http.Request) {
username := r.Context().Value("username").(string)
w.Write([]byte(fmt.Sprintf("Hello, %s", username)))
}
//// Set up the HTTP server and define routes.
func main() {
r := mux.NewRouter()
r.HandleFunc("/login", loginHandler).Methods("POST")
r.Handle("/protected", authMiddleware(http.HandlerFunc(protectedHandler))).Methods("GET")
fmt.Println("Server is running on port 8080...")
http.ListenAndServe(":8080", r)
}HTTPS and TLS encryption
func main() {
mux := http.NewServeMux()
mux.HandleFunc("/secure-endpoint", secureHandler)
log.Println("Server is running on port 8443...")
log.Fatal(http.ListenAndServeTLS(":8443", "server.crt", "server.key", mux))
}
func secureHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("This is a secure endpoint"))
}Input parameter validation for ensuring the integrity and security of your API. It involves checking that the incoming request parameters (query parameters, URL parameters, and request body) confirm to expected formats and constraints. This helps prevent issues such as invalid data, injection attacks, and other potential vulnerabilities.
//// Define Validation Functions
// ValidateQueryParam checks if the query parameter is valid.
func ValidateQueryParam(param string, pattern string) error {
matched, err := regexp.MatchString(pattern, param)
if err != nil {
return err
}
...
}
// ValidateIntParam checks if the query parameter is a valid integer.
func ValidateIntParam(param string) (int, error) {
return strconv.Atoi(param)
}
//// Create middleware for parameter validation
func ValidateMiddleware(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
// Validate a string query parameter
queryParam := r.URL.Query().Get("param")
if err := ValidateQueryParam(queryParam, `^[a-zA-Z]+$`); err != nil {
http.Error(w, "Invalid query parameter", http.StatusBadRequest)
return
}
// Validate an integer query parameter
intParam := r.URL.Query().Get("id")
if _, err := ValidateIntParam(intParam); err != nil {
http.Error(w, "Invalid integer parameter", http.StatusBadRequest)
return
}
// Call the next handler
next.ServeHTTP(w, r)
})
}API access control using Roles
Access Control Lists (ACLs) are a method of defining and enforcing permissions to control which users or systems can access specific resources within your API. Implementing ACLs helps secure your API by ensuring that only authorized users can perform certain actions.
//// Define ACLs and User Roles
// Define roles
const (
RoleAdmin = "admin"
RoleUser = "user"
)
// ACL map: maps roles to allowed methods and paths
var acl = map[string]map[string][]string{
RoleAdmin: {
"GET": {"/hello", "/admin"},
"POST": {"/hello", "/admin"},
"PUT": {"/admin"},
"DELETE": {"/admin"},
},
RoleUser: {
"GET": {"/hello"},
"POST": {"/hello"},
},
}
//// Create a middleware function to enforce the ACL based on the user's role.
func aclMiddleware(role string) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
method := r.Method
path := r.URL.Path
// Check if the role is allowed to access the path with the method
if !isAllowed(role, method, path) {
http.Error(w, "Forbidden", http.StatusForbidden)
return
}
next.ServeHTTP(w, r)
})
}
}
func isAllowed(role, method, path string) bool {
if methods, ok := acl[role]; ok {
if paths, ok := methods[method]; ok {
for _, p := range paths {
if p == path {
return true
}
}
}
}
return false
}
//// Define Handlers for Your API Endpoints
func helloHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Hello, World!"))
}
func adminHandler(w http.ResponseWriter, r *http.Request) {
w.Write([]byte("Admin Panel"))
}
//// http server with acl rules applied to routes
func main() {
mux := http.NewServeMux()
// Add your handlers
mux.Handle("/hello", aclMiddleware(RoleUser)(http.HandlerFunc(helloHandler)))
mux.Handle("/admin", aclMiddleware(RoleAdmin)(http.HandlerFunc(adminHandler)))
// Start the server
fmt.Println("Server is running on port 8080...")
http.ListenAndServe(":8080", mux)
}So based on what we have learned from all the above security best practices, let’s design a client and server communication private and secure. Making client-server communication private and secure involves implementing several best practices and technologies to protect data in transit and ensure that only authorized clients can access the server.
Example:
Generate CA Key and Certificate
openssl genrsa -out ca.key 2048
openssl req -x509 -new -nodes -key ca.key -subj "/CN=myca" -days 365 -out ca.crt
Generate Server Key and Certificate
openssl genrsa -out server.key 2048
openssl req -new -key server.key -subj "/CN=myserver" -out server.csr
openssl x509 -req -in server.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out server.crt -days 365
Generate Client Key and Certificate
openssl genrsa -out client.key 2048
openssl req -new -key client.key -subj "/CN=myclient" -out client.csr
openssl x509 -req -in client.csr -CA ca.crt -CAkey ca.key -CAcreateserial -out client.crt -days 365// Implement server in go server.go
func helloHandler(w http.ResponseWriter, r *http.Request) {
fmt.Fprintf(w, "Hello, secure world!")
}
func main() {
// Load server's certificate and key
serverCert, err := tls.LoadX509KeyPair("server.crt", "server.key")
if err != nil {
log.Fatalf("server: loadkeys: %s", err)
}
// Load CA certificate
caCert, err := ioutil.ReadFile("ca.crt")
if err != nil {
log.Fatalf("server: read ca cert: %s", err)
}
// Create a certificate pool from the CA
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
// Configure the server's TLS settings
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{serverCert},
ClientCAs: caCertPool,
ClientAuth: tls.RequireAndVerifyClientCert,
}
server := &http.Server{
Addr: ":8443",
TLSConfig: tlsConfig,
}
http.HandleFunc("/hello", helloHandler)
log.Println("Starting secure server on :8443")
log.Fatal(server.ListenAndServeTLS("", ""))
}
// Implement client in go client.go
func main() {
// Load client's certificate and key
clientCert, err := tls.LoadX509KeyPair("client.crt", "client.key")
if err != nil {
log.Fatalf("client: loadkeys: %s", err)
}
// Load CA certificate
caCert, err := ioutil.ReadFile("ca.crt")
if err != nil {
log.Fatalf("client: read ca cert: %s", err)
}
// Create a certificate pool from the CA
caCertPool := x509.NewCertPool()
caCertPool.AppendCertsFromPEM(caCert)
// Configure the client's TLS settings
tlsConfig := &tls.Config{
Certificates: []tls.Certificate{clientCert},
RootCAs: caCertPool,
}
transport := &http.Transport{TLSClientConfig: tlsConfig}
client := &http.Client{Transport: transport}
resp, err := client.Get("https://localhost:8443/hello")
if err != nil {
log.Fatalf("client: get: %s", err)
}
defer resp.Body.Close()
body, err := ioutil.ReadAll(resp.Body)
if err != nil {
log.Fatalf("client: readall: %s", err)
}
fmt.Printf("%s\n", body)
}
API Vulnerability
These are some of the list of vulnerability listed by Open Web Application Security Project (OWASP).
Broken Object level authorization
When the autherization dictated by user object accessibility don’t exist or improperly applied, then this allows the users to manipulate API request parameters to which they are not authorised. This is a security vulnerability where an attacker gain access to unauthorized objects, typically by manipulating the input to reference objects without proper authorization checks.
Implement the autherization mechanism to ensure that particular action by a user is autherize to perform any kind of access to the record and update.
Ensure every API end points are secured against any object possible. Consider an API to fetch the user’s profile:
http://example.com/profile/{userId}Our web application uses an user1 to store user profiles of user2 in std::map which user1 is not autherized for.
struct UserProfile {
int userId;
std::string name;
std::string email;
};
std::map<int, UserProfile> userProfiles = {
{1, {1, "Alice", "alice@example.com"}},
{2, {2, "Bob", "bob@example.com"}},
{3, {3, "Charlie", "charlie@example.com"}}
};
//// Function to get the user profile:
UserProfile* getUserProfile(int userId) {
if (userProfiles.find(userId) != userProfiles.end()) {
return &userProfiles[userId];
}
return nullptr;
}
//// Handler function which handles get userprofile request.
void handleProfileRequest(int requestingUserId, int requestedUserId) {
UserProfile* profile = getUserProfile(requestedUserId);
if (profile != nullptr) {
std::cout << "User ID: " << profile->userId << std::endl;
std::cout << "Name: " << profile->name << std::endl;
std::cout << "Email: " << profile->email << std::endl;
} else {
std::cout << "Profile not found!" << std::endl;
}
}The handleProfileRequest function directly retrieves the user profile without checking if requestingUserId is authorized to access requestedUserId. It can be fixed in two ways.
/* Have a check on whether requesting userid has access
to requested UserId profile.
*/
void handleProfileRequest(int requestingUserId, int requestedUserId) {
if (requestingUserId != requestedUserId) {
std::cout << "Unauthorized access!" << std::endl;
return;
}
UserProfile* profile = getUserProfile(requestedUserId);
if (profile != nullptr) {
std::cout << "User ID: " << profile->userId << std::endl;
std::cout << "Name: " << profile->name << std::endl;
std::cout << "Email: " << profile->email << std::endl;
} else {
std::cout << "Profile not found!" << std::endl;
}
}
// By defining Role based access control policy
std::map<int, std::string> userRoles = {
{1, "user"},
{2, "admin"},
{3, "user"}
};
void handleProfileRequest(int requestingUserId, int requestedUserId) {
std::string role = userRoles[requestingUserId];
if (role != "admin" && requestingUserId != requestedUserId) {
std::cout << "Unauthorized access!" << std::endl;
return;
}
UserProfile* profile = getUserProfile(requestedUserId);
if (profile != nullptr) {
std::cout << "User ID: " << profile->userId << std::endl;
std::cout << "Name: " << profile->name << std::endl;
std::cout << "Email: " << profile->email << std::endl;
} else {
std::cout << "Profile not found!" << std::endl;
}
}
int main() {
int requestingUserId = 2; // Admin
int requestedUserId = 1; // Target
handleProfileRequest(requestingUserId, requestedUserId);
return 0;
}Broken Authentication
In which the API doesn’t properly authenticate the user and the application unable to find whether the user is legitimate or not. In this way the attacker can get the partial or complete control of the request.
This can happen because of
1. Application’s authentication mechanisms are improperly implemented.
2. Lack of rate limiting or Brute force protection.
3. Insecure token generation.
4. Doesn’t validate JWT expiry.
5. Use text based or non encrypted password or weak encryption key.
The mitigation for this is to
- Have secure session management.
- Strong authentication mechanism.
The application has a login function that validates user credentials against a hard-coded list of users.
std::map<std::string, std::string> users = {
{"alice", "password123"},
{"bob", "qwerty"},
{"charlie", "letmein"}
};
bool login(const std::string& username, const std::string& password) {
if (users.find(username) != users.end() && users[username] == password) {
std::cout << "Login successful!" << std::endl;
return true;
} else {
std::cout << "Invalid username or password." << std::endl;
return false;
}DDoS Attacks
Implement rate limiting to prevent abuse and ensure fair usage of your API. This can be done using API gateways or middleware. Rate limiting helps protect your API from excessive use and ensures resources are available for all users.
Set limits based on factors like user roles, subscription tiers, or API endpoints to control access and distribute server load evenly. Have the rate limiter restrict the number of requests a client can make to the API within a specific time window.
Security Compliance
Ensuring compliance for API endpoints involves adhering to relevant laws, regulations, and standards that govern data privacy, security, and industry-specific requirements.
Some data privacy regulation body:
- GDPR (General Data Protection Regulation) for EU data.
- CCPA (California Consumer Privacy Act) for California residents.
- HIPAA (Health Insurance Portability and Accountability Act) for healthcare data in the U.S.
- PCI DSS (Payment Card Industry Data Security Standard) for handling payment card information.
To compliant with security standard we have to adhere to security best practices: Encryption with TLS, Access Control, Input Validation, Rate Limiting, Logging and Monitoring.
Optimization
Filtering and Sorting
When clients need to filter or sort resources, it’s recommended to use query parameters in the URL. For example, /users?name=John&age=30 could be used to filter users by name and age. Similarly, /books?sort=title could be used to sort books by their titles.
Filtering and sorting data at an API endpoint is a common requirement for providing clients with flexible and efficient data access. This can be done by accepting query parameters that specify the filter and sort criteria, and then applying these criteria to the data before returning the response.
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Age int `json:"age"`
}
var users = []User{
{ID: 1, Name: "Alice", Age: 30},
{ID: 2, Name: "Bob", Age: 25},
{ID: 3, Name: "Charlie", Age: 35},
{ID: 4, Name: "David", Age: 20},
}
//// Implementing filtering and shorting
func filterAndSortUsers(users []User, nameFilter string, minAge, maxAge int, sortBy string, sortOrder string) []User {
// Filter users
filteredUsers := []User{}
for _, user := range users {
if (nameFilter == "" || strings.Contains(strings.ToLower(user.Name), strings.ToLower(nameFilter))) &&
(minAge == -1 || user.Age >= minAge) &&
(maxAge == -1 || user.Age <= maxAge) {
filteredUsers = append(filteredUsers, user)
}
}
// Sort users
if sortBy == "name" {
sort.Slice(filteredUsers, func(i, j int) bool {
if sortOrder == "desc" {
return filteredUsers[i].Name > filteredUsers[j].Name
}
return filteredUsers[i].Name < filteredUsers[j].Name
})
....
}
}
return filteredUsers
}
//// Create API handler
func usersHandler(w http.ResponseWriter, r *http.Request) {
// Get query parameters
nameFilter := r.URL.Query().Get("name")
minAgeStr := r.URL.Query().Get("min_age")
maxAgeStr := r.URL.Query().Get("max_age")
sortBy := r.URL.Query().Get("sort_by")
sortOrder := r.URL.Query().Get("sort_order")
......
// Filter and sort users
filteredUsers := filterAndSortUsers(users, nameFilter, minAge, maxAge, sortBy, sortOrder)
// Respond with the filtered and sorted users
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(filteredUsers)
}
//// server code
func main() {
http.HandleFunc("/users", usersHandler)
fmt.Println("Server is running on port 8080...")
log.Fatal(http.ListenAndServe(":8080", nil))
} # Filtering can be tested using
# Filtering: GET /posts?author=JohnDoe
# Sorting: GET /posts?sort=created_at&order=desc
# Filter by name containing "a" and sort by age in descending order:
$ curl "http://localhost:8080/users?name=a&sort_by=age&sort_order=desc"
# Filter by age between 20 and 30 and sort by name in ascending order:
curl "http://localhost:8080/users? min_age=20&max_age=30&sort_by=name"Pagination
For endpoints returning large datasets, implement pagination to manage the load and improve performance. Implementing pagination in an API allows you to return large datasets in smaller, more manageable chunks, which can significantly improve performance and user experience.
Example:
func paginateUsers(users []User, page, pageSize int) []User {
start := (page - 1) * pageSize
end := start + pageSize
if start >= len(users) {
return []User{}
}
if end > len(users) {
end = len(users)
}
return users[start:end]
}
To process the specific query, page query need to be defined in apihandler.
func usersHandler(w http.ResponseWriter, r *http.Request) {
pageStr := r.URL.Query().Get("page")
pageSizeStr := r.URL.Query().Get("page_size")
page, pageSize := 1, 10 // default values
if pageStr != "" {
page, _ = strconv.Atoi(pageStr)
}
if pageSizeStr != "" {
pageSize, _ = strconv.Atoi(pageSizeStr)
}
// Paginate users
paginatedUsers := paginateUsers(filteredUsers, page, pageSize)
// Respond with the paginated users
w.Header().Set("Content-Type", "application/json")
json.NewEncoder(w).Encode(paginatedUsers)
}# Pagination can be tested using query
/posts?page=2&limit=10Support Caching
Caching can significantly improve the performance and scalability of your API. Implement caching strategies that align with your application’s requirements, such as client-side caching, server-side caching, or a combination of both.
Note: GET automatically cached. PUT and DELETE not cached. Cache control headers in request.
HTTP caching header .
Caching behavior can be managed with directives like max-age, no-cache,
no-store, must-revalidate.Cache Variables
Expiry specifies a point in time when the response is considered stale.
ETag is the unique identifier for a version of the resource, used for conditional requests.
Last-Modified for the last time the resource was modified, used for conditional requests.
Example:
Create an in-memory data store and handlers serve the data with caching header.
type User struct {
ID int `json:"id"`
Name string `json:"name"`
Age int `json:"age"`
}
var users = []User{
{ID: 1, Name: "Alice", Age: 30},
{ID: 2, Name: "Bob", Age: 25},
{ID: 3, Name: "Charlie", Age: 35},
{ID: 4, Name: "David", Age: 20},
}
var lastModified = time.Now().UTC().Format(http.TimeFormat)
var etag = `"12345"`
func usersHandler(w http.ResponseWriter, r *http.Request) {
// Handle conditional requests
ifNoneMatch := r.Header.Get("If-None-Match")
ifModifiedSince := r.Header.Get("If-Modified-Since")
if ifNoneMatch == etag || ifModifiedSince == lastModified {
w.WriteHeader(http.StatusNotModified)
return
}
// Set caching headers
w.Header().Set("Cache-Control", "max-age=60") // Cache for 60 seconds
w.Header().Set("ETag", etag)
w.Header().Set("Last-Modified", lastModified)
w.Header().Set("Content-Type", "application/json")
// Respond with the users
json.NewEncoder(w).Encode(users)
}# Commands To test
# Initial request
$ curl -i http://localhost:8080/users
# The server checks the If-None-Match header against the current ETag.
# If they match, the server responds with 304 Not Modified.
$ curl -i -H "If-None-Match: \"12345\"" http://localhost:8080/users
# The server checks the If-Modified-Since header against the current
# Last-Modified timestamp. If they match, the server responds with 304 Not
# Modified.
$ curl -i -H "If-Modified-Since: $(date -u -R)" http://localhost:8080/usersAdding APIs to api Gateway
For any large distributed computing architecture we can have multiple instance of api gateway used as active-active or active-passive mode depends on how many RPS or QPS we expected to handle. We have many different type of api gateway we can use in our system. The most popular one are on-prem gateway like NGINX, or kong gateway or cloud gateway provided by all major third party cloud providers like AWS, GCP or Azure. Kubernetes clusters has it’s own API gateway. The procedure for adding a new API to AWS api gateway:
Create a New API: Define a new API in the API Gateway console or via CLI.
Create a new Rest APIaws apigateway create-rest-api --name 'MyNewAPI' --description 'This is my new
API'Generating api id: API_ID=$(aws apigateway get-rest-apis --query 'items[?name==`MyNewAPI`].id'
--output text)
Define Resources and Methods: Each api has to be associated with a resource. So need to define the resources (endpoints) and HTTP methods (GET, POST, PUT, DELETE) for the new API.
Defining new resource aws apigateway create-resource --rest-api-id $API_ID --parent-id $(aws
apigateway get-resources --rest-api-id $API_ID --query 'items[0].id'
--output text) --path-part 'myresource'RESOURCE_ID=$(aws apigateway get-resources --rest-api-id $API_ID --query
'items[?path==`/myresource`].id' --output text)Creating methodaws apigateway put-method --rest-api-id $API_ID --resource-id $RESOURCE_ID
--http-method GET --authorization-type "NONE"
Integrate with Backend: Set up integration with backend services (e.g., Lambda functions, HTTP endpoints, or other AWS services).
Integrate with lambda function: aws apigateway put-integration --rest-api-id $API_ID --resource-id
$RESOURCE_ID --http-method GET --type AWS_PROXY --integration-http-method
POST --uri arn:aws:apigateway:YOUR_REGION:lambda:path/2015-03-31/functions
/arn:aws:lambda:YOUR_REGION:YOUR_ACCOUNT_ID:function:YOUR_FUNCTION_NAME/
invocationsSet Up Request and Response Mappings: Configure request and response mappings to transform incoming requests and outgoing responses as needed.
Refer: Defining API endpoint
Deploy the API: Deploy the API to a stage (e.g., development, staging, production).
aws apigateway create-deployment --rest-api-id $API_ID --stage-name devTest the API: Test the new API to ensure it behaves as expected.