As the APIs grow in number and complexity, documentation becomes a must-have. With time, changes in API parameters, response entities, and optional vs mandatory keys in both parameters and response entities are inevitable. This problem becomes more complex when API versioning comes into the picture.
If the entire documentation is manual, maintenance becomes difficult. In fast-paced development scenarios, usually documentation gets neglected. This leads to communication gaps between the API developers and the API consumers.
The root cause for the documentation maintenance problem is that the documentation is done manually. It’s not automated using the API code.
To solve the problem, we explored multiple tools. We decided to use Swagger UI to document our APIs. Swagger UI helps in rendering the API specification, given in openapi.json, as interactive API documentation.
Two Approaches
To solve the problem of maintenance, we propose the following two approaches:
- If openapi.json is auto-generated using the config files which are already in use inside the code for the request and response validations. In this approach, the documentation creation is not a manual process and is directly dependent on the API validation code. So, documentation is as correct as the API code itself.
- Another approach can be to use the openapi.json file to implement the validation of requests and responses of API. In this approach, the validations are driven directly from the documentation and thus documentation is easy to maintain.
Both the approaches are equally DRY (Don’t repeat yourself). Approach 1 is better for cases in which the API is already implemented and we are trying to document it. Approach 2 is better for cases where API is still not implemented and there is a scope to choose the direction of implementation.
We took approach 1 for the reason that our API validation layer was already implemented and we did not want to make major changes to it.
Expectations / Requirements
Route Level
For each route, the following information is expected to be present in the documentation:
- Path of the route
- Whether the route is GET / POST / PUT etc.
- The main objective of each route should be mentioned in the documentation.
- Grouping of routes into endpoints. For example, all routes which manage CRUD operations on users should be grouped.
Parameter
For each route, we need to document the parameters. The following information is expected for each of these parameters:
- Name of the parameter.
- Location of the parameter: Parameters can come in headers, request body (in case of POST API calls), query string (in case of GET API calls), as a part of the path. This needs to be specified.
- Is the parameter mandatory OR optional?
- Type of the parameter - string, integer, etc.
Response
API response consists of response entities. A single response entity can come in different APIs. Response entities are well defined with standard attributes. So response documentation can be broken down into 2 parts:
- Listing what all response entities come in response for each route.
- What all standard attributes are present in each response entity. This documentation is not route-specific.
So we need to document the response entity attribute definitions in one place for each response entity. After this, for each route, we can list down the response entities which are returned and link them to the corresponding response entity definitions.
Response Entity
We need to document the following information for each attribute of a response entity:
- Name of the attribute
- Type of the attribute
- Some description to explain what that attribute means.
- Is that attribute mandatory or not? This is important because the API consumers should know whether that attribute will always come or sometimes it may be missing.
Implementation details
Route Level
Using the approach suggested by dougwilson in Express issue #3308 - List all routes in express app, we traversed all the routes from a root route index file. This is necessary to avoid missing out on documentation of any route. We traversed separately for each API version.
For each route, which we got in the above traversal, we created an entry in the route specification file. An example of the route specification file is given below. Note that creating this file was an additional effort made for achieving the documentation but to avoid manual mistakes, we validated that all the traversed routes should have an entry in this route specification file.
const webRouteSpec = {
'GET /api/web/users': {
apiName: apiNameConstants.getAllUsers,
summary: 'Get all users',
// description: Optional extended description in CommonMark or HTML.
tag: 'user CRUD'
},
'POST /api/web/signup': {
apiName: apiNameConstants.emailSignUp,
summary: 'user sign up',
tag: 'user CRUD'
}
};
Brief description of the route-specific fields in this file is as follows:
- apiName will be used later to fetch the input param validation config and response validation config later.
- summary is one liner objective of the route.
- description is a detailed optional description for the route.
- tag will be used to group routes. For example, all the routes related to user CRUD operations will be grouped. Openapi specification allows the use of multiple tags for the same route. We avoided it as we wanted that a route should be present in a single group.
In openapi.json, we populated the paths section, using the information from the route specification file. More on the paths section can be found here.
Parameter
We already had a parameter signature file in the code that had a list of all the mandatory and optional parameters for each API and what all validations will be run on the values of these parameters. An example snippet from this file is given below. This file contains key-value pairs. The Key is the apiName and the value is the list of parameter configs.
[apiNameConstants.emailSignUp]: {
mandatory: [
{
parameter: 'email',
validatorMethods: [{ validateString: null }, { isValidEmail: null }],
type: 'string',
description: 'elaborated description if needed.'
},
{
parameter: 'password',
validatorMethods: [{ validateString: null }],
type: 'string'
},
{
parameter: 'api_source',
validatorMethods: [{ validateString: null }],
type: 'string'
}
],
optional: []
}
The keys parameter and validatorMethods were already present and were being used in the validation layer. We introduced 2 new keys - type and description for documentation purposes.
- type is the data type of the value of the parameter. Openapi supports these data types.
- description is an optional description for the parameter.
In openapi.json, we populated the parameters section in the paths section, using the parameter signature file. Detailed documentation on the parameters section is given here.
Response and Response Entity
To list out all the response entities which are sent in an API response, we re-used the response signature file, which was already being used for response formatting and validation purposes in the API code. A small snippet from the response signature file is given below. This file contains key-value pairs. The Key is the apiName and the value is the response signature. There was no addition made to this file for documentation purposes.
[apiNameConstants.getAllUsers]: {
resultType: responseEntityKey.userIds,
resultTypeLookup: responseEntityKey.users,
entityKindToResponseKeyMap: {
[entityTypeConstants.userIds]: responseEntityKey.userIds,
[entityTypeConstants.usersMap]: responseEntityKey.users,
[entityTypeConstants.getAllUsersListMeta]: responseEntityKey.meta
}
}
Each response entity is getting created in a formatter class of its own which has the responsibility of sending the agreed upon attributes in the response entity after validation. We added a schema static method to this class. Following snippet shows an example schema method.
static schema() {
return {
type: 'object',
properties: {
id: {
type: 'integer',
example: 123,
description: 'BE notes: this is the id of users table'
},
email: {
type: 'string',
example: 'david@example.com'
},
name: {
type: 'string',
example: 'David'
},
status: {
type: 'string',
example: 'ACTIVE'
},
uts: {
type: 'integer',
example: 1651666861
}
},
required: ['id', 'email', 'status', 'uts']
};
}
We started consuming required array of attributes in schema to validate the mandatory response entity attributes. This ensured that there is no repetition of the mandatory list of attributes (adhering to the DRY philosophy).
In openapi.json, under components, we can mention schema of reusable components. A more detailed documentation of the component section can be found here. We just had to read from the static schema() method and populate the components section of openapi.json. Further, in the responses section under paths section, we linked the components using $ref. More on this can be found here. This was done in the openapi JSON generator script.
Openapi JSON Generator
We wrote Openapi JSON generator script to stitch the information present in route specification file, parameter signature file, response signature file and the schemas present in formatter classes. After any change in API, we will run this script to refresh the openapi.json file. For each version of API a different openapi.json file will be created and will be used to render the UI for the respective documentation.
Following snippet was added to app.js to route the documentation routes.
const swaggerJSDoc = require('swagger-jsdoc'),
const swaggerUi = require('swagger-ui-express'),
// API Docs for mobile v1 APIs
const swaggerSpecV1 = swaggerJSDoc(require(rootPrefix + '/config/apiParams/v1/openapi.json'));
const swaggerHtmlV1 = swaggerUi.generateHTML(swaggerSpecV1);
app.use('/api-docs/v1', swaggerUi.serveFiles(swaggerSpecV1));
app.get('/api-docs/v1', function(req, res) {
return res.send(swaggerHtmlV1);
});
// API Docs for web APIs
const swaggerSpecWeb = swaggerJSDoc(require(rootPrefix + '/config/apiParams/web/openapi.json'));
const swaggerHtmlWeb = swaggerUi.generateHTML(swaggerSpecWeb);
app.use('/api-docs/web', swaggerUi.serveFiles(swaggerSpecWeb));
app.get('/api-docs/web', function(req, res) {
return res.send(swaggerHtmlWeb);
});
Conclusion
We achieved all the expectations / requirements which we laid for ourselves. Also, for documentation we re-used the config files which were used for various validations and thus avoiding maintenance issues.
