综合编程

Adding an Auditing System into a Rest API

微信扫一扫,分享到朋友圈

Adding an Auditing System into a Rest API
0

One interesting task when developing Rest API service is adding a feature to an existing Rest API where its implementation needs to be duplicated across every Rest resources such as auditing system for all resources. This type of feature if not managed properly will increase overall application complexity and reduce code quality drastically.

This story will show you how to manage and modularize your code using Aspect Oriented Programming (AOP) technique to create a modular code that applied into multiple controllers without modifying the existing code. As an example, we will create a simple auditing system (75 lines) to track user activity and plug it into the existing Rest API created using Plumier.


Image by
Francis Ray from 
Pixabay

Plumier

If you haven’t heard yet, Plumier is a new Node.js framework designed to easily create robust and secure restful API using TypeScript. You can read thefull story behind how it designed and created.

In this story we will not be using specific tools to perform AOP technique, instead, we will use Plumier middleware to do rudimentary Aspect Oriented Programming as most IoC Container libraries do. Just like the IoC Container interceptor, Plumier middleware has interception and metaprogramming capabilities which make it a perfect tool to do that.

As stated everywhere in Plumier documentation, Plumier is not an opinionated framework, meaning there is no opinion as a guide to solving a case in Plumier. So this AOP technique is not the only solution to solve a case in Plumier. You are free to use your solution that match with your requirements, Plumier just provided the building block to do that.

Application Auditing

In this story, we will create a simple auditing system to track user activity on every request in our Restful API. Information will be saved into the database like below

Above data will be used by the administrator or auditor of the application. This data is important for further analysis of user behaviour to get what the user is mostly doing in the system for like/dislike report. This data also can be used as a simple security auditing to get information about a security breach, if a user with a specific role can access private resources etc.

Censorship

Unlike application logging, auditing system requires more specific information that will be used by administrator or auditor for further analysis, for example in the above table is the Data column, which will provide detail information about user requests such as query and body request.

When auditing a process implemented inside middleware it has access to the raw data of the body request, including data that is prohibited to the administrator or auditor to see, such as phone number, email or even user password. Saving this data directly to the database is considered a data violation. Furthermore, we need to hide or censor some part of the data so it not barely saved into the database.

Preparing The Rest API

For the existing Rest API, we will use the example code from Plumier documentation here . The provided example is a simple Rest API managed todo data, created using Plumier and Knex.js for data access. The full source code of the project can be found in this GitHub link .

To be able to save the audit data into the database, it is required to create a new domain and a database migration script.

  1. Create a domain model for the Audit domain at the end of the file src/model/domain.ts Domain is a plain TypeScript class declared with some properties, in the end, it will look like this .
  2. Create a new migration script to add Audit table into the SQL database inside the db/migrations directory, create a new file named 1.1.0.ts and add Knex.js migration script. In the end, it will look like this .

If you want to take a preview on how the final project looks, or you are too lazy to follow the step provided in this story, you can see the project here .

Implementation Without AOP

To get a good understanding of the topic, we will have a quick peek on how auditing system usually implemented without AOP. As an example, we will add the auditing system to the PUT /users/:id resource. I choose this resource because it’s a perfect example for auditing that requires censorship so the code covers the entire auditing system.

Before starting to add the auditing system to the controller lets take a look at the original code for comparison before the auditing system added. Below is the UserController host the resources (some of the controller’s methods removed for clarity).

The code above looks straight forward. It only consists of two processes: hash the password using bcrypt then save the data to the User table. Below is how its look like after the auditing system added.

Compared with the original code, we added a big amount of code to the current logic which makes the overall controller complexity increased. Remember that the auditing system needs to be implemented in other methods handling different HTTP verbs such as GET , POST and DELETE . Here is how its implemented on the delete method.

Based on two implementations above, you might notice some issues occur.

  1. Adding, modifying and bug fixing the auditing logic requires a lot of effort because it’s duplicated in every method.
  2. Testing is required for each method implemented auditing system.
  3. To enable or disable auditing is almost impossible without modifying the controllers.

That’s enough to make overall application complexity drastically increased.

Enter the AOP

Just like other programming tools, AOP is not a tool to solve everything. Based on its description on Wikipedia , AOP best used to modularize cross-cutting code that duplicated everywhere inside an application such as logging, global error handling, caching, etc.

Before choosing AOP to solve a programming problem, we can do some simple analysis to find out whether the new feature will cut across on existing implementations. Let’s review our previous code like pictures below.


The picture above shows the code block for the modify method. Code marked in red is code for the auditing system. Let’s compare with the code for the delete method below.


The picture above shows the auditing system for the delete method. The code block for the auditing system inside delete and modify methods are nearly the same, except that the data saved to the database is different based on the resource data and the censorship process does not occur on the delete method.

Two pictures above clearly showing that the auditing code block is a cross-cutting code that duplicated in some controller’s methods. This simple analysis is a good start to refactor duplication using AOP technique.

Refactor Duplication Into Middleware

Before we walk further into the implementation, we will have a sneak peek to the Plumier middleware for better understanding of the AOP technique. Let’s take a look at the sequence diagram below.


Plumier middleware is a sequence of invocation that runs one to another using chain of execution (named Middleware Pipeline). The invocation can be other middlewares or a controller’s method execution that handle the current request. Like previously mentioned Plumier middleware designed to have interception and metaprogramming capability which can be used to perform simple AOP technique.

Based on our previous analysis we can move the auditing process into Plumier middleware like below.

Above code showing that we move the auditing system that duplicated inside methods into middleware from line 4 through 13.

Line 4 showing that we created an audit object from the invocation context using createAudit function, we will discuss this function later on the next section.

In line 3 we check if the current request context has route information and has current login user information, if it doesn’t match the criteria it will proceed the next execution immediately.

The most important part is in line 6 where we call next.proceed() to execute the next invocation. Calling this method will proceed invocation to the appropriate controller’s method handles the request. So if the request was PUT /users/:id then the call sequence will jump from the middleware to the modify method as pictured below.


The result of the method execution will be returned back to the middleware. It can be used for further processing, but in this example, we just returned it for the previous invocation.

The above middleware if registered in the global scope, will intercept all methods execution during all requests, that’s mean all methods will execute auditing system implicitly.

Metaprogramming

Another important part of the above middleware is creating the audit object created by createAudit function. This process requires more effort because the audit data varies based on the current method executed.

Plumier has metadata information that can be accessed via request context on Invocation object. Request context is a Koa context injected with some metadata information such as.

route
parameters

Based on the information above we can easily program the createAudit function like the snippet below.

The code above shows that we perform some metaprogramming to create the audit object.

  1. resource data extracted from the controller name by removing the Controller word on line 10 and 11.
  2. action data extracted from the current HTTP Method, by translating it into appropriate words defined by AuditActionMap object on line 1.
  3. data extracted from parameters property but we need to do some censorship before it saved to the database.

Censorship function also can be done using metaprogramming. If we look closely, the censorship function is applied based on parameter data type, not to the current method. We can provide another map contains a function to censor the data like below.

The code above shows that we get the data type of the parameter of the current executed method from context.route.action.parameters . Then we loop through the parameter values and censor the data using appropriate censor function from the map defined on line 1. The CensorshipMap can be added later for another data type that requires censorship.

That is the last step we need to create a user activity auditing system. Next, you can register the middleware directly using the use() method of the Plumier application in the startup code or register it using Plumier facility. In this project example, we created a UserActivityFacility to register the middleware to make it consistent with other configuration. You can see how it registered here .

Testing

Appropriate unit testing for middleware can be added into the test directory on the project. Plumier middleware is a stateless class so it is relatively easier to unit test. In this story, we will not create a specific unit test, but we will just execute the current integration tests that were provided from the example by using yarn test from the command line. The execution process should automatically execute the auditing system and added some records to the Audit table like below.


The above picture shows that all data saved properly into the Audit table. The censorship function also works properly on the data column. It’s mean our auditing system work like expected.

That’s it! That’s how you can add a cross-cutting feature into an existing Rest API using AOP and metaprogramming without modifying the existing code. Most importantly, think about this story before you have a plan to solve your case using AOP technique. You need to do some simple analysis if it is able to be solved by AOP and make sure not to overuse it.

Support The Project

As a final word, this story is intended to explain the power of Plumier middleware, there are more features that will be explained in the next story. Currently, Plumier is relatively new in Node.js world. The hardest part of building a framework is building and maintain a solid community. If you think Plumier can help your work and match with your need, kindly help make Plumier a better future by supporting the project on GitHub .

展开阅读全文

EuroPython: EuroPython 2019: Warning - Spoiler alert!

上一篇

一文了解Python常见的序列化操作

下一篇

你也可能喜欢

评论已经被关闭。

插入图片
Adding an Auditing System into a Rest API

长按储存图像,分享给朋友