Case: Logging service integration platform

Background

The biggest online enterprise offers a wide range of products such as books, music, movies, electronics, toys, and more, selling directly or acting as a middleman between third-party retailers and its customers.

Additionally, the company provides web services such as cloud computing by renting data storage and computing resources over the Internet.

Its significant online presence is reflected in the fact that a notable amount of North American internet traffic in 2012 was associated with the company's data centers.

Business Demand

There is a database to store the Logs, however, there is no well-designed UI for getting the data from ElasticSearch for the clients, and clients cannot dynamically filter by the attributes.

Clients are not able to add, update and remove issues manually.

Clients cannot conduct metrics tracking from ElasticSearch and cut trouble tickets for metrics exceeding the threshold.

Solution Provided

Using React to develop a front-end UI, and using Node.js to develop the back-end, the data storage part has AWS DynamoDB and ElasticSearch.

The client can give the input and the React will pack the user input to the back-end, then the Node.js will send the request to the API-Gateway, and the API-Gateway will send the different queries to the different databases (AWS DynamoDB and Elasticsearch) based on the different request.

If the query was sent to AWS DynamoDB, it will read and update the database and then send the query result back to the back-end, if the query was sent to ElasticSearch, it will simply send the query result to the back-end, and the back-end will send the query response to the front-end, and the UI will render the response.

Also, there will be an AWS CloudWatch to monitor the application, if it detect anomalous behavior in the environment, it will trigger the AWS Lambda, which is a serverless compute service, it will send the trouble tickets to the Trouble Ticket Service API and then send the request directly to the API Gateway, then API gateway send the query to databases to store the information.

IMPACT

Clients are able to view data from Elasticsearch on the UI and have the option to dynamically filter by all supported attributes.
Clients are able to download large data sets in an asynchronous fashion.
Clients are able to track issues related to a "groupingKey", and add, update, and remove issues from the UI.
The UI supports metrics tracking from ElasticSearch data, and cut Trouble Tickets (TTs) for metrics exceeding the threshold.