Lambda Architecture Based Big Data System for Air and Ground Surveillance

4th International Conference on Computer Applications and Information Security, Tunis, Tunus, 18 - 20 Mart 2021

Processing both streaming and batch data at the

same time for big data analytics is very rare, even it is not

possible to implement in real time applications such as air/ground

monitoring and surveillance. A good solution requires for a

combination of batch and speed layers which must be processed

together in a strictly coordinated manner. This paper introduces,

a new type of system for handling of the problem. It also

presents design and implementation processes based on Lambda

Architecture which is a new generation big data technology.

In order to explain the novel system, a brief research on the

Lambda Architecture is summarized. Building and configuring

all the necessary Big Data frameworks for a Lambda Architecture

requires three different layers work in a coordinated manner.

This is very hard task to achieve for basic big data systems. A

new type of hybrid system is proposed with a different concept

having different layers on the same platform. The implementation

results have shown that the proposed system provides successful

solutions to achieve both processing batch and stream data, to

visualize data queries and to reduce code maintenance for a

real time air/ground surveillance and monitoring problem of air

traffic over Ankara, Turkey. It is expected that the new system

proposed and implemented in this article might be used not only

this specific application but also in other real-time and eventually

consistent applications.