COMPUTER NETWORKS, cilt.231, 2023 (SCI-Expanded)
SYN flood attack is one of the common ways in which attackers take the advantage of TCP's three-way handshake connection establishment to overwhelm target systems. With the emergence of programmable networks, many promising security functions have been implemented at the network layer. SYN authentication and cookie-based SYN proxy are two significant approaches recommended on programmable switches against SYN flood attacks. However, while the implementation of a cookie-based SYN proxy causes additional delays and packet drops, the SYN authentication approach increases the number of packets required to establish a TCP connection. In this study, two novel functions are implemented on programmable switches using the P4 language, and a new security solution against SYN flood attacks is proposed by combining both functions. The first function is the high-accuracy updatable Bloom filters implemented to track the state of network flows for TCP connection establishment. As the proposed data structure uses a salted input, it is more resistant to target-set coverage attacks. The second is the connection pooling function on programmable switches for requests to the backend servers. In this regard, the TCP three-way handshake is offloaded from the target systems to network switches. Upon the verification of a connection request, dynamic resource allocation is carried out from the connection pool on the P4 switch, enabling the client to connect to the server seamlessly and transparently without the need for additional packets. We implement these functions and demonstrate their feasibility as an effective defense against SYN flood.