Benchmark result

ESPFirewall/lib/Firewall/docs/api/api.tex

@@ -8,22 +8,27 @@ Managing the firewall rules can be done over a REST API\footnote{\href{https://w
 \begin{lstlisting}
 [
     {
-        "endpoint": "https://10.93.0.224:8080/api/firewall/rules",
+        "endpoint": "http://10.93.0.246:8080/api/firewall/rules",
         "description": "Get all Firewall Rules",
         "method": "GET"
     },
     {
-        "endpoint": "https://10.93.0.224:8080/api/firewall/rules/<key>",
+        "endpoint": "http://10.93.0.246:8080/api/firewall/rules/<key>",
         "description": "Get Firewall Rule by key",
         "method": "GET"
     },
     {
-        "endpoint": "https://10.93.0.224:8080/api/firewall/rules",
+        "endpoint": "http://10.93.0.246:8080/api/firewall/rules",
         "description": "Create Firewall Rule",
         "method": "POST"
     },
     {
-        "endpoint": "https://10.93.0.224:8080/api/firewall/rules/<key>",
+        "endpoint": "http://10.93.0.246:8080/api/firewall/rules/<key>",
+        "description": "Update Firewall Rule by key",
+        "method": "PUT"
+    },
+    {
+        "endpoint": "http://10.93.0.246:8080/api/firewall/rules/<key>",
         "description": "Delete Firewall Rule by key",
         "method": "DELETE"
     }

ESPFirewall/lib/Firewall/docs/firewall/firewall.tex

@@ -106,10 +106,10 @@ Following output can be registered.
     I (x) HOOK:  17 10 93 0 211
 \end{verbatim}
 
-Looking at the printed protocols this means evidentially:
-$$ 1 \equiv "ICMP" $$
-$$ 6 \equiv "TCP" $$
-$$ 17 \equiv "UDP" $$
+Looking at the printed protocols this means evidentially how protocols are identified in LwIP:
+$$ "ICMP" \equiv 1 $$
+$$ "TCP" \equiv 6 $$
+$$ "UDP" \equiv 17 $$
 
 \subsubsection{Arduino as an ESP-IDF component}
 
@@ -185,4 +185,81 @@ int lwip_hook_ip4_input(struct pbuf *pbuf, struct netif *input_netif);
 
 After specifying the prototype the function can be placed in the main.cpp file to be compiled and run.
 
+\newpage
+
 \subsection{Benchmark}
+
+To test the performance of the firewall with rules and without rules, as well as many rules, the time is stopped at the beginning and the end of the hook.
+
+\subsubsection{Code}
+
+\lstset{style=c++}
+\begin{lstlisting}
+void print_time_taken(struct timeval start, 
+                      fw::firewall_target_t target)
+{
+    struct timeval stop;
+    gettimeofday(&stop, NULL);
+    u32_t time_taken = (stop.tv_sec - start.tv_sec) * 1000000 + 
+        stop.tv_usec - start.tv_usec;
+    Serial.println(time_taken);
+}
+
+int lwip_hook_ip4_input(struct pbuf *pbuf, struct netif *input_netif)
+{
+    // Firewall is not setup yet
+    if (firewall != NULL)
+    {
+        struct timeval start;
+        gettimeofday(&start, NULL);
+        if (firewall->is_packet_allowed(pbuf))
+        {
+            print_time_taken(start, fw::TARGET_ACCEPT);
+            return 0;
+        }
+        else
+        {
+            print_time_taken(start, fw::TARGET_DROP);
+            pbuf_free(pbuf);
+            return 1;
+        }
+    }
+    return 0;
+}
+\end{lstlisting}
+
+\newpage
+
+\subsubsection{Result}
+
+The results in milliseconds are copied into a spreadsheet to create an chart with the measured processing time of a packet in the hook. The tests were done with 1,5,10 and 15 rules. As the linear trendlines indicate, the amount of rules are heavily responsible for a longer processing time.
+
+\begin{figure}[H]
+    \begin{center} 
+        \includegraphics[width=\textwidth]{chart}
+        \caption{Benchmark graph}
+        \label{fig:Benchmark graph}
+    \end{center}
+\end{figure}
+
+Without any rule, the processing of the included hook takes between 23 to 24 milliseconds. No comparison or preparing of the packet is necessary. 
+
+With a single rule, the processing time already increases rapidly. The amount of time it takes for the packet to be prepared, comparing it to the rules and releasing it, is already between 67 and 99 milliseconds.
+
+\begin{figure}[H]
+\begin{center}
+\begin{tabular}{|l|l|l|l|l|l|}
+    \hline
+    & 0 rule & 1 rule & 5 rules & 10 rules & 15 rules \\
+    \hline
+    \textbf{Average} & 23,81 ms & 74,94 ms & 78,81 ms & 87,07 ms & 94,63 ms \\
+    \textbf{Minimum} & 23 ms & 67 ms & 72 ms & 77 ms & 85 ms \\
+    \textbf{Maximum} & 24 ms & 99 ms & 98 ms & 115 ms & 124 ms \\
+    \hline
+\end{tabular}
+\end{center}
+\caption{Benchmark table}
+\label{fig:Benchmark table}
+\end{figure}
+
+$$ 0\ rule\ \textcolor{red}{\leftarrow 51,13 ms \rightarrow}\ 1\ rule\ \textcolor{teal}{\leftarrow 3,87 ms \rightarrow}\ 5\ rules\ \textcolor{orange}{\leftarrow 8,26 ms \rightarrow}\ 10\ rules\ \textcolor{orange}{\leftarrow 7,56 ms \rightarrow}\  15\ rules $$