Netflow Data Visualization Framework Using Custom Queries

Due to the complexity and size of networks, it takes a lot of effort to make sure everything is working properly, yet system administrators have to provide a reliable and safe service to maintain the user’s trust. In previous work we have created a  parser that goes through big NetFlow data sets and stores the summarized data in a MySQL database. From there we apply visualization techniques using open source toolkits to analyse and visualize the data in an user friendly and efficient way.

In this work we implemented a custom query system, that generates information dynamically to give users more control over what data and period of data is displayed. It consists of a web interface that  is generated dynamically according to the data available in the database. The interface presents the users with choices like network device to display, specific ports, direct connections from one network device to another, and a time interval. A program then translates the user’s choices into database queries that are processed to generate the final visualization. We have also created an interface that allows authorized users to configure the database options for the parsing, and the visualization process.

This work results in a network monitoring system  that allows system administrators to detect attacks and network anomalies, among other means to efficiently analyze complex computer networks.