Read a Network Capture Files might be a common task to extract information, samples or other network traffic information. When doings this in Python, Scapy is the go to tool to get this things out.
Writing a tool to extract information from pcaps, I bumped with a situation where processing time was slow, and memory consumption was going up. Looking for different alternatives using scapy out what was happening lead to a good learning on tool for profiling python.
Reading a pcap file with Scapy, is commonly done by usingrdpcap(). This function reads the whole file and load it up in memory, depending on the size of the file you’re trying to read can take quite some memory.
A small sample script with this function was taking a considerable amount of time so I’ve decided to look for another alternative. I’ve started testing PcapReader(). Iterates over the pcap to read every packet. It is better for memory consumption but would have its quirks, so I’ve decided to test it out
Two versions of a Script to do the same thing. Open up a pcap, read all the packets and extract the URLs from it into a text file. Behaviour is the same, but execution time and memory consumption differ a bit.
