PyPCAPKit Enhancement Proposals

[JarryShaw]

As PyPCAPKit reaches its 16k lines of code and 800th commit, I figure it would be a better idea to record the project enhancement proposals here in the discussion thread. The proposals and/or notes will be documented and maintained here.

Pull requests for the existing proposals and any new ideas are highly welcomed and encouraged. Should you have any questions, please leave a note either in this thread or under the Q&A category discussions.

Wish you enjoy PyPCAPKit!!!

[JarryShaw]

More Protocols, More!!!

As you may have noticed, there are some protocol-named files under the NotImplemented folders. These protocols are what I planned to implement but not yet done. Namely, grouped by each TCP/IP layer and ordered by protocol name alphabetically,

• Link Layer: DSL, EAPOL, FDDI, ISDN, PPP
• Internet Layer: ECN, ESP, ICMP, ICMPv6, IGMP, NDP, Shim6
• Transport Layer: DCCP, QUIC, RSVP, SCTP
• Application Layer: BGP, DHCP, DHCPv6, DNS, IMAP, LDAP, MQTT, NNTP, NTP, ONC/RPC, POP, RIP, RTP, SIP, SMTP, SNMP, SSH, Telnet, TLS/SSL, XMPP

Specifically, I have attempted to implement ESP several years ago, and I abandoned the implementation in the NotImplemented folder due to some design flaws within PyPCAPKit at that time. But now, the protocol should be able to implement quite smoothly.

More over, MH (pcapkit.protocols.internet.mh.MH) requires some help to implement all the message data types, you can find more information in the specific file.

Also, for the existing protocols, I am looking for a helping hand to implement the construction features, as defined in the Protocol.make method. You can find some reference from the PCAP's Frame header class. See comment below.

[JarryShaw]

After merging the test/rc/schema branch, all protocols are now by nature designed and implemented with support of both parsing and construction - through the specified Schema classes. Therefore, when creating new protocols, it is only required to create the schema definition first, and then create the Protocol.read and Protocol.make functions accordingly.

[JarryShaw]

PyPCAPKit now has built-in support for PCAP-NG file format starting from version 1.0.0.

[JarryShaw]

After reviewing the source code of Scapy, DPKT, etc., and the protocol specifications, this approach seems to be unrealistic - it is difficult to determine the length of some fields as they may depend on the values of preceding fields. Thus, I decided to discard this proposal.

[JarryShaw]

Logging Integration

As PyPCAPKit now has the pcapkit.utilities.logging.logger in place, I'm expecting to fully extend its functionality in the entire module. Ideas and contributions are welcomed to integrate the logging system into PyPCAPKit.

[JarryShaw]

New Engines

Although PyPCAPKit already has support for some popular PCAP parsing libraries, I'm expecting to extend the list of supported engines furthermore. The candidate engines include:

• pypcap
• pycapfile

Implementation for support of new engines would include adding corresponding handler methods and code blocks into pcapkit.foundation.extraction.Extractor (see support for Scapy, DPKT, and/or PyShark), as well as, the unified auxiliary tools located in pcapkit.toolkit.

[JarryShaw]

Test Cases

PyPCAPKit still does not have a systematic testing suite to be bundled with it. The only test cases I have worked out are those in the /tests folder - mostly functional tests. As PyPCAPKit is growing bigger and bigger, a comprehensive test suite is coming much more of demand for a more reliable development process.