Skip to content

Add generic codec base class combining shared ASN.1/CBOR codec logic#29

Draft
Copilot wants to merge 44 commits into
masterfrom
copilot/create-generic-codec-class
Draft

Add generic codec base class combining shared ASN.1/CBOR codec logic#29
Copilot wants to merge 44 commits into
masterfrom
copilot/create-generic-codec-class

Conversation

Copy link
Copy Markdown

Copilot AI commented Mar 17, 2026 •
edited
Loading

  • Rebase branch onto latest master (38 commits ahead)
  • Identify shared field logic between ASN1F_field and CBORF_field
  • Add GenericCodecField_element and GenericCodecField[_I, _A] to scapy/libs/codec.py
  • Add GenericCodecOptionalField to scapy/libs/codec.py
  • Update ASN1F_element, ASN1F_field, ASN1F_optional in scapy/asn1fields.py to inherit from the new bases
  • Update CBORF_element, CBORF_field, CBORF_optional in scapy/cbor/cborfields.py to inherit from the new bases
  • Remove now-redundant imports (copy, BasePacket, Generic) from field modules
  • Fix linting issues (E302, E501) introduced by the changes
  • Verify no test regressions (ASN1, CBOR, x509, SNMP test suites)
  • Reintegrate master (second rebase onto latest origin/master)

💬 Send tasks to Copilot coding agent from Slack and Teams to turn conversations into code. Copilot posts an update in your thread when it's finished.

@polybassa
Create generic codec class combining identical parts from ASN1 and CBOR
e30162f 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
Copilot AI changed the title [WIP] Add generic codec class for ASN1 and CBOR Add generic codec base class combining shared ASN.1/CBOR codec logic Mar 17, 2026
Copilot AI requested a review from polybassa March 17, 2026 22:34
@polybassa
Copy link
Copy Markdown
Owner

polybassa commented May 24, 2026

@copilot Reintegrate master and check for more generic codec and generic codec field elements

evverx and others added 18 commits May 25, 2026 18:20
@evverx @polybassa
ci: install cbor2 on Packit (secdev#4948)
c6460fd 
to prevent the newly introduced CBOR tests from failing with:
```
 ###(090)=[failed] Interop: Large unsigned integer

 >>> large_int = 18446744073709551615  # 2^64 - 1
 >>> encoded = cbor2.dumps(large_int)
 NameError: name 'cbor2' is not defined
```
and so on.
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@Ebrix
[MS-RRP] Registry: Fix lpClassOut name request (secdev#4953)
c8e7fc0 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@evverx @polybassa
dns: fix off-by-four in EDNS0 OWNER Option (secdev#4955)
dbffb3e 
The lengths taken from
https://datatracker.ietf.org/doc/html/draft-cheshire-edns0-owner-option-01
are off by four because they include the length of the two-byte EDNS0
option code and the length of the two-byte EDNS0 option length but
according to https://datatracker.ietf.org/doc/html/rfc6891#section-6.1.2
the length should include the size of data only so all the lengths
should be decreased by 4 to parse the OWNER option containing the wakeup
MAC correctly.

Without this patch the wakeup MAC gets cut off:
```
>>> EDNS0OWN(b'\x00\x04\x00\x0e\x00\x9b\x11"3DUffUD3"\x11')
<EDNS0OWN  optcode=Owner optlen=14 v=0 s=155 primary_mac=11:22:33:44:55:66 |<Padding  load=b'fUD3"\x11' |>>
```
With this patch it's included as expected
```
<EDNS0OWN  optcode=Owner optlen=14 v=0 s=155 primary_mac=11:22:33:44:55:66 wakeup_mac=66:55:44:33:22:11 |>
```
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@evverx @polybassa
tests: patch localtime in the "cert remainingdays" test (secdev#4956)
08da307 
to make sure the code where the invalid timezone is passed to exercise
the localtime code path doesn't fail as time goes by.

The other test was added to exercise the code path where the second
format is passed.
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@sacca97 @polybassa
More Bluetooth 5.0+ packets (secdev#4905)
25ee498 
* add some packets

* some formatting

* consistent address variables naming in layers/bluetooth.py

* minor fixes

* more fixing names

* Fix HCI_Cmd_LE_Set_Extended_Scan_Parameters

* more fixes

* more fixes

* more fixes

* fix formatting

* more tests

* added deprecated_fields for backward compatibility

* fix flake 8 errors

---------

Co-authored-by: Tommaso Sacchetti <sacca@Tommasos-MacBook-Air.local>
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@guedou @polybassa
Pickle - reconstructs the packet from field values, payload and metad…
3d3afd4 
…ata (secdev#4919)

* Pickle - reconstructs the packet from field valies, payload and metadata

* Pickle - support extra slots
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@polybassa
Add CBOR fields (cborfields.py, cborpacket.py) and tests (secdev#4958)
4f297d5 
* Add CBOR fields (cborfields.py, cborpacket.py) and tests

* Remove unused CBOR fields and codecs from cborfields.py

---------

Co-authored-by: Nils Weiss <nils.weiss@dissecto.com>
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@T3pp31 @polybassa
bfd: fix ConditionalField condition for optional_auth always being Tr…
a133af3 
…ue (secdev#4937) (secdev#4965)

The condition `pkt.flags.names[2] == "A"` always evaluated to True because
FlagValue.names returns the flag definition string "MDACFP", not the set of
currently active flags. Replace with `pkt.flags.A` to properly check the
Authentication Present bit.
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
NTLM: bad doc (secdev#4966)
569ee59 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
Users should disclose the use of AI (secdev#4968)
ccaaec6 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
Improve CMS/Kerberos support of other hashes (secdev#4974)
211f600 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
Add WinSSP SSP for implicit authentication (secdev#4975)
2987d3e 
* Add WinSSPI SSP for Windows

* Add WinSSP test
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
Follow semver: change dev version -> post (secdev#4976)
fd754c6 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
Set sys.stdout to utf-8 (secdev#4977)
11c4dbb 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
WinSSP: support Encryption, fix DCE/RPC SMB IO (secdev#4978)
0cd256b 
* WinSSP: support Encryption, fix DCE/RPC SMB IO

* Add use_winssp boolean

* Show output of test when it timeout

* Flake8 for ldaphero

* Fix doc

* Add debug log for HTTP_Server

* WinSSP: support channel bindings
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
Minor fixes (secdev#4980)
2458b9f 
* Fix when last_traceback is None

* Missing docstring for ldaphero
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
Revert last commit... (secdev#4981)
f503225 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@polybassa
Introduce single-layer and compatibility modes for UDS, KWP, OBD, and…
91d2efb 
… GMLAN protocols; add documentation and tests. (secdev#4962)

Co-authored-by: Nils Weiss <nils.weiss@dissecto.com>
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
polybassa and others added 22 commits May 25, 2026 18:20
@polybassa
Enhance pickle compatibility: support legacy tuple state and maintain… (
038ccc6 
secdev#4982)

* Enhance pickle compatibility: support legacy tuple state and maintain packet metadata during serialization

The new regression tests protect against breaking older pickle payloads by ensuring legacy tuple-state unpickling still works.
They also lock in current behavior for the new state format by checking full metadata and comments fidelity after pickle roundtrip.
This gives coverage for both backward compatibility and current serialization behavior in one place: test/regression.uts.

* update

* fix mypy

---------

Co-authored-by: Nils Weiss <nils.weiss@dissecto.com>
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@T3pp31 @polybassa
dot15d4: fix aux_sec_header incorrect parsing (secdev#4969)
e48f6b2 
* dot15d4: fix aux_sec_header incorrect parsing (secdev#4928)

Fix two bugs preventing correct parsing of 802.15.4 frames with the
security bit set:

1. Replace `is True` identity check with truthiness check in
   ConditionalField lambdas for Dot15d4Data, Dot15d4Beacon, and
   Dot15d4Cmd. In Python 3, `1 is True` is False because `is` checks
   object identity, not equality, so aux_sec_header was never parsed.

2. Add extract_padding() to Dot15d4AuxSecurityHeader so that remaining
   bytes after the header fields are returned to the parent packet
   instead of being consumed as payload.

* test: wrap encrypted beacon test with no_debug_dissector

The encrypted beacon payload cannot be parsed as ZigBeeBeacon, which
raises an exception when conf.debug_dissector is True (as in CI).

* dot15d4: prevent dissection of encrypted payloads as upper layers

When sec_sc_seclevel >= 4 (ENC, ENC-MIC-*), the payload after
aux_sec_header is encrypted and must not be passed to upper layer
dissectors (SixLoWPAN, ZigBee, etc.). Add encrypted payload checks
to guess_payload_class() in Dot15d4Data, Dot15d4Beacon, and
Dot15d4Cmd.
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@polybassa
Add SAE J1939 protocol support to Scapy contrib (secdev#4963)
1667337 
* Add SAE J1939 protocol support to Scapy contrib

* Refactor NativeJ1939Socket test: replace recv with sniff and remove unused import

* Enhance NativeJ1939Socket: handle SO_BROADCAST, update filter padding, tighten ancillary checks, and extend interoperability tests

* Enhance J1939 support: improve rdcandump handling, refine packet assertions, and streamline subprocess communication

* Refactor J1939 filter documentation: add noqa comments for line length and clarify struct size details

* Enhance J1939 segmented communication tests: add comprehensive tests for TP.CM, RTS, CTS, ACK, and ABORT scenarios

* Enhance J1939 support: add comprehensive unit tests for J1939_CAN and J1939_TP_CM, including dispatch hooks, field validations, and filter functionality

* Enhance J1939 support: update address masking in filters, improve destination addressing in send functions, and adjust thread handling for sniffing

* Enhance J1939 support: backfill socket-level constants for compatibility with older Python versions

* Enhance J1939 support: backfill socket-level constants for improved compatibility and refactor imports

* Enhance J1939 support: add line length comments to improve code readability and maintainability
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@BenGardiner @polybassa
can, isotp, soft: threding fixes and small buffer device support impr…
e2a3d87 
…ovements (secdev#4983)

* try harder to drain before close to avoid warnings
* unit tests to trigger issues encountered using soft sockt with slcan,
  candle, cantact on windows
* threading robustness improvements
* robustness improvements when using can adapters with small buffers
* improvements to the heuristics for detecting can adapters with no
  hardware filtering
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@mohamedsolaiman @polybassa
Fix incorrect layer bindings in LDP contrib module (secdev#4985)
35b33d9 
* fix wrong layer binding in LDP contrib module

was testing LDP discovery over UDP and noticed it wasnt dissecting
properly. the bind_bottom_up calls for UDP were wrong - they had
(TCP, UDP) instead of (UDP, LDP). LDP discovery uses UDP port 646
so this needs to bind UDP to LDP not TCP to UDP.

* add test for LDP over UDP dissection

added a test to make sure LDP hello over UDP port 646
gets dissected properly with the fixed bindings

---------

Co-authored-by: Mohamed Solaiman <mohamedsolaiman@users.noreply.github.com>
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@amlamarra
Add support for HSRPv1 Advertise packets (secdev#4972)
15018c2 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@evverx @polybassa
ci: drop downstream import check (secdev#4988)
fcf7d86 
to prevent it from interfering with the upstream CI.

The import check was added recently in
https://src.fedoraproject.org/rpms/scapy/c/878585466261f17c01516a653d19cf47022c2f9f?branch=rawhide
and it started failing when secdev#4975
was merged with
```
+ /usr/bin/python3 -sP /usr/lib/rpm/redhat/import_all_modules.py \
  -f /builddir/build/BUILD/scapy-2.7.0-build/scapy-2.7.0-1.20260423030854887593.pr2.57.g5d1727ff.fc43.x86_64-pyproject-modules \
  -e scapy.arch.bpf.core -e scapy.arch.bpf.supersocket -e scapy.arch.windows \
  -e scapy.arch.windows.native -e scapy.arch.windows.structures \
  -e scapy.contrib.cansocket_python_can -e scapy.tools.generate_bluetooth \
  -e scapy.tools.generate_ethertypes -e scapy.tools.generate_manuf \
  -e scapy.tools.scapy_pyannotate -e scapy.libs.winpcapy
...
...
  File "/builddir/build/BUILD/scapy-2.7.0-build/BUILDROOT/usr/lib/python3.14/site-packages/scapy/arch/windows/__init__.py", line 18, in <module>
    import winreg
ModuleNotFoundError: No module named 'winreg'
```
Since it can't keep up with upstream PRs it can be trimmed.

Big endian builds are back to normal
(https://forge.fedoraproject.org/infra/ansible/pulls/3316) and left
intact.

Closes secdev#4979
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@XenoKovah @gpotter2 @polybassa
bluetooth: add missing HCI events, EIR elements, and SMP packet types (…
81a960f 
…secdev#4987)

* bluetooth: add missing HCI events, EIR elements, and SMP packet types

New packet definitions covering frequently-seen Bluetooth Core 5.4 events
and Generic Access Profile data types that scapy did not yet decode:

  HCI events
  - HCI_Event_Connection_Request                        (code=0x04)
  - HCI_Event_Remote_Host_Supported_Features_Notification (code=0x3d)
  - HCI_Event_Vendor (vendor-specific debug)            (code=0xff)
  - HCI_LE_Meta_LE_Read_Remote_Features_Complete        (LE Meta event=0x04)

  EIR/AD elements
  - EIR_RandomTargetAddress                             (type=0x18)
  - EIR_LERole                                          (type=0x1c)
  - EIR_BroadcastName                                   (type=0x30)
  - EIR_3DInformation                                   (type=0x3d)

  SMP
  - SM_Keypress_Notification                            (sm_command=0x0e)

Each packet has an entry in test/scapy/layers/bluetooth.uts with a
sample drawn from a real HCI snoop log (BR/EDR Connection Request,
Remote Host Supported Features Notification, LE Read Remote Features
Complete, vendor-specific debug event, EIR 3D Information, EIR
Broadcast Name) and synthetic round-trip build/parse tests for the
remainder.

* Apply suggestions from code review

Co-authored-by: Gabriel <10530980+gpotter2@users.noreply.github.com>

---------

Co-authored-by: Gabriel <10530980+gpotter2@users.noreply.github.com>
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
Add packaging status to the README
1237480 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
pr: Introduce AI-Assisted requirements
3744656 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
pr: Update Github PR template
4143b57 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
pr: fix actions deprecation warnings
8292422 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
pr: bump CI versions
4d638b2 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
gh: skip the steps, not the job for push jobs
58406a6 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2
gh: improve description of SPDX test
5b48d18 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
atmt: also use Session in IO supersockets
bfc4cb3 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
fwdm: use more generic TLS client version for ForwardMacine
5a7eecb 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
snmp: add snmpget function
e050f76 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@gpotter2 @polybassa
smb: new smbclient() features
0d7616a 
AI-Assisted: no

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@MatrixEditor
Fix NETLOGON_LOGON_QUERY parsing: Add mailslot name alignment (secdev…
6145e80 
…#4952)

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@polybassa
Initial plan
44a7f99 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@polybassa
Add generic codec field base classes (GenericCodecField, GenericCodec…
64b2505 
…OptionalField)

Agent-Logs-Url: https://github.com/polybassa/scapy-1/sessions/0d67a9ce-67ba-4f73-bdd8-3210d6deaf47

Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@polybassa
Copy link
Copy Markdown
Owner

polybassa commented May 25, 2026

@copilot reintegrate master

Copilot AI and others added 2 commits May 25, 2026 19:22
@polybassa
Initial plan
81b33c9 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
@polybassa
Initial plan
745d2e7 
Co-authored-by: polybassa <1676055+polybassa@users.noreply.github.com>
Copy link
Copy Markdown
Author

Copilot AI commented May 25, 2026

@copilot reintegrate master

Done — rebased onto the latest origin/master (commit 65816b7). Branch now includes all 38 upstream commits. Tests confirmed no regressions (commit 745d2e7).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@polybassa polybassa Awaiting requested review from polybassa

Assignees

@polybassa polybassa

Copilot code review Copilot

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.