hickory-proto: NSEC3 closest-encloser proof validation enters unbounded loop on cross-zone responses

The NSEC3 closest-encloser proof validation in hickory-proto's DnssecDnsHandle walks from the QNAME up to the SOA owner name, building a list of candidate encloser names. The iterator used assumes the QNAME is a descendant of the SOA owner, terminating only when the current candidate equals the SOA name. When the SOA in a response's authority section is not an ancestor of the QNAME, the loop stalls at the DNS root and never terminates, repeatedly calling Name::base_name() and pushing newly allocated Name and hashed-name entries into the candidate Vec.

The bug is reachable by any caller of DnssecDnsHandle — including the resolver, recursor, and client — when built with the dnssec-ring or dnssec-aws-lc-rs feature and configured to perform DNSSEC validation. It is triggered while validating a NoData or NXDomain response whose authority section contains an SOA record from a zone other than an ancestor of the QNAME, on a code path that requires NSEC3 closest-encloser proof. In practice this can be reached through an insecure CNAME chain that crosses zone boundaries into a DNSSEC-signed zone returning NoData, but the minimum condition is just a mismatched SOA owner on a response requiring NSEC3 validation.

A debug_assert_ne!(name, Name::root()) guards the loop body, so debug builds abort with a panic on the first iteration past the root. Release builds compile the assertion out and run the loop unbounded, allocating until the process exhausts available memory (OOM). A reachable upstream attacker who can return such a response can therefore crash a debug-built validator or exhaust memory on a release-built one.

The affected code was migrated from hickory-proto to hickory-net as part of the 0.26.0 release. The hickory-proto 0.26.x release no longer offers DnssecDnsHandle and so we recommend all affected users update to hickory-net 0.26.1 when the implementation of that type is required.