Researchers have built a new testbed, DYNAS, that expands the older NASimEmu network‑attack model by adding realistic defensive plumbing. DYNAS injects stateful intrusion‑detection‑system (IDS) feedback, simulates scan errors, spills over temporary service outages, and imposes timing limits on attack actions. These features create a more convincing battlefield for AI agents that learn to conduct network penetration tests.

A key part of the study is Pen‑DHRL, a hierarchical reinforcement‑learning controller that manages subgoals with a global context decorated by the IDS state. Pen‑DHRL runs a long‑term manager that picks persistent objectives while a worker module decides on lower‑level moves.

In head‑on benchmark runs, Pen‑DHRL outperforms three baseline methods (GNN, Invariant, Attention) when the agent is trained without any active IDS and tested with an IDS enabled. On the “Bridge” scenario Pen‑DHRL earns about 64 % more reward per step, and on the “Dynamic” scenario it gains roughly 24 % more reward per step. For the “VarA” and “VarB” scenarios it captures roughly 26 % and 30 % more hosts than the next best baseline.

When defensive dynamics are turned on from the very start of training, flat‑reward baselines fail badly: they earn negative reward per step, capture almost no hosts, and often hit the interaction limit. Pen‑DHRL, by contrast, continues to deliver positive episode returns across all four scenarios, showing that its hierarchical design can handle fully dynamic environments.

These results suggest that DYNAS, together with Pen‑DHRL, provides a more realistic testbed for training AI that can learn to traverse networks while coping with evolving IDS defenses. The approach remains preliminary and is limited to controlled simulation settings; future work is needed to assess deployment in real networks.