diff --git a/CHANGELOG.md b/CHANGELOG.md index 5c58f07..b758893 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,18 @@ The format follows [Keep a Changelog](https://keepachangelog.com/); versions fol ### Added +- **Lose packets in runs instead of one at a time.** A new "Losses in a row" field, and + `--loss-burst`, sets how many packets are lost in a row on average. "Loss" still decides how + much goes missing overall, so 5 percent stays 5 percent and simply arrives in clusters. Spread + out, most connections absorb it. In runs of twenty it stalls transfers and forces reconnects, + which is usually what you meant to test. Set 0, the default, to spread it evenly as before. + Profiles remember it, and the log says how often to expect a run. + +- **A "Loss runs" counter** on the Statistics tab, in the stats CSV and in the reproduction + report, says how many runs of lost packets a session actually produced. Zero there, with a + run length set, means the session was too short to see one rather than the setting doing + nothing. + - **Aim at one address family.** Two new switches under the destination target, in the GUI and on the command line, impair IPv4 only or IPv6 only. The other family keeps flowing untouched: nothing is blocked and nothing is slowed, it is simply left alone. They work diff --git a/README.md b/README.md index 59aab49..532d3d9 100644 --- a/README.md +++ b/README.md @@ -296,14 +296,30 @@ one. With the given probability it appends extra delay (ms) to a **single packet momentary "lag" actually arrives. The chance is per packet and applies **in each direction**, so a round trip hits it about twice as often as the number suggests. -**Impairment (%)** - *Loss*: percentage of packets vanishing without a trace (5% is already a +**Impairments** - *Loss*: percentage of packets vanishing without a trace (5% is already a clearly failing network). *Corruption*: percentage of packets with a flipped data bit - it affects **only payload-bearing packets**. Packets with no data (e.g. pure ACK, SYN) have nothing to flip, so they pass untouched and are **not counted as corrupted**. *Duplication*: percentage of packets sent twice. +**Losses in a row** - real links rarely lose packets one at a time. A microwave oven, a lift or a +switch between transmitters takes the connection away for a moment, and everything sent in that +moment is gone. This field says how many packets are lost in a row **on average**, while *Loss* +still decides how much is lost in total. It matters more than it looks: 5% spread out is something +most connections absorb, while the same 5% in runs of twenty stalls a transfer, breaks a live +connection and sends an application down its reconnect path. Set 0 to spread the loss evenly, which +is what the tool did before this setting existed. + +Two limits worth knowing, and the log states both when you apply the settings. A very high loss +cannot arrive in very short runs, because runs that short leave too little room between them, so +the run says what it will really deliver. And a long run length puts the runs far apart, so a short +session may not see one at all. Each direction gets its own runs, so a run of twenty means twenty +in a row in that direction. + **Link flapping** - cyclic total loss of traffic: every *Period* seconds the link is dead for the -given percentage of the time. Simulates a flickering connection. +given percentage of the time. Simulates a flickering connection. This is **not** the same as losses +in a row: flapping is a fixed cycle you can predict, the runs above are random and short in the +middle of otherwise normal traffic. **Advanced (NAT / connections):** - *Target destination (IP/port)* - impair only traffic to/from chosen servers. Both fields accept @@ -370,7 +386,8 @@ next to the value says so instead of inventing a source. A few of them are worth measured median, not a worst case. Aircraft with newer low-orbit equipment behave like "Satellite (low orbit)" instead. -A profile stores **what the link is like**: loss, corruption, duplication, latency, jitter, latency +A profile stores **what the link is like**: loss, how much of it arrives in a row, corruption, +duplication, latency, jitter, latency spikes, link outages (flapping), the speed limits and the buffer. Everything else - the target, the destination, blocking, RST, MTU, NAT expiry, the schedule, the seed - stays out of it. Saving a profile warns you about the ones you currently have switched on. Use **"Save file..."** for the @@ -566,7 +583,8 @@ beneath it (in the UI language), and the CLI ends with a readable `error: ...` - The throughput chart has a Y axis with values (KB/s), a grid, a "nicely" rounded scale and current down/up readouts in the corner. Download/Upload (KB/s live), Packets (how many passed), Queued -(waiting - grows with delay/limit), Lost, Corrupted, Duplicated, Buffer overflow (dropped when the +(waiting - grows with delay/limit), Lost, Loss runs (how many RUNS that loss arrived +in - see "Losses in a row"), Corrupted, Duplicated, Buffer overflow (dropped when the tool is overloaded), Dropped at stop (were still queued when STOP was pressed), Send failed (the tool captured them but could not put them back on the wire - the connection went down, or the driver refused), Rate-limit drop (dropped by a full speed-limit buffer - counted separately from @@ -694,6 +712,7 @@ BeanNetworkTester.exe --simulate --duration 30 --format json > run.ndjson | Flag | Unit | Description | |---|---|---| | `--loss` | % | percentage of dropped packets | +| `--loss-burst` | packets | average number of packets lost in a row (0 = loss spread evenly). Shapes `--loss`, it does not add to it | | `--corrupt` | % | percentage of packets with a flipped bit | | `--dup` | % | percentage of packets sent twice | | `--latency` | ms | fixed delay added to every packet | @@ -916,6 +935,7 @@ what `packets_seen` counted in the first place - so every row records it in `cap | `packets_seen` | packets captured | | `packets_in_scope` | of those, the ones targeting selected for impairment | | `dropped_loss` | dropped by the Loss setting | +| `loss_runs` | how many RUNS that loss arrived in (see "Losses in a row"). 0 with a run length set means the session was too short to see one | | `dropped_overflow` | dropped because the tool's own queue was full (see the note on it below) | | `corrupted` | packets whose payload was flipped | | `duplicated` | extra copies queued | diff --git a/README.pl.md b/README.pl.md index 4efcb35..1cd03e8 100644 --- a/README.pl.md +++ b/README.pl.md @@ -231,14 +231,30 @@ Z podanym prawdopodobieństwem dokleja dodatkowe opóźnienie (ms) do **pojedync tak, jak chwilowy „lag” naprawdę wygląda. Szansa liczona jest na pakiet i działa **w każdą stronę**, więc pojedyncze odpytanie trafia na nią mniej więcej dwa razy częściej, niż sugeruje ta liczba. -**Zakłócenia (%)** - *Utrata*: procent pakietów znikających bez śladu (5% to już wyraźnie +**Zakłócenia** - *Utrata*: procent pakietów znikających bez śladu (5% to już wyraźnie zrywająca się sieć). *Uszkodzenie*: procent pakietów z przekłamanym bitem danych - dotyczy tylko pakietów z ładunkiem (payloadem). Pakiety bez danych (np. czyste ACK, SYN) nie mają czego przekłamać, więc przechodzą nietknięte i **nie są liczone jako uszkodzone**. *Duplikacja*: procent pakietów wysyłanych podwójnie. +**Straty pod rząd** - prawdziwe łącza rzadko gubią pakiety po jednym. Mikrofalówka, winda albo +przełączenie się między nadajnikami zabiera połączenie na moment, a wszystko wysłane w tym +momencie przepada. To pole mówi, ile pakietów ginie pod rząd **średnio**, a *Utrata* nadal +decyduje, ile ginie w sumie. To znaczy więcej, niż wygląda: 5% rozłożonych równomiernie +większość połączeń wchłania, a te same 5% w seriach po dwadzieścia zatrzymuje transfer, zrywa +połączenie na żywo i wysyła aplikację w ścieżkę ponownego łączenia. Ustaw 0, żeby rozłożyć stratę +równomiernie, czyli tak, jak narzędzie działało, zanim to ustawienie powstało. + +Dwa ograniczenia, o których warto wiedzieć, i log mówi o obu przy zastosowaniu ustawień. Bardzo +wysoka strata nie zmieści się w bardzo krótkich seriach, bo tak krótkie serie zostawiają za mało +miejsca między sobą, więc przebieg mówi, ile naprawdę zgubi. A długa seria stawia serie daleko od +siebie, więc krótka sesja może nie zobaczyć żadnej. Każdy kierunek ma własne serie, więc seria po +dwadzieścia znaczy dwadzieścia pod rząd w tym kierunku. + **Przerwy w łączu (flapping)** - cykliczne całkowite zrywanie ruchu: co *Okres* sekund łącze -jest martwe przez podany procent czasu. Symuluje migające połączenie. +jest martwe przez podany procent czasu. Symuluje migające połączenie. To **nie** to samo co straty +pod rząd: flapping jest stałym cyklem, który da się przewidzieć, a serie wyżej są losowe i krótkie, +w środku normalnego ruchu. **Zaawansowane (NAT / połączenia):** - *Celuj w cel (IP/port)* - psuj tylko ruch do/od wybranych serwerów. Oba pola przyjmują listy, zakresy, CIDR, wildcardy, porównania, wykluczenia i wyrażenia regularne - patrz [Składnia filtrów](#składnia-filtrów-proces--ip--port). Np. IP `10.0.0.1-10.0.0.50,!10.0.0.7`, port `80,443,8000-8100`. Puste = dowolne. @@ -269,7 +285,7 @@ Ich liczby pochodzą z opublikowanych pomiarów wszędzie tam, gdzie pomiary ist - **Pociąg / metro (tunele)** - jedyny preset, który kładzie łącze całkowicie na kilka sekund (3 s na każde 30), więc aplikacja musi się **połączyć od nowa**, a nie tylko zwolnić. - **Wi-Fi w samolocie** - ten klasyczny, satelitarny: ~750 ms pingu i 7% strat, przy czym to jest zmierzona **mediana**, nie najgorszy przypadek. Samoloty z nowszym sprzętem niskoorbitalnym zachowują się jak „Satelita niskoorbitalny”. -Profil zapisuje to, **jakie jest łącze**: stratę, uszkodzenia, duplikację, opóźnienie, jitter, skoki latencji, przerwy w łączu (flapping), limity prędkości i bufor. Reszta ustawień - cel, adres docelowy, blokada, RST, MTU, wygasanie NAT, harmonogram, seed - do profilu nie wchodzi. Przy zapisie zobaczysz ostrzeżenie z listą tych, które akurat masz włączone. Pełną konfigurację zapisujesz przyciskiem **„Zapisz plik...”**. Wybranie profilu albo presetu ustawia **wszystkie** te pola naraz, także te, których dany preset nie wymienia - wracają wtedy do wartości domyślnej, żeby „Idealna sieć” naprawdę znaczyła idealną. Profile zapisane wcześniejszą wersją wczytują się bez zmian. +Profil zapisuje to, **jakie jest łącze**: stratę, to ile jej przychodzi pod rząd, uszkodzenia, duplikację, opóźnienie, jitter, skoki latencji, przerwy w łączu (flapping), limity prędkości i bufor. Reszta ustawień - cel, adres docelowy, blokada, RST, MTU, wygasanie NAT, harmonogram, seed - do profilu nie wchodzi. Przy zapisie zobaczysz ostrzeżenie z listą tych, które akurat masz włączone. Pełną konfigurację zapisujesz przyciskiem **„Zapisz plik...”**. Wybranie profilu albo presetu ustawia **wszystkie** te pola naraz, także te, których dany preset nie wymienia - wracają wtedy do wartości domyślnej, żeby „Idealna sieć” naprawdę znaczyła idealną. Profile zapisane wcześniejszą wersją wczytują się bez zmian. W CLI (`--preset`) preset można podać przez **kanoniczne id** albo **nazwę w dowolnym języku UI** (bez rozróżniania wielkości liter i polskich znaków - `"Idealna siec"` też zadziała). Id: `presets.perfect`, `presets.good_wifi`, `presets.5g`, `presets.dsl`, `presets.lte`, `presets.leo`, `presets.distant`, `presets.weak_wifi`, `presets.cafe`, `presets.bufferbloat`, `presets.metro`, `presets.3g`, `presets.roaming`, `presets.satellite`, `presets.inflight`, `presets.modem56k`, `presets.terrible`. @@ -450,7 +466,8 @@ powód (w języku interfejsu), a CLI kończy się czytelnym `error: ...` - nigdy ## Statystyki (co znaczą liczniki) Wykres przepustowości ma teraz oś Y z wartościami (KB/s), siatkę, „ładnie” zaokrągloną skalę oraz bieżące odczyty down/up w rogu. Pobieranie/Wysyłanie (KB/s na żywo), Pakiety (ile przeszło), W kolejce (czekające - rośnie przy -opóźnieniu/limicie), Utracone, Uszkodzone, Zduplikowane, Bufor przepełn. (porzucone przy +opóźnieniu/limicie), Utracone, Serie strat (w ilu SERIACH przyszła ta strata - patrz +„Straty pod rząd”), Uszkodzone, Zduplikowane, Bufor przepełn. (porzucone przy przeciążeniu narzędzia), Porzuc. przy stopie (czekały w kolejce, gdy nacisnięto STOP), Nie odesłane (narzędzie je przechwyciło, ale nie zdołało odesłać do sieci - padło połączenie albo sterownik odrzucił pakiet), Odrzuc. przez limit (porzucone przez pełny bufor limitu prędkości - @@ -545,6 +562,7 @@ BeanNetworkTester.exe --simulate --duration 30 --format json > run.ndjson | Flaga | Jednostka | Opis | |---|---|---| | `--loss` | % | procent gubionych pakietów | +| `--loss-burst` | pakietów | średnia liczba pakietów gubionych pod rząd (0 = strata rozłożona równomiernie). Kształtuje `--loss`, nie dokłada się do niego | | `--corrupt` | % | procent pakietów z przekłamanym bitem | | `--dup` | % | procent pakietów wysłanych podwójnie | | `--latency` | ms | stałe opóźnienie doklejane do każdego pakietu | @@ -766,6 +784,7 @@ w ogóle policzył - więc każdy wiersz zapisuje to w kolumnie `capture_narrowe | `packets_seen` | przechwycone pakiety | | `packets_in_scope` | z tego te, które celowanie wybrało do psucia | | `dropped_loss` | odrzucone przez ustawienie Strata | +| `loss_runs` | w ilu SERIACH przyszła ta strata (patrz „Straty pod rząd”). 0 przy ustawionej długości serii znaczy, że sesja była za krótka, żeby zobaczyć choć jedną | | `dropped_overflow` | odrzucone, bo kolejka samego narzędzia była pełna | | `corrupted` | pakiety z przekłamaną zawartością | | `duplicated` | dołożone kopie | diff --git a/beantester/cli.py b/beantester/cli.py index 3476989..03175fc 100644 --- a/beantester/cli.py +++ b/beantester/cli.py @@ -177,6 +177,8 @@ def build_arg_parser(): help="which traffic to capture at all (IPv4 and IPv6). Ports are " "filtered with --dst-port, not here") p.add_argument("--loss", type=float, help="packet loss [%%]") + p.add_argument("--loss-burst", type=float, + help="average packets lost in a row (0 = spread evenly)") p.add_argument("--corrupt", type=float, help="corruption [%%]") p.add_argument("--dup", type=float, help="duplication [%%]") p.add_argument("--latency", type=float, help="latency [ms]") diff --git a/beantester/core.py b/beantester/core.py index a7439b0..252702a 100644 --- a/beantester/core.py +++ b/beantester/core.py @@ -35,6 +35,66 @@ class Decision(NamedTuple): scoped: bool = True +def burst_loss_params(loss, mean_burst): + """Turn "this much loss, arriving in runs this long" into a two-state chain. + + Returns ``(p, r, achievable)`` - the good-to-bad and bad-to-good transition + probabilities, plus the loss fraction that pair actually delivers - or + ``None`` when the loss should stay INDEPENDENT, which is the behaviour that + predates this function and the one every default still takes. + + The model is Gilbert's two-state burst-noise channel (Gilbert 1960, extended + by Elliott 1963), the same one ``tc netem`` offers as ``loss gemodel``. It is + used here in its Simple Gilbert form - the good state loses nothing, the bad + state loses everything - so ONE draw per packet decides the transition and + the verdict follows from the state. That is not a micro-optimisation: it + keeps the packet path on exactly the number of RNG draws it made before + bursts existed, and a stored ``Reproduce:`` command has to replay identically. + + A tester can answer "how much loss" and "how long a run", so those are the + inputs and the transition probabilities are derived from them: + + r = 1 / mean_burst a bad run is geometric, so its mean is 1/r + p = loss * r / (1 - loss) from the stationary share pi_B = p / (p + r) + + The reparametrisation through the average run length is the one described by + Hasslinger and Hohlfeld, "The Gilbert-Elliott Model for Packet Loss in Real + Time Services on the Internet", MMB 2008. Their printed formula for ``p`` + disagrees with the notation of their own figure, so the line above was + re-derived here and then MEASURED rather than trusted: over 2 million packets + the delivered loss and the delivered mean run length both land on the request + inside the sampling noise, from 0.5% upward. + + 🔴 Not every request is possible, and that is what ``achievable`` is for. + ``p <= 1`` needs ``mean_burst >= loss / (1 - loss)``, so 90% loss cannot + arrive in runs of 5 - runs that short leave too little room between them. + The pair is CLAMPED to the most that length can carry, and the caller is + expected to say so out loud instead of letting a run quietly miss its own + setting (``settings.apply_settings``). Measured: 90% asked for in runs of 5 + delivers 83.3%, and this function's own answer for that case is 83.33%. + """ + if not loss or not mean_burst or mean_burst <= 1.0: + # A mean run of one packet is not "bursty with short bursts" - with r = 1 + # the chain can never leave a loss next to a loss, so it is ANTI + # correlated rather than independent (measured: 50% loss at a run length + # of 1 lands on a perfectly alternating pattern). Anything at or below + # one therefore means the plain independent draw. + return None + r = 1.0 / mean_burst + room = 1.0 - loss + if room <= 0.0: + # Total loss. Every packet goes whatever the chain says, so the chain may + # as well stay bad - and this branch is what keeps the division below + # from raising on exactly this input. + return (1.0, r, 1.0) + p = loss * r / room + if p >= 1.0: + # More loss than runs this short can carry. The good state then lasts a + # single packet, which is the most this run length can deliver. + return (1.0, r, 1.0 / (1.0 + r)) + return (p, r, loss) + + MAX_FLOWS = 200_000 # hard ceiling on each flow table (see _FlowTable) FLOW_ROTATE_S = 30.0 # a flow survives at least this long without traffic @@ -208,6 +268,20 @@ def __init__(self): self._lock = threading.Lock() # impairments self.loss = 0.0 + # Burst loss: the average length, in PACKETS, of a run of lost packets. + # 0 means the independent draw this tool has always made. The chain's two + # transition probabilities are derived once per apply (_recompute_burst) + # and there is one chain PER DIRECTION - see _loses for why. + self.loss_burst = 0.0 + self._burst_p = None + self._burst_r = 0.0 + self._loss_bad = {True: False, False: False} + # Runs STARTED, both directions together. The engine merges it into the + # statistics snapshot, because "did the model fire at all" is not + # answerable from the drop count: a long run length can space the runs + # far enough apart that a short session sees none, and a session that + # changed nothing has to be distinguishable from a tool that is broken. + self.loss_bursts = 0 self.corrupt = 0.0 self.dup = 0.0 self.latency_s = 0.0 @@ -316,6 +390,79 @@ def set_params(self, loss_pct, corrupt_pct, dup_pct, self.jitter_s = max(0.0, jitter_ms) / 1000.0 self.rate_down = self._rate_bps(down_kbps) self.rate_up = self._rate_bps(up_kbps) + # The burst chain is derived from the loss AND from the run length, + # so it has to be re-derived here too - see _recompute_burst. + self._recompute_burst() + + def set_loss_burst(self, mean_packets): + """Average length, in packets, of a run of lost packets. 0 = independent. + + Separate from ``set_params`` rather than an eighth argument to it: that + signature is called directly by tests, rigs and every caller that has + ever configured a core, and widening it would move all of them for a + field most of them do not set. + """ + with self._lock: + self.loss_burst = max(0.0, float(mean_packets or 0.0)) + self._recompute_burst() + + def _recompute_burst(self): + """Re-derive the chain from the two fields that feed it. + + Called from BOTH setters that can change either half, and that is the + point. ``p`` depends on the loss as much as on the run length, so a + single owner would leave the ORDER of two setter calls deciding whether + the answer is right - and ``set_params`` is called on its own by tests, + by rigs and by anything that only means to change the loss. The symptom + would have been quiet: the delivered loss drifting away from the field + that asked for it, with nothing going red. + + Only a REAL change restarts the chain. A scenario stepping the speed + limit calls every setter on every step change (``scenario_runner``), and + restarting here unconditionally would cut every run in flight - at 50 + packets a second a run of 20 lasts 400 ms, so most of them. + """ + params = burst_loss_params(self.loss, self.loss_burst) + p = None if params is None else params[0] + r = 0.0 if params is None else params[1] + if (p, r) != (self._burst_p, self._burst_r): + self._loss_bad[True] = self._loss_bad[False] = False + self._burst_p, self._burst_r = p, r + + def _loses(self, rng, is_outbound): + """Does this packet fall to the configured loss? Step 8's whole question. + + Independent by default - the draw this tool has always made, reached + through one attribute test. With a run length configured it walks the + two-state chain instead, ONE CHAIN PER DIRECTION, keyed the same way the + token bucket in step 11 is. + + Per direction rather than one shared chain, and that is measured, not + tidiness: a shared chain delivers HALF of what the user typed, because + with the default two-way filter a run of 20 is split across both + directions. Measured over 2 million packets, one direction saw runs of + 10.4 at a 50/50 mix and 11.9 at 90/10 - so the error follows the traffic + and there is not even a fixed factor a reader could correct for. The + whole-link outage that really does cut both ways at once is ``flap`` + (step 5), which is a different impairment on purpose. + + The chain advances only for packets that REACH step 8. Anything the + targeting gate, an address-class switch, blocking, NAT, RST, flapping, + the MTU hole or SYN dropping already took is not part of the run - so + with a process target the run length is counted in the target's packets, + which is the number the tester meant. + """ + if self._burst_p is None: + return rng.random() < self.loss + bad = self._loss_bad[is_outbound] + if bad: + if rng.random() < self._burst_r: + bad = False + elif rng.random() < self._burst_p: + bad = True + self.loss_bursts += 1 + self._loss_bad[is_outbound] = bad + return bad def set_buffer(self, buffer_ms): """Bounded link buffer for the rate limiter, in ms. 0 == unbounded.""" @@ -543,6 +690,10 @@ def reset_buckets(self, now): self._flow_last.clear() self._reset_until.clear() self._prune_next = 0.0 + # A session that ended mid-run must not start the next one inside it: + # the same reason the schedule and the token buckets restart here. + self._loss_bad[True] = self._loss_bad[False] = False + self.loss_bursts = 0 # -- helpers ------------------------------------------------------------ # @staticmethod @@ -787,8 +938,12 @@ def decide(self, size, is_outbound, local_port, now, rng, if is_syn and self.syn_drop > 0 and rng.random() < self.syn_drop: return Decision(True, False, [], "syn") - # 8) loss - if self.loss > 0 and rng.random() < self.loss: + # 8) loss - independent, or arriving in runs when a burst length is + # set. The whole question moves into _loses() rather than growing a + # branch here: this function sits ON the complexity ceiling pinned in + # pyproject.toml, where the rule is to move code out instead of + # raising the number. Measured after the change: still 27. + if self.loss > 0 and self._loses(rng, is_outbound): return Decision(True, False, []) # 9) corruption diff --git a/beantester/engine.py b/beantester/engine.py index 7d7a33a..af677aa 100644 --- a/beantester/engine.py +++ b/beantester/engine.py @@ -372,6 +372,9 @@ def set_params(self, *a): def set_buffer(self, *a): self.core.set_buffer(*a) + def set_loss_burst(self, *a): + self.core.set_loss_burst(*a) + def set_target(self, active, ports=None): """Point the engine at a set of local ports (or a live port container). @@ -576,6 +579,13 @@ def reset_stats(self): # narrowed to the traffic the tool was asked to impair - and because # the stats CSV takes its column order from this dict. self.st = dict(seen=0, scoped_seen=0, + # Sits next to drop_loss because it describes the SAME + # loss: how many runs it arrived in. Zeroed here for + # the CSV column order (this dict is where it comes + # from); the value itself is the core's, and + # stats_snapshot merges it - the core zeroes it in + # reset_buckets, which start() calls in the same breath. + loss_bursts=0, drop_loss=0, drop_overflow=0, corrupted=0, duplicated=0, drop_syn=0, drop_mtu=0, drop_nat=0, drop_rst=0, drop_lan=0, drop_internet_only=0, @@ -898,6 +908,13 @@ def stats_snapshot(self): s = dict(self.st) with self._cv: s["queue"] = len(self._heap) + # Counted by the DECISION core, not by this loop: only ``decide()`` can + # see a run of losses begin. Read without ``core._lock`` on purpose and + # for the same reason ``SocketWatcher.pid_for`` takes none - it is one + # int, the capture thread holds that lock for every packet, and a snapshot + # is allowed to be a moment old. (A free-threaded build would have to + # revisit this, along with the other lock-free reads - backlog B-2.) + s["loss_bursts"] = self.core.loss_bursts return s def _bump(self, key, n=1): diff --git a/beantester/fields.py b/beantester/fields.py index 277149e..8199cb0 100644 --- a/beantester/fields.py +++ b/beantester/fields.py @@ -81,6 +81,14 @@ class Field(NamedTuple): help_body: str = "" # i18n key of the "?" help-sheet body (optional) impairs: str = "" # "" | IMPAIRS_ALL | IMPAIRS_MATCHED (see above) narrows: bool = False # bounds what EVERY later impairment can reach + # Key of the impairment this field only SHAPES. A parameter sits behind its + # trigger's gate in decide(), so it arms nothing by itself - which is why it + # must not declare ``impairs`` (that would warn about a run damaging nothing). + # Naming the trigger instead of leaving the field blank is what lets the + # pass-through suite derive its own sweep: see PARAMETER_KEYS below, and + # tests/test_passthrough.py::test_a_parameter_at_its_maximum_still_damages_nothing, + # which used to be two field names typed out by hand in that file. + parameter_of: str = "" FIELD_DEFS = ( @@ -164,12 +172,34 @@ class Field(NamedTuple): cli="spike-prob", impairs=IMPAIRS_ALL), Field("spike_ms", NUMBER, "fields.spike_ms", "latency", unit="ms", bounds=MS, width=8, tip="tips.spike", in_profile=True, - cli="spike-ms"), + cli="spike-ms", parameter_of="spike_prob"), # -- impairments ------------------------------------------------------- # Field("loss", NUMBER, "fields.loss", "impairments", unit="%", bounds=PCT, width=6, tip="tips.loss", in_profile=True, cli="loss", impairs=IMPAIRS_ALL), + # How the loss above is DISTRIBUTED, not how much of it there is: the average + # number of packets lost in a row. 0 spreads it evenly, which is what this + # tool did before the field existed and is still the default. + # + # parameter_of="loss" and NOT impairs: decide() step 8 reads it behind + # `self.loss > 0`, so a run length on its own damages nothing and calling it + # an impairment would warn about a session that changes no packet. + # + # in_profile because burstiness is a property of the LINK, exactly like the + # flapping pair and the buffer: a profile that stored "5% loss" without + # saying whether it arrives evenly or in runs stored half of what the link + # does to an application. + # + # The 1000-packet ceiling is a JUDGEMENT, not a measurement. It is far past + # anything a transport notices (a run of 1000 is a multi-second outage on + # most flows) and it keeps the derived gap between runs inside numbers a + # person can read. + Field("loss_burst", NUMBER, "fields.loss_burst", "impairments", + unit_key="fields.unit_pkt_off", bounds=(0.0, 1000.0), width=6, + tip="tips.loss_burst", in_profile=True, cli="loss-burst", + parameter_of="loss", help_title="dialogs.loss_burst_help_title", + help_body="dialogs.loss_burst_help"), Field("corrupt", NUMBER, "fields.corruption", "impairments", unit="%", bounds=PCT, width=6, tip="tips.corrupt", in_profile=True, cli="corrupt", impairs=IMPAIRS_ALL), @@ -187,7 +217,7 @@ class Field(NamedTuple): cli="flap-period", impairs=IMPAIRS_ALL), Field("flap_down", NUMBER, "fields.flap_down_pct", "flapping", unit="%", bounds=PCT, width=6, tip="tips.flap", in_profile=True, - cli="flap-down"), + cli="flap-down", parameter_of="flap_period"), # -- destination ------------------------------------------------------- # Field("dst_ip", EXPR, "fields.ip", "destination", expr_kind=KIND_IP, @@ -245,7 +275,8 @@ class Field(NamedTuple): Field("rst_prob", NUMBER, "fields.rst", "advanced", unit="%", bounds=PCT, width=6, tip="tips.rst", cli="rst-prob", impairs=IMPAIRS_ALL), Field("rst_cooldown", NUMBER, "fields.rst_cooldown", "advanced", unit="s", - bounds=(0.0, 3600.0), width=6, tip="tips.rst_cooldown", cli="rst-cooldown"), + bounds=(0.0, 3600.0), width=6, tip="tips.rst_cooldown", cli="rst-cooldown", + parameter_of="rst_prob"), # -- schedule ---------------------------------------------------------- # Field("rate_schedule", SCHEDULE, "fields.schedule", "schedule", width=34, @@ -340,7 +371,12 @@ class Section(NamedTuple): Section("speed_limit", "frames.speed_limit", ("down", "up", "buffer"), columns=2), Section("latency", "frames.latency", ("latency", "jitter", "spike_prob", "spike_ms"), columns=2), - Section("impairments", "frames.impairments", ("loss", "corrupt", "dup"), columns=3), + # Two columns, not three: it puts "Loss" and the run length that shapes it + # on ONE row, with corruption and duplication on the next. Three columns + # would split the pair across a row boundary, and two fields that answer + # "how much" and "in what shape" read as one setting or as neither. + Section("impairments", "frames.impairments", + ("loss", "loss_burst", "corrupt", "dup"), columns=2), Section("flapping", "frames.flapping", ("flap_period", "flap_down"), columns=2), # columns=2 for the family pair only: both expression fields above carry # span=True and keep a row each regardless, so this changes nothing they do. @@ -403,6 +439,11 @@ def expression_fields(): # global impairment when its expression covers the whole space; see it for why. MATCHED_IMPAIRING_KEYS = tuple(f.key for f in FIELD_DEFS if f.impairs == IMPAIRS_MATCHED) NARROWING_KEYS = tuple(f.key for f in FIELD_DEFS if f.narrows) +# Fields that only SHAPE an impairment (see Field.parameter_of). They damage +# nothing on their own, and the pass-through suite proves exactly that by turning +# each of them up to its maximum with every impairment still off - a stronger +# statement than the two names it used to keep in a list of its own. +PARAMETER_KEYS = tuple(f.key for f in FIELD_DEFS if f.parameter_of) def overriding_field(field): diff --git a/beantester/gui/csv_export.py b/beantester/gui/csv_export.py index e725ff8..9cc227f 100644 --- a/beantester/gui/csv_export.py +++ b/beantester/gui/csv_export.py @@ -20,7 +20,7 @@ # Internal stat keys are engine-speak ("seen"); a CSV is read by people and # by spreadsheets, so it gets column names that mean something. CSV_COLUMNS = {"seen": "packets_seen", "scoped_seen": "packets_in_scope", - "drop_loss": "dropped_loss", + "drop_loss": "dropped_loss", "loss_bursts": "loss_runs", "drop_overflow": "dropped_overflow", "drop_syn": "dropped_syn", "drop_mtu": "dropped_mtu", "drop_nat": "dropped_nat", "drop_rst": "dropped_rst", "rst_reset": "connections_reset", diff --git a/beantester/gui/pages/stats.py b/beantester/gui/pages/stats.py index 8fbb511..3af3935 100644 --- a/beantester/gui/pages/stats.py +++ b/beantester/gui/pages/stats.py @@ -64,6 +64,7 @@ ("seen", "stats.packets", "", "tips.stat_seen"), ("queue", "stats.queued", "", "tips.stat_queue"), ("drop_loss", "stats.dropped", "", "tips.stat_loss"), + ("loss_bursts", "stats.loss_runs", "", "tips.stat_loss_runs"), ("corrupted", "stats.corrupted", "", "tips.stat_corrupted"), ("duplicated", "stats.duplicated", "", "tips.stat_duplicated"), ("drop_overflow", "stats.overflow", "", "tips.stat_overflow"), @@ -490,7 +491,7 @@ def refresh_counters(self): # can be impaired), and drop_overflow / drop_shutdown / drop_send stay on # the FULL traffic on purpose - they are what the TOOL lost, including # traffic the user never targeted, and narrowing them would hide it. - for key in ("seen", "queue", "drop_loss", "corrupted", "duplicated", + for key in ("seen", "queue", "drop_loss", "loss_bursts", "corrupted", "duplicated", "drop_overflow", "drop_shutdown", "drop_send", "drop_rate", "drop_syn", "drop_mtu", "drop_nat", "drop_rst", "drop_lan", "drop_internet_only", diff --git a/beantester/repro.py b/beantester/repro.py index 331edb5..a9ef8b6 100644 --- a/beantester/repro.py +++ b/beantester/repro.py @@ -13,7 +13,8 @@ def settings_to_cli(settings, seed=None, simulate=False): """Build the list of CLI arguments that reproduce the given settings.""" g = lambda k: settings.get(k, DEFAULT_SETTINGS[k]) args = [] - numeric = [("loss", "--loss"), ("corrupt", "--corrupt"), ("dup", "--dup"), + numeric = [("loss", "--loss"), ("loss_burst", "--loss-burst"), + ("corrupt", "--corrupt"), ("dup", "--dup"), ("latency", "--latency"), ("jitter", "--jitter"), ("down", "--down"), ("up", "--up"), ("buffer", "--buffer"), ("syn_drop", "--syn-drop"), ("max_size", "--max-size"), @@ -96,6 +97,10 @@ def build_repro_report(engine, settings): # from before this change is not comparable with one from after. effective_loss_pct=round(impairment_loss_pct(stats), 2), effective_corruption_pct=round(corruption_pct(stats), 2), + # How many RUNS the loss arrived in. Zero with a run length configured + # means the session was too short to see one, which is the difference + # between a run that proved nothing and a tool that is broken. + loss_runs=stats.get("loss_bursts", 0), # connections_reset held drop_rst - the PACKETS a reset connection swallows # during its cooldown, which for a 30 s cooldown on a busy flow is thousands # against a handful of actual resets. The three RST numbers answer three diff --git a/beantester/settings.py b/beantester/settings.py index cc1484f..a099d26 100644 --- a/beantester/settings.py +++ b/beantester/settings.py @@ -10,17 +10,19 @@ from . import crashlog from . import fields as F from . import portmap +from .core import burst_loss_params from .jsonfile import load_json, write_json from .fields import FIELD_DEFS, FIELDS from .i18n import T, translate from .matchers import KIND_PROCESS, parse_matcher, port_expression from .processes import TARGET_FIELD from .targeting import ports_shared_with_others -from .utils import number_string +from .utils import number_string, to_number from .validators import parse_number, parse_seed DEFAULT_SETTINGS = dict( loss=0, corrupt=0, dup=0, latency=0, jitter=0, down=0, up=0, + loss_burst=0, # average packets lost in a row; 0 = spread evenly. See fields.py buffer=1000, # link buffer (ms) for the speed limit; 0 = unbounded. See fields.py filter="both", target="", dst_ip="", dst_port="", lan_mode=False, ipv4_only=False, ipv6_only=False, @@ -500,6 +502,41 @@ def _destination_is_frozen(engine, dst_ip, dst_port): or str(getattr(core, "dst_port", "")) != str(dst_port)) +def _say_what_the_burst_loss_will_do(loss_pct, mean_burst, log): + """Two things a person cannot read off the two fields in front of them. + + Said at APPLY time, before the run, because both of them are the difference + between "the tool is broken" and "the tool did what it was told": + + * **Some pairs are impossible.** Runs of five cannot carry 90% loss - runs + that short leave too little room between them - so the pair is clamped and + the run delivers less loss than the field asks for. Silently delivering a + different number than the one on screen is the exact failure this project + keeps removing, and the arithmetic behind the clamp lives in ONE place + (``core.burst_loss_params``), which is the same place that answers here. + Below 50% loss this can never fire. + * **A long run can put the runs far apart.** 0.1% loss in runs of 1000 is one + run per million packets, which on a quiet connection is hours. The settings + look reasonable and nothing happens, which reads exactly like a broken + tool. So the run length AND the expected distance between runs are said out + loud, in packets, which is the unit the field is in. + """ + loss = to_number(loss_pct) / 100.0 + params = burst_loss_params(loss, to_number(mean_burst)) + if params is None: + return + _p, _r, achievable = params + if achievable < loss: + log(T("log.loss_burst_clamped", burst=number_string(mean_burst), + asked=number_string(loss_pct), + delivered=number_string(round(achievable * 100.0, 2)))) + # Packets per cycle: one run of `mean_burst` for every `mean_burst/achievable` + # packets that go past. Rounded to whole packets - the field is in packets and + # a fractional one would read as precision this cannot have. + log(T("log.loss_burst_gap", burst=number_string(mean_burst), + gap=number_string(round(to_number(mean_burst) / achievable)))) + + def apply_settings(engine, s, log=lambda *_: None): """Configure the engine from a flat settings dict (shared by GUI and CLI). @@ -512,6 +549,8 @@ def apply_settings(engine, s, log=lambda *_: None): engine.set_params(g("loss"), g("corrupt"), g("dup"), g("latency"), g("jitter"), g("down"), g("up")) engine.set_buffer(g("buffer")) + engine.set_loss_burst(g("loss_burst")) + _say_what_the_burst_loss_will_do(g("loss"), g("loss_burst"), log) dst_ip = setting_expression("dst_ip", g("dst_ip")) dst_port = setting_expression("dst_port", g("dst_port")) # With the driver filter narrowed, the destination fields are START-ONLY, and diff --git a/beantester/summary.py b/beantester/summary.py index 24d1c90..c0d83ec 100644 --- a/beantester/summary.py +++ b/beantester/summary.py @@ -1,4 +1,5 @@ """Human-readable, translated one-line summary of the active impairments.""" +from .core import burst_loss_params from .i18n import translate from .settings import DEFAULT_SETTINGS, parse_schedule, setting_expression from .utils import number_string, to_number @@ -24,6 +25,12 @@ def settings_summary(s, lang=None, prefix_key="summary.prefix"): parts.append(tr("summary.jitter", v=num("jitter"))) if to_number(g("loss")): parts.append(tr("summary.loss", v=num("loss"))) + # Same loss figure, very different link: the run length is asked of the + # function that DECIDES it rather than compared against a threshold here, + # so the strip cannot claim runs the engine is not producing. + if burst_loss_params(to_number(g("loss")) / 100.0, + to_number(g("loss_burst"))) is not None: + parts.append(tr("summary.loss_burst", v=num("loss_burst"))) if to_number(g("corrupt")): parts.append(tr("summary.corrupt", v=num("corrupt"))) if to_number(g("dup")): diff --git a/lang/en.json b/lang/en.json index 682e65b..ad1d012 100644 --- a/lang/en.json +++ b/lang/en.json @@ -103,6 +103,8 @@ "dialogs.internal_error_title": "Internal error", "dialogs.load_config": "Load config", "dialogs.load_scenario": "Load scenario", + "dialogs.loss_burst_help": "Real links rarely lose packets one at a time. A microwave oven, a lift, or a switch between transmitters takes the connection away for a moment, and everything sent in that moment is gone.\n\nThis field says how many packets are lost in a row on average. The \"Loss\" field beside it still decides how much is lost in total, so 5 percent stays 5 percent. It simply arrives in a few clusters instead of being sprinkled evenly.\n\nWhy it matters: 5 percent spread out is something most connections absorb without a visible problem. The same 5 percent in runs of twenty stalls a transfer, breaks a live connection, and sends an application down its reconnect path, which is usually the code you wanted to test.\n\nSet 0 to spread the loss evenly, which is how this tool behaved before this setting existed.\n\nTwo limits worth knowing. A very high loss cannot arrive in very short runs, because runs that short leave too little room between them, so the log tells you what the session will really deliver. And a long run length puts the runs far apart, so a short session may not see one at all. The log says roughly how often to expect them.\n\n\"Link outages (flapping)\" further down is a different thing. It cuts the connection on a fixed cycle, on purpose. This one is random and short, in the middle of otherwise normal traffic.", + "dialogs.loss_burst_help_title": "Losses in a row", "dialogs.match_help": "An expression is a comma-separated list of terms. Positives are OR-ed, \"!\" subtracts.\n\n value a single value (a bare process name matches as a substring)\n a, b, c a list\n a-b a range, both ends included (80-88 = 80 through 88)\n >N =N <=N comparisons (numbers only)\n * ? wildcards (chrome*, fire?ox)\n re:PATTERN a regular expression (case-insensitive, escape a comma as \\,)\n 10.0.0.0/24 CIDR (IP fields only, IPv4 and IPv6)\n !value exclude\n\nExamples:\n chrome.exe, !chromedriver\n 80, 443, 8000-8100\n 10.0.0.1-10.0.0.50, !10.0.0.7\n re:^fire\n\nAn empty field means \"everything\". A field with only exclusions means \"everything except those\". A term that cannot be evaluated for a packet (a port rule against ICMP, an IPv4 rule against an IPv6 address) simply does not match.", "dialogs.match_help_title": "Filter expression syntax", "dialogs.missing_library": "Missing library", @@ -194,6 +196,7 @@ "fields.locked_running": "Locked while a session runs (STOP unlocks it).", "fields.loop": "Loop", "fields.loss": "Loss:", + "fields.loss_burst": "Losses in a row:", "fields.match_help": "?", "fields.max_size": "Max size (MTU):", "fields.narrow_filter": "Capture only the targeted traffic", @@ -223,6 +226,7 @@ "fields.target_no_match": "No running process matches this target - traffic is NOT being impaired.", "fields.target_process": "Process:", "fields.unit_b_off": "B (0=off)", + "fields.unit_pkt_off": "packets (0=off)", "fields.unit_rows": "rows", "fields.unit_s_off": "s (0=off)", "fields.upload": "Upload:", @@ -238,7 +242,7 @@ "frames.destination": "Destination targeting", "frames.event_log": "Event log (timestamped)", "frames.flapping": "Link outages (flapping)", - "frames.impairments": "Impairments (%)", + "frames.impairments": "Impairments", "frames.latency": "Latency (ping)", "frames.profiles": "Profiles", "frames.repro": "Reproducibility & scenario", @@ -280,6 +284,8 @@ "log.layout_reset": "Window layout reset.", "log.loaded_profile": "Loaded profile", "log.loop": "loop", + "log.loss_burst_clamped": "Loss cannot reach {asked}% in runs of {burst} packets, so this session will lose {delivered}%.", + "log.loss_burst_gap": "Loss will arrive in runs of about {burst} packets, roughly one run every {gap} packets.", "log.marker_needs_start": "Bug marker works after start (START).", "log.narrow_applied": "Capturing only the targeted traffic - the driver hands over your destination's traffic and nothing else, so the counters and the connection list cover that traffic only.", "log.narrow_no_effect": "\"Capture only the targeted traffic\" had no effect - this destination cannot be turned into a driver filter (a wildcard, an re: pattern, or no destination at all). Capturing everything, as usual.", @@ -400,6 +406,7 @@ "stats.flap_cut": "Link outage", "stats.lan_cut": "LAN: internet cut", "stats.local_cut": "Local network cut", + "stats.loss_runs": "Loss runs", "stats.mtu_dropped": "MTU dropped", "stats.nat_expired": "NAT expired", "stats.overflow": "Buffer overflow", @@ -430,6 +437,7 @@ "summary.lan": "LAN mode (no internet)", "summary.latency": "+{v} ms ping", "summary.loss": "{v}% loss", + "summary.loss_burst": "in runs of {v} packets", "summary.mtu": "MTU {v} B", "summary.nat": "NAT expires after {v} s", "summary.none": "Active: no impairments (link unchanged).", @@ -513,6 +521,7 @@ "tips.load_config": "Load all settings from a JSON file (same format as CLI: --config).", "tips.log_lines": "How many log lines to keep in the strip at the bottom. Older lines are dropped once the limit is reached.", "tips.loss": "Percent of packets that vanish without a trace. Even 5% feels like a dropping connection.", + "tips.loss_burst": "Makes the loss set beside it arrive in runs instead of one packet at a time, and the number is how many packets are lost in a row on average. A run of losses hurts a connection far more than the same total spread evenly, so keep it low unless you are testing how an application recovers.", "tips.mark_bug": "Inserts a timestamped marker into the log - click exactly when you see the bug.", "tips.match_syntax": "Filter syntax (the same in every field):\n 80,443 a list - any of them\n 1000-2000 a range, both ends included\n >1024 <=80 comparisons: > < >= <=\n !53 exclusion - everything except 53\n 8* chrome* wildcard (* and ?)\n re:^chrome regular expression\nIP also takes CIDR (192.168.1.0/24) and IPv6.\nTerms are combined: any positive term matches, every ! term excludes. A field with only exclusions means 'everything except those'. Empty = everything.", "tips.mtu": "Drop packets larger than the given number of bytes. Reproduces the 'MTU black hole' common in tunnels/VPN/behind NAT: small packets pass, large ones vanish. 0 = off.", @@ -549,6 +558,7 @@ "tips.stat_lan": "Packets to/from the internet dropped in LAN mode.", "tips.stat_local": "Packets to/from the local network dropped by \"Internet only\".", "tips.stat_loss": "Packets dropped because of the configured Loss. Link outages are counted separately, under Link outage.", + "tips.stat_loss_runs": "How many runs of lost packets this session produced. Zero with \"Losses in a row\" set means the session was too short to see one, not that nothing was configured.", "tips.stat_mtu": "Packets dropped as too large (MTU black hole).", "tips.stat_nat": "Packets dropped after the NAT mapping expired.", "tips.stat_overflow": "Packets dropped because the queue overflowed (heavy overload). This counter always covers ALL captured traffic, even when the view is narrowed to the target: these are packets the TOOL lost, and hiding the ones outside your target would hide its own damage.", diff --git a/lang/pl.json b/lang/pl.json index fa40f81..1586a3d 100644 --- a/lang/pl.json +++ b/lang/pl.json @@ -103,6 +103,8 @@ "dialogs.internal_error_title": "Błąd wewnętrzny", "dialogs.load_config": "Wczytaj konfigurację", "dialogs.load_scenario": "Wczytaj scenariusz", + "dialogs.loss_burst_help": "Prawdziwe łącza rzadko gubią pakiety po jednym. Mikrofalówka, winda albo przełączenie się między nadajnikami zabiera połączenie na moment, a wszystko wysłane w tym momencie przepada.\n\nTo pole mówi, ile pakietów ginie pod rząd średnio. Pole \"Utrata\" obok nadal decyduje, ile ginie w sumie, więc 5 procent zostaje 5 procentami. Po prostu przychodzi w kilku skupiskach zamiast równomiernie.\n\nDlaczego to ważne: 5 procent rozłożonych równo większość połączeń wchłania bez widocznego problemu. Te same 5 procent w seriach po dwadzieścia zatrzymuje transfer, zrywa połączenie na żywo i wysyła aplikację w ścieżkę ponownego łączenia, czyli zwykle w ten kod, który chciałeś przetestować.\n\nUstaw 0, żeby rozłożyć stratę równomiernie, czyli tak, jak to narzędzie działało, zanim to ustawienie powstało.\n\nDwa ograniczenia, o których warto wiedzieć. Bardzo wysoka strata nie zmieści się w bardzo krótkich seriach, bo tak krótkie serie zostawiają za mało miejsca między sobą, więc log mówi, ile sesja naprawdę zgubi. A długa seria stawia serie daleko od siebie, więc krótka sesja może nie zobaczyć żadnej. Log mówi, jak często można się ich spodziewać.\n\n\"Przerwy w łączu (flapping)\" niżej to co innego. Tamto zrywa połączenie w stałym cyklu i tak ma być. To jest losowe i krótkie, w środku normalnego ruchu.", + "dialogs.loss_burst_help_title": "Straty pod rząd", "dialogs.match_help": "Wyrażenie to lista członów po przecinku. Pozytywy łączy OR, \"!\" odejmuje.\n\n wartość pojedyncza wartość (goła nazwa procesu działa jak podciąg)\n a, b, c lista\n a-b zakres, obustronnie domknięty (80-88 = od 80 do 88)\n >N =N <=N porównania (tylko liczby)\n * ? wildcardy (chrome*, fire?ox)\n re:WZORZEC wyrażenie regularne (bez wielkości liter, przecinek escapuj \\,)\n 10.0.0.0/24 CIDR (tylko pola IP, IPv4 i IPv6)\n !wartość wykluczenie\n\nPrzykłady:\n chrome.exe, !chromedriver\n 80, 443, 8000-8100\n 10.0.0.1-10.0.0.50, !10.0.0.7\n re:^fire\n\nPuste pole = wszystko. Pole z samymi wykluczeniami = wszystko oprócz nich. Człon, którego nie da się ocenić dla danego pakietu (reguła portu wobec ICMP, reguła IPv4 wobec adresu IPv6) po prostu nie pasuje.", "dialogs.match_help_title": "Składnia wyrażeń filtrujących", "dialogs.missing_library": "Brak biblioteki", @@ -194,6 +196,7 @@ "fields.locked_running": "Zablokowane w trakcie sesji (STOP odblokuje).", "fields.loop": "Pętla", "fields.loss": "Utrata:", + "fields.loss_burst": "Straty pod rząd:", "fields.match_help": "?", "fields.max_size": "Maks. rozmiar (MTU):", "fields.narrow_filter": "Przechwytuj tylko ruch celu", @@ -223,6 +226,7 @@ "fields.target_no_match": "Żaden działający proces nie pasuje do tego celu - ruch NIE jest modyfikowany.", "fields.target_process": "Proces:", "fields.unit_b_off": "B (0=wył.)", + "fields.unit_pkt_off": "pakietów (0=wył.)", "fields.unit_rows": "wierszy", "fields.unit_s_off": "s (0=wył.)", "fields.upload": "Wysyłanie:", @@ -238,7 +242,7 @@ "frames.destination": "Celuj w adres docelowy", "frames.event_log": "Dziennik zdarzeń (ze znacznikami czasu)", "frames.flapping": "Przerwy w łączu (flapping)", - "frames.impairments": "Zakłócenia (%)", + "frames.impairments": "Zakłócenia", "frames.latency": "Opóźnienie (ping)", "frames.profiles": "Profile", "frames.repro": "Powtarzalność i scenariusz", @@ -280,6 +284,8 @@ "log.layout_reset": "Układ okna zresetowany.", "log.loaded_profile": "Wczytano profil", "log.loop": "pętla", + "log.loss_burst_clamped": "Strata nie osiągnie {asked}% przy seriach po {burst} pakietów, więc ta sesja zgubi {delivered}%.", + "log.loss_burst_gap": "Strata będzie przychodzić seriami po około {burst} pakietów, mniej więcej jedna seria na {gap} pakietów.", "log.marker_needs_start": "Znacznik błędu zadziała po uruchomieniu (START).", "log.narrow_applied": "Przechwytywanie tylko ruchu celu - sterownik podaje ruch z Twoim celem i nic więcej, więc liczniki i lista połączeń obejmują wyłącznie ten ruch.", "log.narrow_no_effect": "„Przechwytuj tylko ruch celu” nic nie dało - tego celu nie da się zamienić na filtr sterownika (wildcard, wzorzec re: albo brak celu). Przechwytywane jest wszystko, jak zwykle.", @@ -400,6 +406,7 @@ "stats.flap_cut": "Przerwa w łączu", "stats.lan_cut": "LAN: internet odcięty", "stats.local_cut": "Sieć lokalna odcięta", + "stats.loss_runs": "Serie strat", "stats.mtu_dropped": "MTU odrzucone", "stats.nat_expired": "NAT wygasło", "stats.overflow": "Bufor przepełn.", @@ -430,6 +437,7 @@ "summary.lan": "tryb LAN (bez internetu)", "summary.latency": "+{v} ms pingu", "summary.loss": "{v}% strat", + "summary.loss_burst": "seriami po {v} pakietów", "summary.mtu": "MTU {v} B", "summary.nat": "NAT wygasa po {v} s", "summary.none": "Aktywne: brak zakłóceń (łącze bez zmian).", @@ -513,6 +521,7 @@ "tips.load_config": "Wczytaj wszystkie ustawienia z pliku JSON (ten sam format co w CLI: --config).", "tips.log_lines": "Ile linii logu trzymać w pasku na dole. Starsze linie są usuwane po osiągnięciu limitu.", "tips.loss": "Procent pakietów, które znikają bez śladu. Już 5% to odczuwalnie zrywająca się sieć.", + "tips.loss_burst": "Sprawia, że strata ustawiona obok pojawia się seriami, a nie po jednym pakiecie, a liczba mówi, ile pakietów ginie pod rząd średnio. Seria strat szkodzi połączeniu dużo bardziej niż ta sama liczba rozłożona równomiernie, więc trzymaj ją nisko, chyba że sprawdzasz, jak aplikacja się podnosi.", "tips.mark_bug": "Wstawia do dziennika znacznik z czasem - kliknij dokładnie wtedy, gdy zobaczysz błąd.", "tips.match_syntax": "Składnia filtrów (taka sama w każdym polu):\n 80,443 lista - dowolna z wartości\n 1000-2000 zakres, oba końce włącznie\n >1024 <=80 porównania: > < >= <=\n !53 wykluczenie - wszystko oprócz 53\n 8* chrome* wildcard (* i ?)\n re:^chrome wyrażenie regularne\nIP przyjmuje też CIDR (192.168.1.0/24) oraz IPv6.\nCzłony łączą się: pasuje dowolny człon pozytywny, każdy człon z ! wyklucza. Pole z samymi wykluczeniami znaczy 'wszystko oprócz nich'. Puste = wszystko.", "tips.mtu": "Gub pakiety większe niż podana liczba bajtów. Odwzorowuje 'czarną dziurę MTU' typową w tunelach/VPN/za NAT: małe pakiety przechodzą, duże znikają. 0 = wyłączone.", @@ -549,6 +558,7 @@ "tips.stat_lan": "Pakiety do/od internetu odrzucone w trybie LAN.", "tips.stat_local": "Pakiety do/od sieci lokalnej odrzucone przez „Tylko internet”.", "tips.stat_loss": "Pakiety porzucone z powodu ustawionej Utraty. Przerwy w łączu mają własny licznik - Przerwa w łączu.", + "tips.stat_loss_runs": "Ile serii gubionych pakietów wyszło w tej sesji. Zero przy ustawionym polu \"Straty pod rząd\" znaczy, że sesja była za krótka, żeby zobaczyć choć jedną, a nie że nic nie było ustawione.", "tips.stat_mtu": "Pakiety odrzucone jako za duże (czarna dziura MTU).", "tips.stat_nat": "Pakiety odrzucone po wygaśnięciu mapowania NAT.", "tips.stat_overflow": "Pakiety porzucone, bo kolejka się przepełniła (silne przeciążenie). Ten licznik zawsze obejmuje CAŁY przechwycony ruch, nawet gdy widok jest zawężony do celu: to są pakiety zgubione przez NARZĘDZIE, a ukrycie tych spoza celu ukryłoby jego własne szkody.", diff --git a/lang/zh.json b/lang/zh.json index 7684f3a..f51a2e1 100644 --- a/lang/zh.json +++ b/lang/zh.json @@ -103,6 +103,8 @@ "dialogs.internal_error_title": "内部错误", "dialogs.load_config": "加载配置", "dialogs.load_scenario": "加载场景", + "dialogs.loss_burst_help": "真实链路很少一个一个地丢包。微波炉、电梯，或者在不同基站之间切换，都会让连接中断片刻，而这段时间里发出的数据全部消失。\n\n这个设置表示平均连续丢失多少个数据包。旁边的\"丢包\"仍然决定总共丢多少，所以 5% 依然是 5%，只是集中成几串出现，而不是均匀分散。\n\n为什么重要：均匀分散的 5% 丢包，大多数连接都能吸收，看不出明显问题。同样的 5% 如果以每串二十个的方式出现，就会让传输停滞、让实时连接断开，并促使应用走重连流程，而那通常正是你想测试的代码。\n\n设为 0 表示均匀分散丢包，也就是本工具在这个设置出现之前的行为。\n\n有两点限制值得了解。很高的丢包率无法以很短的串出现，因为那么短的串之间留不下足够的间隔，所以日志会告诉你本次会话实际会丢多少。另外，串越长，两串之间的间隔就越远，短时间的会话可能一串都遇不到。日志会说明大致的出现频率。\n\n下方的\"链路中断（周期断线）\"是另一回事，它按固定周期切断连接，这是刻意设计的。而这里是随机且短暂的，发生在原本正常的流量之中。", + "dialogs.loss_burst_help_title": "连续丢包", "dialogs.match_help": "表达式由逗号分隔的多个项目组成。正向项目按“或”匹配，“!”用于排除。\n\n value 单个值（直接填写进程名时按子字符串匹配）\n a, b, c 列表\n a-b 范围，包含两端（80-88 表示 80 到 88）\n >N =N <=N 数值比较（仅限数字）\n * ? 通配符（chrome*、fire?ox）\n re:PATTERN 正则表达式（不区分大小写，模式内的逗号要写成 \\,）\n 10.0.0.0/24 CIDR（仅适用于 IP 字段，支持 IPv4 和 IPv6）\n !value 排除\n\n示例：\n chrome.exe, !chromedriver\n 80, 443, 8000-8100\n 10.0.0.1-10.0.0.50, !10.0.0.7\n re:^fire\n\n空字段表示“全部”。只有排除项的字段表示“除这些之外的全部”。若某个条件无法用于某个数据包（例如用端口规则匹配 ICMP，或用 IPv4 规则匹配 IPv6 地址），该条件只会视为不匹配。", "dialogs.match_help_title": "过滤表达式语法", "dialogs.missing_library": "缺少库", @@ -194,6 +196,7 @@ "fields.locked_running": "会话运行期间已锁定（停止后解锁）。", "fields.loop": "循环", "fields.loss": "丢包:", + "fields.loss_burst": "连续丢包:", "fields.match_help": "?", "fields.max_size": "最大大小（MTU）:", "fields.narrow_filter": "仅捕获目标流量", @@ -223,6 +226,7 @@ "fields.target_no_match": "没有正在运行的进程符合此目标。流量当前不会受到弱网影响。", "fields.target_process": "进程:", "fields.unit_b_off": "B（0=关闭）", + "fields.unit_pkt_off": "个数据包（0=关闭）", "fields.unit_rows": "行", "fields.unit_s_off": "秒（0=关闭）", "fields.upload": "上传:", @@ -238,7 +242,7 @@ "frames.destination": "目标地址筛选", "frames.event_log": "事件日志（带时间戳）", "frames.flapping": "链路中断（周期断线）", - "frames.impairments": "弱网效果（%）", + "frames.impairments": "弱网效果", "frames.latency": "延迟（Ping）", "frames.profiles": "配置方案", "frames.repro": "可复现性与场景", @@ -280,6 +284,8 @@ "log.layout_reset": "窗口布局已重置。", "log.loaded_profile": "已加载配置方案", "log.loop": "循环", + "log.loss_burst_clamped": "在每串 {burst} 个数据包的情况下，丢包率无法达到 {asked}%，因此本次会话将丢失 {delivered}%。", + "log.loss_burst_gap": "丢包将以每串约 {burst} 个数据包的方式出现，大约每 {gap} 个数据包出现一串。", "log.marker_needs_start": "启动会话后才能添加故障标记。", "log.narrow_applied": "当前仅捕获目标流量：驱动只会把指定目标地址的流量交给本工具，因此计数器和连接列表也只覆盖这些流量。", "log.narrow_no_effect": "“仅捕获目标流量”未生效：此目标地址无法转换为驱动过滤器（使用了通配符、re: 正则表达式，或根本未设置目标地址）。将照常捕获全部流量。", @@ -400,6 +406,7 @@ "stats.flap_cut": "链路中断", "stats.lan_cut": "局域网模式：互联网被切断", "stats.local_cut": "本地网络被切断", + "stats.loss_runs": "丢包串数", "stats.mtu_dropped": "因 MTU 丢弃", "stats.nat_expired": "NAT 已过期", "stats.overflow": "缓冲区溢出", @@ -430,6 +437,7 @@ "summary.lan": "局域网模式（无互联网）", "summary.latency": "延迟 +{v} ms", "summary.loss": "{v}% 丢包", + "summary.loss_burst": "以每串 {v} 个数据包出现", "summary.mtu": "MTU {v} B", "summary.nat": "NAT 在 {v} 秒后过期", "summary.none": "已启用：无弱网效果（链路保持不变）。", @@ -513,6 +521,7 @@ "tips.load_config": "从 JSON 文件加载全部设置，格式与 CLI 的 --config 相同。", "tips.log_lines": "底部日志区域最多保留的行数。达到上限后，更旧的行会被移除。", "tips.loss": "让指定百分比的数据包无声消失。即使只有 5%，连接也会明显不稳定。", + "tips.loss_burst": "让旁边设置的丢包成串出现，而不是一个一个地丢，数值表示平均连续丢失多少个数据包。同样的丢包总量，成串出现对连接的影响远大于均匀分布，因此除非你要测试应用的恢复能力，否则请保持较小的数值。", "tips.mark_bug": "在日志中插入带时间戳的标记。看到故障的准确时刻立即点击。", "tips.match_syntax": "过滤语法（所有字段通用）：\n 80,443 列表，匹配其中任意一个\n 1000-2000 范围，包含两端\n >1024 <=80 比较：> < >= <=\n !53 排除，匹配除 53 之外的全部\n 8* chrome* 通配符（* 和 ?）\n re:^chrome 正则表达式\nIP 字段还支持 CIDR（192.168.1.0/24）和 IPv6。\n多个项目的组合规则：任意正向项目匹配即可，每个“!”项目都会排除。只有排除项的字段表示“除这些之外的全部”。空字段 = 全部。", "tips.mtu": "丢弃大于指定字节数的数据包，用于复现隧道、VPN 或 NAT 后常见的“MTU 黑洞”：小包可通过，大包直接消失。0 = 关闭。", @@ -549,6 +558,7 @@ "tips.stat_lan": "在局域网模式下，被丢弃的互联网数据包。", "tips.stat_local": "因“仅互联网”模式而被丢弃的本地网络数据包。", "tips.stat_loss": "因配置的“丢包”效果而被丢弃的数据包。链路中断会在“链路中断”中单独统计。", + "tips.stat_loss_runs": "本次会话产生了多少串连续丢包。如果设置了\"连续丢包\"却显示 0，说明会话太短，还没有出现一串，而不是没有设置。", "tips.stat_mtu": "因超过 MTU（MTU 黑洞）而被丢弃的数据包。", "tips.stat_nat": "NAT 映射过期后被丢弃的数据包。", "tips.stat_overflow": "因队列溢出（严重过载）而被丢弃的数据包。即使视图已收窄到目标，此计数器也始终覆盖所有已捕获流量，因为这些数据包是本工具丢失的。隐藏目标之外的部分会掩盖工具自身造成的损害。", diff --git a/tests/test_burst_loss.py b/tests/test_burst_loss.py new file mode 100644 index 0000000..75bcbeb --- /dev/null +++ b/tests/test_burst_loss.py @@ -0,0 +1,367 @@ +"""Loss that arrives in RUNS, not one packet at a time: the two-state chain. + +Real links do not lose packets one at a time. A microwave oven, a lift, a +handover between transmitters - each takes the link away for a stretch, and 5% +of loss spread evenly is something TCP shrugs off while 5% arriving in three +runs of twenty collapses the window and drives the reconnect path, which is the +code a tester actually wants to exercise. ``beantester.core.burst_loss_params`` +turns "this much loss, in runs this long" into the transition probabilities of +Gilbert's two-state channel, and ``BeanCore._loses`` walks it. + +What these tests are here to hold down, in the order it would hurt to lose it: + +* **the default is byte for byte what it was.** With no run length set, step 8 + makes the same single draw it always made, in the same place in the RNG + sequence - otherwise every stored ``Reproduce:`` command would replay into a + different session. +* **the numbers mean what they say.** The delivered loss lands on the field that + asked for it, and a run is as long as the field that asked for it - including + under two-way traffic, which is where a single shared chain would silently + deliver half. +* **the impossible corner is loud, not quiet.** Some loss/length pairs cannot + exist. They are clamped, and ``achievable`` reports what the pair really does. +* **the chain is re-derived from BOTH of its inputs.** ``p`` depends on the loss + as well as the run length, so changing only the loss has to move it. +""" +import random + +from fakes import check + +from beantester.core import BeanCore, burst_loss_params + +PACKETS = 200000 + + +def _core(loss=0.0, burst=0.0): + core = BeanCore() + core.set_params(loss, 0, 0, 0, 0, 0, 0) + core.set_loss_burst(burst) + core.reset_buckets(0.0) + return core + + +def _drops(core, packets=PACKETS, seed=7, alternate=False): + """Run packets through decide(); return (dropped, {direction: [run lengths]}). + + ``alternate`` sends every other packet the other way, which is what the + default two-way traffic filter hands the engine. + """ + rng = random.Random(seed) + runs = {True: [], False: []} + current = {True: 0, False: 0} + dropped = 0 + for i in range(packets): + outbound = bool(i % 2) if alternate else True + decision = core.decide(1200, outbound, 5000, i * 0.001, rng, + remote_ip="1.2.3.4", remote_port=443, is_tcp=True) + if decision.drop: + dropped += 1 + current[outbound] += 1 + elif current[outbound]: + runs[outbound].append(current[outbound]) + current[outbound] = 0 + for direction, length in current.items(): + if length: + runs[direction].append(length) + return dropped, runs + + +def _mean(values): + return sum(values) / len(values) if values else 0.0 + + +# --------------------------------------------------------------------------- # +# The arithmetic, on its own - it is a pure function and it is where the model +# lives, so it is worth pinning without an engine anywhere near it. +# --------------------------------------------------------------------------- # +def test_no_run_length_means_the_independent_draw(): + for loss, burst in ((0.05, 0.0), (0.05, 1.0), (0.05, 0.5), (0.0, 20.0)): + check(f"loss={loss} burst={burst} stays independent", + burst_loss_params(loss, burst) is None, + f"(got {burst_loss_params(loss, burst)})") + + +def test_the_transition_probabilities_follow_the_two_fields(): + p, r, achievable = burst_loss_params(0.05, 20.0) + check("r is one over the mean run length", abs(r - 0.05) < 1e-12, f"(r={r})") + # p = loss * r / (1 - loss) = 0.05 * 0.05 / 0.95 + check("p is derived from the loss and r", abs(p - (0.05 * 0.05 / 0.95)) < 1e-12, + f"(p={p})") + check("an achievable pair delivers exactly what was asked", + achievable == 0.05, f"(achievable={achievable})") + + +def test_a_pair_that_cannot_exist_is_clamped_and_says_so(): + """90% loss cannot arrive in runs of five - there is no room between them.""" + p, r, achievable = burst_loss_params(0.9, 5.0) + check("the good state is squeezed to a single packet", p == 1.0, f"(p={p})") + # the most a run length of 5 can carry is (1/r) / (1/r + 1) = 1 / (1 + r) + check("achievable reports the most that run length can carry", + abs(achievable - 1.0 / 1.2) < 1e-12, f"(achievable={achievable})") + check("and it is BELOW what was asked for", achievable < 0.9, + f"(achievable={achievable})") + + +def test_total_loss_does_not_divide_by_zero(): + """``1 - loss`` is the denominator, so 100% is the input that would raise.""" + p, r, achievable = burst_loss_params(1.0, 5.0) + check("total loss keeps the chain in the bad state", p == 1.0, f"(p={p})") + check("and reports total loss", achievable == 1.0, f"(achievable={achievable})") + check("r is still the run length", abs(r - 0.2) < 1e-12, f"(r={r})") + + +# --------------------------------------------------------------------------- # +# The packet path +# --------------------------------------------------------------------------- # +def test_the_default_path_is_the_draw_it_has_always_been(): + """A core that never heard of run lengths and one told 0 must agree exactly. + + Not "roughly": the same seed has to produce the same decision at the same + index, because a stored reproduction command replays a session by seed and + the number of draws step 8 makes is part of that sequence. + """ + untouched = BeanCore() + untouched.set_params(10, 0, 0, 0, 0, 0, 0) + untouched.reset_buckets(0.0) + told_zero = _core(loss=10, burst=0) + + a, _ = _drops(untouched, packets=20000) + b, _ = _drops(told_zero, packets=20000) + check("the same seed drops the same packets with and without the feature", + a == b, f"(untouched={a}, told zero={b})") + check("and it really was dropping something", a > 0, f"(dropped={a})") + + +def test_no_loss_means_the_chain_is_never_consulted(): + """Step 8's left half short-circuits, so a run length alone damages nothing. + + This is also why the pass-through path pays nothing for this feature: with + no loss configured ``_loses`` is not called at all. + """ + core = _core(loss=0, burst=20) + dropped, _ = _drops(core, packets=20000) + check("a run length with no loss drops nothing", dropped == 0, + f"(dropped={dropped})") + + +def test_the_delivered_loss_matches_the_field_that_asked_for_it(): + """The tolerances here are MEASURED, and this comment is where they come from. + + Burst loss is far noisier per packet than independent loss, because the unit + of randomness is the RUN, not the packet: 200 000 packets at 5% in runs of 20 + contain only about 500 runs. Measured over twelve seeds at this size the + delivered loss spread 4.715% to 5.843% (sd 0.29) and the mean run length + 18.71 to 22.06 (sd 0.78). Pushed to 2 million packets the mean converges on + the model - 4.9765% against a stationary 5.0000% - so the spread is the + sample, not a bias. + + The seeds are fixed, so nothing here can flake: a tolerance is only wide + enough that the test pins the RULE instead of one sample, and tight enough + that a fifth of the loss going missing still turns it red. + """ + delivered = [] + for seed in (7, 11, 23): + dropped, runs = _drops(_core(loss=5, burst=20), seed=seed) + share = 100.0 * dropped / PACKETS + delivered.append(share) + check(f"seed {seed}: 5% asked for in runs of 20 delivers about 5%", + abs(share - 5.0) <= 1.2, f"(delivered {share:.3f}%)") + check(f"seed {seed}: the runs are about as long as the field asked", + abs(_mean(runs[True]) - 20.0) <= 3.0, + f"(mean run {_mean(runs[True]):.2f})") + check("and across seeds the average lands on the number that was asked for", + abs(_mean(delivered) - 5.0) <= 0.8, + f"(mean of {[f'{x:.3f}' for x in delivered]})") + + +def test_the_losses_actually_arrive_in_runs(): + """The whole point: consecutive drops, not the same total sprinkled evenly. + + Compared against the independent draw at the SAME loss rate, which is the + honest control - a run length of 20 has to move the shape, not just the + total. + """ + bursty, _ = _drops(_core(loss=5, burst=20), packets=50000) + _, even_runs = _drops(_core(loss=5, burst=0), packets=50000) + _, bursty_runs = _drops(_core(loss=5, burst=20), packets=50000) + check("independent loss almost never repeats", + _mean(even_runs[True]) < 1.2, f"(mean run {_mean(even_runs[True]):.2f})") + check("burst loss does, by more than an order of magnitude", + _mean(bursty_runs[True]) > 10 * _mean(even_runs[True]), + f"(bursty {_mean(bursty_runs[True]):.2f} vs even " + f"{_mean(even_runs[True]):.2f})") + check("and the longest run is long enough to matter to a TCP window", + max(bursty_runs[True]) >= 20, f"(longest {max(bursty_runs[True])})") + check("while the total is still what was asked", abs(bursty / 50000 - 0.05) < 0.01, + f"(delivered {100.0 * bursty / 50000:.2f}%)") + + +def test_each_direction_gets_a_run_of_the_length_that_was_asked_for(): + """One chain per direction, and this is the test that says why. + + With the default two-way filter a SHARED chain splits every run across both + directions, so each side would see about half of the configured length - + measured 10.4 at a 50/50 mix. Worse, the error follows the traffic mix + rather than being a constant a reader could correct for. Anyone folding the + two chains back into one turns this red. + + Measured over twelve seeds with the traffic split evenly: outbound runs + averaged 19.15 to 22.06 and inbound 17.43 to 22.85 (the inbound half is + noisier because it carries half the packets and so half the runs). The + assertion is therefore the RULE - each direction is nowhere near half - with + a floor well below anything observed, plus a tighter check on both + directions pooled. + """ + core = _core(loss=5, burst=20) + dropped, runs = _drops(core, alternate=True) + for direction, label in ((True, "outbound"), (False, "inbound")): + mean = _mean(runs[direction]) + check(f"{label} sees a full-length run, not the half a shared chain gives", + mean >= 15.0, f"(mean run {mean:.2f}, a shared chain measured 10.4)") + pooled = _mean(runs[True] + runs[False]) + check("pooled, the runs are the length that was configured", + abs(pooled - 20.0) <= 3.0, f"(mean run {pooled:.2f})") + delivered = 100.0 * dropped / PACKETS + check("and the total loss is still the number that was asked for", + abs(delivered - 5.0) <= 1.2, f"(delivered {delivered:.3f}%)") + + +def test_an_impossible_pair_delivers_what_it_promised_to_deliver(): + """The clamp is not a silent cap: what comes out is what ``achievable`` said.""" + _p, _r, achievable = burst_loss_params(0.9, 5.0) + core = _core(loss=90, burst=5) + dropped, _ = _drops(core) + delivered = dropped / PACKETS + check("the run delivers the clamped figure, not the one that was typed", + abs(delivered - achievable) < 0.01, + f"(delivered {100 * delivered:.2f}%, achievable {100 * achievable:.2f}%)") + + +# --------------------------------------------------------------------------- # +# State: what happens between packets +# --------------------------------------------------------------------------- # +def test_changing_only_the_loss_re_derives_the_chain(): + """``p`` depends on BOTH fields, so the loss setter has to move it too. + + Without this the delivered loss would drift away from the field that asked + for it the moment a session changed its loss without touching the run + length - and nothing else in the suite would have noticed, because the + chain would still be a perfectly valid chain for the OLD number. + """ + core = _core(loss=5, burst=20) + before = core._burst_p + core.set_params(20, 0, 0, 0, 0, 0, 0) + check("a new loss gives a new transition probability", core._burst_p != before, + f"(p stayed {before})") + expected, _r, _a = burst_loss_params(0.2, 20.0) + check("and it is the one the new loss implies", + abs(core._burst_p - expected) < 1e-12, + f"(p={core._burst_p}, expected {expected})") + + +def test_re_applying_the_same_settings_does_not_cut_a_run_in_flight(): + """A scenario stepping an unrelated field calls every setter every time. + + Restarting the chain there would cut most runs short: at 50 packets a second + a run of 20 lasts 400 ms, which is longer than a scenario step. + """ + core = _core(loss=50, burst=50) + rng = random.Random(3) + for i in range(200): # get the chain into a bad run + core.decide(1200, True, 5000, i * 0.001, rng, remote_ip="1.2.3.4", + remote_port=443, is_tcp=True) + if core._loss_bad[True]: + break + check("the chain reached a bad run to test with", core._loss_bad[True], "") + + core.set_loss_burst(50) # same value, applied again + check("re-applying the same run length leaves the run alone", + core._loss_bad[True], "(the run was cut)") + core.set_params(50, 0, 0, 0, 0, 0, 0) # same loss, applied again + check("re-applying the same loss leaves the run alone", + core._loss_bad[True], "(the run was cut)") + + core.set_loss_burst(10) # a REAL change + check("a real change does restart the chain", not core._loss_bad[True], + "(the run survived a change it should not have)") + + +def test_a_session_never_starts_inside_a_run(): + """``reset_buckets`` runs at every start, exactly like the token buckets.""" + core = _core(loss=90, burst=50) + rng = random.Random(5) + for i in range(200): + core.decide(1200, True, 5000, i * 0.001, rng, remote_ip="1.2.3.4", + remote_port=443, is_tcp=True) + core._loss_bad[True] = core._loss_bad[False] = True + core.reset_buckets(10.0) + check("both directions start the session in the good state", + not core._loss_bad[True] and not core._loss_bad[False], + f"({core._loss_bad})") + + +def test_the_run_counter_answers_did_this_fire_at_all(): + """A drop count cannot say whether the MODEL did anything. + + 0.1% loss in runs of 1000 is one run per million packets, which on a quiet + connection is hours: the settings look reasonable, nothing happens, and that + reads exactly like a broken tool. So the runs are counted, and the count is + the difference between "too short a session" and "this is not working". + """ + core = _core(loss=5, burst=20) + check("a fresh core has counted nothing", core.loss_bursts == 0, + f"({core.loss_bursts})") + dropped, runs = _drops(core, packets=50000) + started = len(runs[True]) + len(runs[False]) + check("the counter matches the runs actually observed", + abs(core.loss_bursts - started) <= 1, + f"(counted {core.loss_bursts}, observed {started})") + check("and it counted something at all", core.loss_bursts > 0, + f"({core.loss_bursts} with {dropped} packets dropped)") + + # The one packet of slack above is real and worth naming: a run still in + # progress when the window ends has been STARTED but not yet observed as + # finished, so the two can differ by exactly one. + quiet = _core(loss=5, burst=0) + _drops(quiet, packets=20000) + check("independent loss starts no runs at all", quiet.loss_bursts == 0, + f"({quiet.loss_bursts})") + + +def test_the_run_counter_reaches_the_statistics_snapshot(): + """It is counted by the core and read by the engine, so the wiring is a + separate question from the counting - and this is the half a scenario, the + CSV, the repro report and the tile all depend on.""" + from beantester.engine import BeanEngine + + engine = BeanEngine() + engine.set_params(50, 0, 0, 0, 0, 0, 0) + engine.set_loss_burst(10) + engine.core.reset_buckets(0.0) + check("a stopped engine reports the counter at zero", + engine.stats_snapshot()["loss_bursts"] == 0, + f"({engine.stats_snapshot()['loss_bursts']})") + + rng = random.Random(4) + for i in range(5000): + engine.core.decide(1200, bool(i % 2), 5000, i * 0.001, rng, + remote_ip="1.2.3.4", remote_port=443, is_tcp=True) + snapshot = engine.stats_snapshot() + check("the engine reports what the core counted", + snapshot["loss_bursts"] == engine.core.loss_bursts, + f"(snapshot {snapshot['loss_bursts']}, core {engine.core.loss_bursts})") + check("and it is not zero after a run of impaired traffic", + snapshot["loss_bursts"] > 0, f"({snapshot['loss_bursts']})") + + +def test_turning_bursts_off_returns_to_the_independent_draw(): + core = _core(loss=10, burst=30) + check("armed", core._burst_p is not None, "") + core.set_loss_burst(0) + check("disarmed", core._burst_p is None, f"(p={core._burst_p})") + dropped, runs = _drops(core, packets=50000) + check("and the losses stop clustering", _mean(runs[True]) < 1.2, + f"(mean run {_mean(runs[True]):.2f})") + check("while still losing about the configured share", + abs(100.0 * dropped / 50000 - 10.0) < 1.0, + f"(delivered {100.0 * dropped / 50000:.2f}%)") diff --git a/tests/test_cli_runtime.py b/tests/test_cli_runtime.py index b1f477a..1b4287e 100644 --- a/tests/test_cli_runtime.py +++ b/tests/test_cli_runtime.py @@ -213,6 +213,7 @@ def __init__(self, flip_at=None, **stats): def set_seed(self, *_a, **_k): pass def set_params(self, *_a, **_k): pass def set_buffer(self, *_a, **_k): pass + def set_loss_burst(self, *_a, **_k): pass def set_dest(self, *_a, **_k): pass def set_ip_family(self, *_a, **_k): pass def set_lan(self, *_a, **_k): pass @@ -368,30 +369,18 @@ def test_exit_code_runtime_without_pydivert(): """ _needs_permission_to_answer("a capture that cannot start") - class _CannotStartEngine: - fault = False + class _CannotStartEngine(_TargetedEngine): + """The same engine surface, with the ONE method that has to fail. - def set_seed(self, *_a, **_k): pass - def set_params(self, *_a, **_k): pass - def set_buffer(self, *_a, **_k): pass - def set_dest(self, *_a, **_k): pass - def set_ip_family(self, *_a, **_k): pass - def set_lan(self, *_a, **_k): pass - def set_internet_only(self, *_a, **_k): pass - def set_block(self, *_a, **_k): pass - def set_advanced(self, *_a, **_k): pass - def set_spike(self, *_a, **_k): pass - def set_nat(self, *_a, **_k): pass - def set_rst(self, *_a, **_k): pass - def set_flap(self, *_a, **_k): pass - def set_schedule(self, *_a, **_k): pass - def set_target(self, *_a, **_k): pass + This used to mirror every setter by hand. A second copy of an interface + is the copy that falls behind, and it did: the engine gaining a setter + turned this test red for a reason that has nothing to do with what it + asserts. ``_OneTickEngine`` already subclasses for the same reason. + """ def start(self, *_a, **_k): raise RuntimeError("WinDivert could not be opened") - def stop(self, *_a, **_k): pass - out, err = io.StringIO(), io.StringIO() clock = FakeClock() code = run_cli(["--loss", "5", "--duration", "1"], sleep=clock.sleep, @@ -1078,6 +1067,34 @@ def test_the_lan_abbreviation_still_reaches_lan_mode(monkeypatch): args.internet_only is False) +def test_the_loss_flag_survived_gaining_a_neighbour(monkeypatch): + """🔴 MEASURED, and the cost `--loss-burst` was allowed to charge. + + ``allow_abbrev`` is on by decision (ADR 2026-08-02), so a second option + starting with ``loss`` changes what the prefixes mean. Measured on this + parser: ``--loss`` still resolves, because argparse prefers an EXACT match + over a prefix one, and ``--loss-b`` reaches the new flag. What did NOT + survive is ``--los``, which used to work and is now ambiguous - a real if + small cost, accepted deliberately rather than discovered later, and pinned + here so nobody spends an afternoon on it as a bug. + """ + parser = cli_module.build_arg_parser() + args = parser.parse_args(["--loss", "5"]) + check("--loss still means the loss percentage", args.loss == 5.0, f"({args.loss})") + check("and it did not swallow the run length", args.loss_burst is None, + f"({args.loss_burst})") + args = parser.parse_args(["--loss-b", "20"]) + check("--loss-b reaches the run length", args.loss_burst == 20.0, + f"({args.loss_burst})") + try: + parser.parse_args(["--los", "5"]) + code = 0 + except SystemExit as exc: + code = exc.code + check("--los is now ambiguous, which is the accepted cost of the name", + code == 2, f"(exit {code})") + + def test_a_bounded_run_is_not_warned_about(monkeypatch): """Three ways to bound a run, and each one has to buy silence. diff --git a/tests/test_field_registry.py b/tests/test_field_registry.py index 0da8423..9dba876 100644 --- a/tests/test_field_registry.py +++ b/tests/test_field_registry.py @@ -172,7 +172,8 @@ def test_numeric_fields_declare_bounds(): def test_profile_scope_is_derived(): check("registry: a profile stores the link-characteristic fields", - set(F.PROFILE_FIELDS) == {"loss", "corrupt", "dup", "latency", "jitter", + set(F.PROFILE_FIELDS) == {"loss", "loss_burst", "corrupt", "dup", + "latency", "jitter", "down", "up", "buffer", "spike_prob", "spike_ms", "flap_period", "flap_down"}, f"({F.PROFILE_FIELDS})") diff --git a/tests/test_gui_layout.py b/tests/test_gui_layout.py index 279fc6e..e3445d2 100644 --- a/tests/test_gui_layout.py +++ b/tests/test_gui_layout.py @@ -276,11 +276,22 @@ def test_every_page_is_registered_and_built(): def test_fields_with_a_help_sheet_get_the_question_mark_button(): - """Filter-expression fields share the syntax cheat sheet; buffer has its own - "?" sheet (help_body). The schedule field must NOT grow one.""" + """Filter-expression fields share the syntax cheat sheet, and a field that + declares ``help_body`` gets its own "?" sheet. The schedule must NOT grow one. + + Derived from the registry rather than listed here: the set used to be six + names typed out, so adding a field with a help sheet meant editing this line + as well as the registry - and the one that gets forgotten is always the test. + The expression half stays explicit because those five share ONE sheet for a + reason unrelated to any ``help_body``. + """ run_gui(""" - assert set(app.form.helps) == {"target", "dst_ip", "dst_port", "block_ip", "block_port", "buffer"}, app.form.helps + from beantester.fields import FIELD_DEFS + expression_help = {"target", "dst_ip", "dst_port", "block_ip", "block_port"} + own_sheet = {f.key for f in FIELD_DEFS if f.help_body} + assert set(app.form.helps) == expression_help | own_sheet, app.form.helps assert "rate_schedule" not in app.form.helps + assert own_sheet, "no field declares a help sheet any more" """) diff --git a/tests/test_mutation_registry.py b/tests/test_mutation_registry.py index 490a8be..7802c5a 100644 --- a/tests/test_mutation_registry.py +++ b/tests/test_mutation_registry.py @@ -1619,6 +1619,59 @@ "new": ' return ""', "test": "test_clearing_the_search_puts_every_style_back", }, + { + # The measured reason there are two chains: one shared chain splits every + # run across both directions, so each side sees about half the length the + # user typed - and the error follows the traffic mix, so there is not even + # a constant anyone could correct for. + "label": "burst loss: the two run chains collapse into one shared flag", + "file": "beantester/core.py", + "old": " self._loss_bad[is_outbound] = bad\n return bad", + "new": " self._loss_bad[True] = bad\n return bad", + "test": "test_each_direction_gets_a_run_of_the_length_that_was_asked_for", + }, + { + # The bug the second analysis caught before it shipped: `p` depends on the + # loss as much as on the run length, so leaving this to `set_loss_burst` + # alone would let the ORDER of two setter calls decide correctness. + "label": "burst loss: changing only the loss leaves the chain stale", + "file": "beantester/core.py", + "old": " self.rate_up = self._rate_bps(up_kbps)\n" + " # The burst chain is derived from the loss AND from the run length,\n" + " # so it has to be re-derived here too - see _recompute_burst.\n" + " self._recompute_burst()", + "new": " self.rate_up = self._rate_bps(up_kbps)", + "test": "test_changing_only_the_loss_re_derives_the_chain", + }, + { + # A clamp that reports the number it was ASKED for is the silent lie this + # whole feature was built to avoid: the session would deliver 83% while + # every surface said 90%. + "label": "burst loss: an impossible pair claims to deliver what was asked", + "file": "beantester/core.py", + "old": " return (1.0, r, 1.0 / (1.0 + r))", + "new": " return (1.0, r, loss)", + "test": "test_a_pair_that_cannot_exist_is_clamped_and_says_so", + }, + { + # Restarting a session inside a run means the first packets of the next + # one vanish for a reason belonging to the previous session. + "label": "burst loss: a new session starts inside the previous run", + "file": "beantester/core.py", + "old": " self._loss_bad[True] = self._loss_bad[False] = False\n" + " self.loss_bursts = 0", + "new": " self.loss_bursts = 0", + "test": "test_a_session_never_starts_inside_a_run", + }, + { + # The counter exists to tell "too short a session" from "this is not + # working", so a counter stuck at zero is worse than none at all. + "label": "burst loss: the run counter never counts", + "file": "beantester/core.py", + "old": " bad = True\n self.loss_bursts += 1", + "new": " bad = True", + "test": "test_the_run_counter_answers_did_this_fire_at_all", + }, ] # The runner's own check: a patch that cannot compile must be reported as BROKEN, not diff --git a/tests/test_passthrough.py b/tests/test_passthrough.py index 9b40509..58f8790 100644 --- a/tests/test_passthrough.py +++ b/tests/test_passthrough.py @@ -30,7 +30,8 @@ from beantester.core import BeanCore, Decision from beantester.engine import BeanEngine -from beantester.fields import (FIELDS, IMPAIRING_KEYS, NARROWING_KEYS, off_value) +from beantester.fields import (FIELDS, IMPAIRING_KEYS, NARROWING_KEYS, + PARAMETER_KEYS, off_value) from beantester.presets import PRESETS, SETTING_TO_PRESET, preset_to_settings from beantester.settings import DEFAULT_SETTINGS, apply_settings from beantester.synthetic import SyntheticDivert @@ -45,14 +46,26 @@ # had to be remembered in two places - and the one that gets forgotten is always # the test, so the damage ships looking harmless. # -# The two additions are PARAMETERS, not triggers: `spike_ms` and `flap_down` arm -# nothing on their own (they sit behind `spike_prob` and `flap_period`), so the -# registry rightly does not call them impairments - but a default that shipped -# with either one hot would still be a default nobody chose, and pass-through is -# the one place that should insist on the whole form being cold. +# The additions are PARAMETERS, not triggers: each of them sits behind another +# field's gate in `decide()` and arms nothing on its own, so the registry rightly +# does not call them impairments (`Field.parameter_of` names their trigger). They +# are swept anyway, because a default that shipped with any of them hot would +# still be a default nobody chose, and pass-through is the one place that should +# insist on the whole form being cold. +# +# 🔴 This dict cannot be derived outright, and saying why is cheaper than +# rediscovering it: a registry knows that `rst_cooldown` parametrises `rst_prob`, +# but not that `spike_ms` ought to ship at 0 while `rst_cooldown` ought to ship at +# 3 s. `rst_cooldown` is therefore the one parameter deliberately outside this +# sweep - 0 there means "no cooldown at all", a different setting rather than a +# cold one. What IS now derived is the bookkeeping: a parameter that ought to be +# here and is not turns +# ``test_every_cold_parameter_is_in_this_sweep`` red, so the list can no longer +# quietly fall behind the registry the way it did while it was two names typed by +# hand. IMPAIRMENT_OFF = dict( {key: off_value(FIELDS[key]) for key in IMPAIRING_KEYS + NARROWING_KEYS}, - spike_ms=0, flap_down=0, + spike_ms=0, flap_down=0, loss_burst=0, ) # The profile fields that can impair traffic, and their "no impairment" value. @@ -116,6 +129,64 @@ def test_the_perfect_preset_is_completely_harmless(): perfect[key] == off, f"(is {perfect[key]!r})") +def test_every_cold_parameter_is_in_this_sweep(): + """Bookkeeping between the registry and the hand-written half of the sweep. + + ``IMPAIRMENT_OFF`` derives its impairments and its bounds from the registry + and then names a few PARAMETERS by hand, because no registry can know which + of them ought to ship at zero (see the comment there). That hand half is + exactly the shape that falls behind, so it is checked rather than trusted: a + parameter whose neutral value IS the registry's off value has to be swept, + and one that ships at something else is a deliberate choice this test makes + visible instead of silent. + """ + swept, deliberate = [], [] + for key in PARAMETER_KEYS: + (swept if key in IMPAIRMENT_OFF else deliberate).append(key) + if key in IMPAIRMENT_OFF: + check(f"the sweep uses {key}'s registry off value", + IMPAIRMENT_OFF[key] == off_value(FIELDS[key]), + f"(sweeps {IMPAIRMENT_OFF[key]!r}, registry says " + f"{off_value(FIELDS[key])!r})") + for key in deliberate: + check(f"{key} is outside the sweep because it ships at a chosen value, " + f"not because somebody forgot it", + DEFAULT_SETTINGS[key] != off_value(FIELDS[key]), + f"(default {DEFAULT_SETTINGS[key]!r} equals the off value, so it " + f"belongs in IMPAIRMENT_OFF)") + check("the registry knows about some parameters at all " + "(an empty sweep would pass every check above)", + len(swept) >= 2, f"(swept: {swept})") + + +def test_a_parameter_at_its_maximum_still_damages_nothing(): + """The property behind ``Field.parameter_of``, and it is the strong one. + + A parameter shapes an impairment, it does not arm one. Turning every one of + them up to its registry maximum with the rest of the form at its defaults + must therefore still pass every packet through untouched. This says far more + than "their defaults are zero": it would catch a parameter wired so that it + acts on its own - the mistake that would make ``impairs=""`` a lie and would + hide a run's real blast radius from ``settings.unbounded_impairment``. + """ + for key in PARAMETER_KEYS: + field = FIELDS[key] + settings = dict(DEFAULT_SETTINGS) + settings[key] = field.bounds[1] + core = core_for(settings) + rng = random.Random(11) + damaged = 0 + for i in range(2000): + now = i * 0.001 + decision = core.decide(1200, bool(i % 2), 5000 + (i % 7), now, rng, + remote_ip="1.2.3.4", remote_port=443, + is_syn=(i % 50 == 0), is_tcp=bool(i % 3)) + if not is_pass_through(decision, now): + damaged += 1 + check(f"{key} at its maximum ({field.bounds[1]}) damages nothing on its own", + damaged == 0, f"({damaged} of 2000 packets were touched)") + + def test_perfect_is_the_only_harmless_preset(): """Every OTHER preset must impair something. A 'bad network' preset that silently does nothing would pass every damage test while being useless.""" @@ -199,7 +270,7 @@ def test_default_core_is_pass_through_across_a_deterministic_sweep(): DAMAGE_COUNTERS = ["drop_loss", "drop_flap", "drop_lan", "drop_internet_only", "drop_block", "drop_mtu", "drop_nat", "drop_rst", "drop_syn", "drop_overflow", "drop_rate", - "corrupted", "duplicated", "rst_sent"] + "corrupted", "duplicated", "rst_sent", "loss_bursts"] def _run_until(engine, predicate, timeout=6.0):