Concepts

Mechanics of Anomaly Scoring

Trailing Window Model

The pipeline performs anomaly scoring by looking at a visitor’s activity using a trailing window model, where activity details are measured within a configured span of time.

A window is defined by three concepts:

Length
the time span covered by the trailing window. For a window with a five minute length, each scoring decision is based on that visitor’s requests from the previous five minutes, ending at the event being evaluated.
Cadence
the shortest amount of time between two scoring decisions for the same visitor. Cadence exists because adjacent requests often describe the same behavior. Scoring every request in a burst would produce repeated decisions over nearly identical windows, so the detector emits at a controlled rhythm instead.
Minimum number of events
minimum number of requests a window must contain to be scored.

The window is anchored to the visitor’s most recent request and reaches backwards from there. As new requests arrive, old ones age out depending on the defined window length, so the window continuously reflects the visitors current behavior rather than what they did when they first showed up.

Trailing window scoring modelA timeline showing events entering and aging out of a five-minute trailing window, followed by cadence and minimum-event gates that determine whether the current event is eligible to score.The trailing window includes the most recent eventsand slides forward over time.Window length = 5 minTrailing window: last 5 minutesOlder events age out11:59aged out12:00aged out12:0112:0212:0312:0412:0512:06Current event = 12:05Window start12:00Events inside window12:01, 12:02, 12:03, 12:04, 12:05 (5 events)Cadence gateThe detector waits a fixed intervalof 60s before scoring the visitor again.12:0412:0512:06Last scoreCurrent eventeligible to scoreNext score earliestMinimum eventsScore only when enoughevents are in the window.Not enoughNot enoughMinimum events met (5 events)At 12:05, the window contains 5 events → cadence gate is open → minimum events met → score.

Features

As mentioned earlier, the trailing window collects activity details for a given visitor within a defined timespan, the system reduces that window into a series of features used to determine if user behavior can be considered anomalous given the learned common behavior of a particular source. These features describe the visitor’s activity level, timing, resource mix, HTTP behavior, and request headers.

Volume and pace
window_request_countHow many requests the visitor made inside the window.
window_span_secondsHow much time the window’s observed activity spans.
Rhythm
mean_inter_request_gap_msThe average delay between page requests.
inter_request_gap_cvHow variable the delay between page requests is.
Content mix
distinct_endpoints_hitHow many distinct endpoints the visitor touched.
html_to_asset_ratioHow much of the traffic is page navigation versus supporting assets.
Client hygiene
error_4xx_rateHow often the visitor receives client-error responses.
method_entropyHow varied the visitor’s HTTP methods are.
has_referrer_ratioHow often requests include a referrer header.
Feature extractionRecent requests in a visitor's trailing window are reduced into volume and pace, rhythm, content mix, and client hygiene features.Trailing windowNowOlderGET /homeGET /app.jsGET /productsGET /img/hero.jpgGET /checkout404 /adminRecent requests for one visitorFeatureextractionVolume and pacewindow_request_countwindow_span_secondsRhythmmean_inter_request_gap_msinter_request_gap_cvContent mixdistinct_endpoints_hithtml_to_asset_ratioClient hygieneerror_4xx_ratemethod_entropyhas_referrer_ratio

The rhythm features are computed over page requests rather than every request. This avoids treating a browser’s burst of image, script, and stylesheet loads as navigation behavior. The goal is to capture how the visitor moves through the site, not how the browser fetches assets after a page load.

These features are adapted from published research on session-level bot detection in e-commerce traffic (Rovetta et al.), which uses HTTP-level behavioral signals like request counts, timing, error rates, resource mix, referrer presence, and request methods to distinguish human from automated sessions.

HBOS

Anomaly scoring is done by comparing a visitor’s behavior against a sources baseline. A baseline is a learned model of what normal activity looks like for a given source. Each source has its own baseline, built from a collection of that source’s past visitors activity which is modeled using the trailing window model. The baseline is produced with an unsupervised algorithm called (HBOS), which builds one histogram per feature, capturing how that feature is usually distributed for the source. To score a visitors new window of activty, the system looks up each feature value in its corresponding histogram and asks: how common is a value like this, according to the baseline? A value in a densely populated bin is common, and therefore unsurprising. A value in a sparse bin is uncommon, and therefore surprising.

HBOS histogram lookupA dense bin at eight requests represents a common value. A small sparse bin at twenty requests represents an uncommon value.HBOS histogram lookupExample feature: window_request_countHistogram bins show how often each request count appearedacross past windows for a given source's baseline.Dense binFrequency in baseline8 requests05101520Request count in windowCommon valueLow surpriseSparse binFrequency in baseline20 requests05101520Request count in windowUncommon valueHigh surprise

Each feature’s likelihood is turned into a surprise term, the less likely the value, the larger the term and because HBOS treats features as independent, the observation’s total anomaly score is the sum of those surprises: one per feature, each from looking up that feature’s value in its own histogram.

An observation scores high not by being unusual on a single feature, but by accumulating surprise across the features it’s unusual on. The score rises with both the number of unusual feature values and the severity of their surprise, in which a higher score proposes anomalous behavior.

Once a raw score is computed, it is normalized into a percentile by comparing it to the source’s own baseline score distribution. A normalized score of 0.95 means the window is more unusual than about 95% of the windows used to fit that source’s baseline.

Raw score to percentile normalizationFour feature surprise terms sum to a raw score of 3.8. That score falls at the ninety-fifth percentile of the source baseline score distribution and becomes a normalized score of 0.95.Raw score → percentile normalizationPer-feature surprise terms are summed, then compared with the source baseline score distribution.window_request_count0.8surprisehtml_to_asset_ratio1.6surpriseerror_4xx_rate0.3surprisehas_referrer_ratio1.1surpriseRaw scoresum of surprises0.8+1.6+0.3+1.1=3.8Raw scorefor new windowSource baseline score distribution(from historical windows for this source)low raw scorehigh raw scorep5050th pctp7575th pctp9090th pctNew windowraw score = 3.8p9595th pct95Normalized score0.95More unusual than about95% of baseline windows.Higher raw score = more accumulated surprise across features.

This is possible because, after the feature histograms are built, the baseline worker scores the same baseline windows against those histograms. Those raw scores form a reference distribution, and the system stores its quantiles. Later, when a new window is scored, its raw score is compared against those stored quantiles so the result has a clear source-relative meaning.

As mentioned earlier, the score measures how atypical a window of activity is relative to a source’s learned baseline. It does not identify bots, prove automation, or judge intent, the system is designed to flag behavior defined as anomalous sending a downstream signal for enforcement layers to proceed accordingly.