A uniform distribution gives values within an interval equal likelihood or density. In beaconing analysis, random jitter spreads connection intervals across a range instead of producing one consistent sleep time. section 7.1
Jittered beacon intervals
An agent sleeping for 60 seconds with 20% jitter produces a uniformly random sleep-time distribution between 48 and 72 seconds. Increasing jitter makes the distribution flatter and the connection timing less predictable. section 7.1 section 7.1.5
Use the distribution to adapt beaconing detection:
- Group connection events by source IP, destination IP, and destination port, retaining groups with enough samples for analysis. section 7.1.5
- Calculate or inspect time_diff_sec rather than relying only on standard deviation across all intervals. section 7.1.5
- Restrict values to quantile ranges such as 20%–80%, 25%–75%, or 40%–60% to reduce outlier influence. section 7.1.5
- Compare the resulting distributions and standard deviations while checking that the filtered data still contains enough samples. section 7.1.5
Listing 7.24 Retrieving and processing the data
import pandas as pd
df_original = pd.read_json("ch7_stream_events.json")
print(len(df_original))
df = df_original
count_threshold = 100
df = df.groupby(['src_ip', 'dest_ip', 'dest_port']).filter \
(lambda x : len(x)>count_threshold)
df = df.reset_index()
print(len(df.index), "records with count >", count_threshold)
df['timestamp'] = pd.to_datetime(df['timestamp'], format='mixed')
df['endtime'] = pd.to_datetime(df['endtime'], format='mixed')
df['epoch_timestamp'] = df['timestamp'].astype('int64') // 10**9
df['epoch_endtime'] = df['endtime'].astype('int64') // 10**9
df = df.sort_values(by=['epoch_timestamp'], ascending=True)
df['epoch_timestamp'] = df['epoch_timestamp'].astype(int)
df['epoch_endtime'] = df['epoch_endtime'].astype(int)
df['bytes'] = df['bytes'].astype(int)
df['bytes_in'] = df['bytes_in'].astype(int)
df.dtypes
df['time_diff_sec'] = df.groupby(['src_ip', 'dest_ip', 'dest_port'])\
['epoch_timestamp'].transform(lambda x: x - x.shift(1))
df['lower_quartile'] = df.groupby(['src_ip', 'dest_ip', 'dest_port'])\
['time_diff_sec'].transform(lambda x: x.quantile(q=0.25))
df['upper_quartile'] = df.groupby(['src_ip', 'dest_ip', 'dest_port'])\
['time_diff_sec'].transform(lambda x: x.quantile(q=0.75))
df['std1'] = df.groupby(['src_ip', 'dest_ip', 'dest_port'])\
['time_diff_sec'].transform('std')
df['var1'] = df.groupby(['src_ip', 'dest_ip', 'dest_port'])\
['time_diff_sec'].transform('var')
df['count1'] = df.groupby(['src_ip', 'dest_ip', 'dest_port'])\
['time_diff_sec'].transform('count')
A pandas workflow prepares connection events by filtering frequent communication pairs, normalizing timestamps, and converting fields for analysis. section 7.1.5
