BGP Measurements & Analysis
Large-scale network analysis processing BGP routing data to track internet routing dynamics, autonomous system growth, and prefix advertisement patterns using pybgpstream.
Overview
A data engineering project that processes massive BGP (Border Gateway Protocol) routing datasets to analyze internet routing behavior. The project tracks unique advertised prefixes, monitors autonomous system growth, and identifies the top ASes by prefix growth over time using real BGP data from global route collectors.
Technologies
Key Features
- ✓Processing of multi-gigabyte BGP routing tables
- ✓Unique prefix tracking across time snapshots
- ✓Autonomous system identification and counting
- ✓Top 10 ASes by prefix growth analysis
- ✓AS path parsing and relationship analysis
- ✓Temporal trend analysis of routing changes
Challenges
Processing massive BGP routing datasets efficiently, parsing complex AS path structures with AS-SETs, handling data from multiple global vantage points, and correlating routing changes across time.
Results
Successfully analyzed global BGP routing data to identify routing trends and AS behavior patterns, providing insights into internet routing dynamics at scale.