Analytics Components
Ping Proxies employs multiple complementary systems to track and analyze proxy usage:- Raw Logs (
log
): Detailed individual request data, retained for 7 days - Log Summaries (
log_summary
): Aggregated daily usage patterns, retained for 90+ days - Residential Ledger (
residential_ledger
): Residential bandwidth accounting records, retained indefinitely - Analytics Graphs: Visualizations generated from logs and summaries, with indefinite historical data access
/analytics/graph
endpoint processes data from both logs and log summaries to provide comprehensive visualizations regardless of the age of the data being analyzed.
Log Objects
Raw Logs (log
)
Raw logs represent individual proxy requests and provide the most detailed information. They are stored for 7 days.
- Tracked at the
proxy_user_id
level - Created for every single proxy request
- Contains detailed information such as:
- Client IP address
- Request size in bytes
- HTTP status/error codes
- Precise request datetime
- Authentication type
- Hostname being accessed
- Geographic information (country, city)
- ASN information
Log Summaries (log_summary
)
Log summaries aggregate raw logs into daily summaries. They provide an efficient way to analyze usage patterns without storing every individual request.
- Initially organized by
proxy_user_id
,network
, andhostname
for the first 90 days - After 90 days, further consolidated to just
proxy_user_id
andnetwork
level (hostname details are removed) - Includes metrics like:
- Total requests
- Total bytes transferred
- Success/error counts
Residential Ledger (residential_ledger
)
The residential ledger specifically tracks bandwidth usage for residential proxies, which operate on a bandwidth-based billing model rather than a per-proxy model.
- Tracks daily residential bandwidth usage at the customer account level
- Records various types of bandwidth changes:
- Usage (decrements)
- Top-ups (increments when purchasing additional bandwidth)
- Service purchases (addition of bandwidth with new residential services)
- Refunds and adjustments (manual or automatic credits)
- Provides a complete audit trail of all changes to your residential bandwidth allocation
- Critical for billing and quota management
Data Flow Process
The tracking system follows specific data flows depending on the proxy type:Datacenter and ISP Proxy Requests
- When a request is made through a datacenter or ISP proxy:
- A
log
record is created - The corresponding
log_summary
is incremented or created if it doesn’t exist
- A
Residential Proxy Requests
Residential proxies involve an additional tracking layer due to their bandwidth-based billing model:- When a request is made through a residential proxy:
- A
log
record is created - The corresponding
log_summary
is incremented or created - The customer’s
residential_ledger
usage record is incremented or created
- A
Accessing Analytics Data
Residential Ledger Endpoints
To monitor residential bandwidth usage:Analytics Graph Endpoint
For visualizing usage patterns and trends:- Automatically analyzes logs and log summary records
- Produces summaries and graphing data over specified time periods
- Supports filtering by proxy user, network, and hostname
- Offers flexible time intervals (minute, hour, day, month)
Raw Log Endpoints
For detailed troubleshooting within the 7-day retention window:Log Summary Endpoints
For aggregated historical data analysis:Effective Monitoring Strategies
For effective monitoring of your proxy usage:- Use the
/analytics/graph
endpoint for most usage monitoring scenarios - Only use
/log_summary
or/log
endpoints when you need to see information on specific requests or detailed breakdowns by hostname, proxy user ID, or network on a daily basis - The
/residential_ledger
endpoints are best used specifically for monitoring daily residential bandwidth allocation changes
Reference Table: Analytics Endpoints
Endpoint | Purpose | Retention | Granularity |
---|---|---|---|
/log/search | Search raw logs | 7 days | Individual requests |
/log/retrieve/{log_id} | Get specific raw log | 7 days | Individual request |
/log_summary/search | Search log summaries | 90+ days | Daily aggregations |
/log_summary/retrieve/{log_summary_id} | Get specific log summary | 90+ days | Daily aggregation |
/residential_ledger/search | Search residential usage records | Indefinite | Daily usage |
/residential_ledger/retrieve/{residential_ledger_id} | Get specific residential usage entry | Indefinite | Daily usage |
/analytics/graph | Visualize usage patterns | Indefinite | Configurable intervals |
/residential/summary | Get overall residential bandwidth status | Current | Account-level summary |